A new study conducted by researchers at Harvard Business School has found that workers who use AI chat bots perform worse than their peers who do not. The study examined 760 consultants, some of whom were allowed to use chat GTP (Generative Pre-trained Transformer) and some who were not. The researchers gave them a series of tasks and found that those who used chat GTP performed 40% better in tasks such as brainstorming, writing a press release, and idea generation. However, when it came to more complex tasks like problem-solving and data analysis, those who relied on chat GTP did not perform as well. They tended to use it indiscriminately without checking their work.
The study highlights the need for human oversight when using AI chat bots. Just as a new intern’s work would be checked, the same should be done for the output of a chat bot. Overconfidence in the capabilities of AI chat bots can lead to less desirable results. The group that was most confident in using chat GTP for complex tasks ended up with the least desirable products. This overconfidence stems from blindly trusting the capabilities of a new robot.
While AI chat bots have their strengths, such as assisting real estate agents in writing listings or helping software engineers with coding, it is important to double-check their output. AI chat bots have limitations and boundaries, and it is crucial to understand where they excel and where they fall short. The study also found that AI chat bots lack diversity of ideas. When consultants asked for beverage ideas, the suggestions generated by the AI were similar, indicating a lack of variety.
It is worth noting that AI chat bots have access to a limited amount of information. The newest models may have access up to the present day, but they are still limited by the data they have been trained on. Teachers and workers should be cautious of relying solely on AI chat bots and should use other methods, such as manual grading or oral exams, to assess student work.
While AI has its place in various industries, including vehicles and aircraft, there are certain skills and experiences that only humans possess. AI cannot replace the human touch and the value of human expertise. It is important to recognize the limitations of AI and not solely rely on it for critical tasks. The study serves as a reminder to approach AI chat bots with caution and to always verify their output.
