Summary

  • ChatGPT failed miserably against Atari 2600 in chess due to confusion.
  • AI LLMs like ChatGPT have seen issues like hallucinations in producing incorrect information.
  • Not only did ChatGPT struggle in chess, but other AI models like OpenAI o3 also struggled in games like Pokemon Red.

A recent experiment that pit ChatGPT against an Atari 2600 in a chess match had a surprising conclusion, with ChatGPT getting "absolutely wrecked" by the old console. Despite the Atari 2600 approaching its 50th anniversary, it seems that ChatGPT couldn't figure out how to best it at the game. In fact, it hardly made it through the match at all.

ChatGPT and other AI LLMs have been touted as the future, with many users utilizing the services for everything from generating images, setting one's schedule, or cheating on homework. However, there have also been a lot of significant issues with the technology as well, with some recent LLM models seeing an increase in "hallucinations," or essentially the LLM producing information that's simply wrong.

Gollum Chat GPT
Lord of the Rings: Gollum Apology Was Written Using ChatGPT, According to Report

A new report claims the apology posted for Lord of the Rings: Gollum was written using ChatGPT without the dev team's consent.

By 

Now, an engineer named Robert Jr. Caruso has shared the results of his experiment pitting ChatGPT in a chess match against an Atari 2600. According to Caruso, he was having a conversation with ChatGPT about chess, when the AI itself suggested taking on the Atari 2600 to demonstrate its own prowess in the game. However, the process was anything but smooth. Caruso says that ChatGPT repeatedly got confused about where the pieces were, which were under its control, and repeatedly made poor decisions, like sacrificing knights to pawns. ChatGPT reportedly complained that the Atari icons were "too abstract" for it to understand, but Caruso notes that even after switching to standard chess notation, ChatGPT still made the same blunders. In the end, after 90 minutes of struggling, ChatGPT actually forfeited the match.

Another AI Struggles To Play a Game

Some might argue that ChatGPT wasn't intended for this particular task, and it's not the only AI that's had some difficulties trying to play a game recently. A ChatGPT user came up with an experiment to see if the Open AI o3 model could handle playing Pokemon Red. While it has been making progress, it's far slower in figuring out what to do compared to what a human player - and even many young kids. At the time of publication, the AI playing Pokemon Red still hadn't reached Victory Road, and had spent a whopping 366 hours, or over 15 consecutive full days, trying to get there.

With that said, there are some AI that may fare against chess or games in general better than either OpenAI's ChatGPT or o3 models. Google recently claimed that its own Google Gemini had beat Pokemon Blue, which is certainly impressive. However, it also took 800 hours to complete its goal.

Image
Atari
Atari
Display card main info widget
Date Founded
June 27, 1972
Headquarters
Sunnyvale, California, United States
Parent Company
Atari Games
Subsidiaries
NightDive Studios, Digital Eclipse, Rockstar North
Known For
Space Invaders
Display card main info widget end

Checkbox: control the expandable behavior of the extra info