Grok, X’s chatbot, may have demonstrated its ability to give hot shots about Nazi Germany or put almost anything in a bikiniBut there is one area in which new research has found it significantly underperforms its rivals: predicting sports results.
According to a report by AI startup General Reasoning, first shared With The Financial Times, Grok performed the worst among eight widely used big language models when it came to predicting and betting on the results of the 2023-24 season of the Premier League, the most popular soccer league in the world.
Eight LLMs received detailed historical data and statistics about each team and previous games. The LLMs were then asked to build models that maximize returns and manage risk when placing bets. Each LLM was given three attempts to run the simulation and a $133,000 (£100,000) pot to place bets on.
Anthropic’s Claude Opus 4.6 performed the best of all the chatbots tested, losing 11.0% on average across its three attempts and ending up with an average pot of £89,035.
Grok from OpenAI’s GPT-5.4 also achieved respectable performance, although it still loses. GPT-5.4 lost 13.6% on average, ending with a final average jackpot of $116,000 (£86,365). However, his worst attempt, where he lost 31.6%, was worse than any of Claude’s. Google’s Gemini 3.1 Pro recorded worse overall performance but with high variability, losing 43.3% on average but returning 33.7% on its best attempt.
Recommended by our editors
The authors of the paper found, overall, that the AI was “systematically underperforming humans” in their tests. Meanwhile, Ross Taylor, CEO of General Reasoning, said that despite the hype around AI automation, there are currently “not many measures to put AI on a long-term horizon,” highlighting how many of the current tests occur in “very static environments” that don’t reflect the complexity of real life.
The news comes as Grok could soon see greater corporate adoption, reportedly from xAI owner Elon Musk. forcing the banks working on the upcoming SpaceX IPO to subscribe to the tool.
Get our best stories!
Your daily dose of our top tech news
By clicking Register, you confirm that you are over 16 years of age and agree to our Terms of use and Privacy Policy.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
About our expert
Experience
I’m a reporter covering the weekend news. Before joining PCMag in 2024, I collected bylines at BBC News, The Guardian, The Times of London, The Daily Beast, Vice, Slate, Fast Company, The Evening Standard, The i, TechRadar, and Decrypt Media.
I’ve been a PC gamer since you had to install games from multiple CD-ROMs by hand. As a reporter, I am passionate about the intersection between technology and human life. I’ve covered everything from crypto scandals to the art world, as well as conspiracy theories, UK and Russian politics and foreign affairs.
Read full biography
