Apple sparks debate: evaluation design now key to AI reasoning claims

Apple's recent 53-page research paper has sparked intense debate by challenging the AI industry's claims about the reasoning abilities of large language models (LLMs). The study found that models like OpenAI's o3, Anthropic's Claude 3.7, and Google's Gemini struggle with even simple puzzles, with accuracy dropping below 80% on seven-disk tasks and failing on eight-disk puzzles.

The paper argues these so-called reasoning LLMs do not truly "think" or "reason" from first principles but instead operate more like advanced autocomplete engines. This has fueled widespread discussion on social media, with many interpreting the findings as a high-profile admission of AI's current limitations.

Noted AI critic Gary Marcus supported the findings, stating the paper echoes arguments he has made since 1998 about AI's overestimated reasoning capabilities.

"In many ways, the paper echoes and amplifies an argument that I have been making since 1998," Marcus wrote.

The debate highlights a growing consensus that "evaluation design is now as important as model design," emphasizing that how AI reasoning is tested is crucial to understanding its true capabilities.

Despite billions invested in developing reasoning AI, the findings suggest current approaches may be a dead end, as users often mistakenly anthropomorphize AI's abilities.

This controversy underscores the need for more rigorous and transparent evaluation methods to accurately assess AI reasoning, shaping future AI research and public expectations.

Apple researchers have ignited debate by challenging claims that top AI models like OpenAI's o3 and Google's Gemini truly reason. Their study found these models struggle with simple puzzles, sparking discussions on AI's actual capabilities and emphasizing the critical role of evaluation design in assessing AI reasoning.