The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understanding

Mo, Lemao Liu, Junjie Wu, Cindy, Shunchi Zhang, Jiangnan Li, Dit-Yan Yeung, Jie Zhou

The Stochastic Parrot on LLM's Shoulder: A Summative Assessment of Physical Concept Understanding: 130 upvotes on Hugging Face Daily Papers, #1 of 21 papers on 2025-02-14. Day-by-day upvote history. It lost 64 votes when the Hub removed votes in bulk.

In a systematic way, we investigate a widely asked question: Do LLMs really understand what they say?, which relates to the more familiar term Stochastic Parrot. To this end, we propose a summative assessment over a carefully designed physical concept understanding task, PhysiCo. Our task alleviates the memorization issue via the usage of grid-format inputs that abstractly describe physical phenomena. The grids represents varying levels of understanding, from the core phenomenon, application examples to analogies to other abstract patterns in the grid world. A comprehensive study on our task demonstrates: (1) state-of-the-art LLMs, including GPT-4o, o1 and Gemini 2.0 flash thinking, lag behind humans by ~40%; (2) the stochastic parrot phenomenon is present in LLMs, as they fail on our grid task but can describe and recognize the same concepts well in natural language; (3) our task challenges the LLMs due to intrinsic difficulties rather than the unfamiliar grid format, as in-context learning and fine-tuning on same formatted data added little to their performance.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.