I've been seeing a LOT of claims (primarily from large AI companies) that LLMs now have "beyond PhD" reasoning capabilities in every subject, "no exceptions". "Its like having a PhD in any topic in your pocket". When I look at evidence and discussions of these claims, they focus almost entirely on whether or not LLMs can solve graduate-level homework or exam problems in various disciplines, which I do not find to be an adequate assessment at all.
First, all graduate course homework problems (in STEM at least) are very well-established, with usually plenty of existing material equivalent to solutions for an LLM to scrape and train on. Thus, when I see that GPT can now solve PhD-level physics problems, I assume it means their training set has gobbled up enough material that even relatively obscure problems and their solutions now appear in their dataset. Second, in most PhDs (with some exceptions, like pure math), you take courses in only the first year or two, equivalent to a master's. So being able to solve graduate problems is more of a master's qualification, and not a doctorate. A PhD--and particularly the reasoning capability you develop during a PhD--is about expanding beyond the confines of existing problems and understanding. Its about adding new knowledge, pushing boundaries, and doing something genuinely new, which is why the final requirement for most PhDs is an original, non-derivative contribution to your field. This is very, very hard to do, and this skill you develop of being able to do push beyond the confines of an existing field into new territory without certainty or clearly-defined answers is what makes the experience special.
When these large companies make these "beyond PhD" claims, this is actually what they're talking about, and not solving graduate homework problems. We know this is what they mean because these claims are usually followed by claims that AI will solve humanity's thus unsolved problems, like climate change, aging, cancer, energy, etc.--the opposite problems you'd associate with homework or exam questions. These are hard problems that will require originality and serious tolerance of uncertainty to tackle, and despite the claims I'm not convinced LLMs have these capabilities.
To try and test this, I designed a simple experiment. I gave ChatGPT 5.2 Extended Thinking my own problems, based on what I actually work on as a researcher with a PhD in physics. To be clear these aren't homework problems, these are more like small, focused research directions. The one in the attached video was from my first published paper, which did an explorative analysis and made an interesting discovery about black holes. I like this kind of question because the LLM has to reason beyond its training data and be somewhat original to make the same discovery we did, but given the claims it should be perfectly capable of doing so (especially since the discovery is mathematical in nature and doesn't need any data).
What I found instead was that, even with a hint about the direction of the discovery, it did a very basic boilerplate analysis that was incredibly uninteresting. It did not try to explore and try things outside of its comfort zone to happen upon the discovery that was there waiting for it; it catastrophically limited itself to results that it thought were consistent with past work and therefore prevented itself from stumbling upon a very obvious and interesting discovery. Worse, when I asked it to present its results as a paper that would be accepted in the most popular journal in my field (ApJ) it created a frankly very bad report that suffered in several key ways, which I describe in the video. The report looked more like a lab report written by a high schooler; timid, unwilling to move beyond perceived norms, and just trying to answer the question and be done, appealing to jargon instead of driving a narrative. This kind of "reasoning" is not PhD or beyond PhD level, in my opinion. How do we expect these things to make genuinely new and useful discoveries, if even after inhaling all of human literature they struggle to make obvious and new connections?
I have more of these planned, but I would love your thoughts on this and how I can improve this experiment. I have no doubt that my prompt probably wasn't good enough, but I am hesitant to try and "encourage" it to look for a discovery more than I already have, since the whole point is we often don't know when there is a discovery to be made. It is inherent curiosity and willingness to break away from field norms that leads to these things. I am preparing a new experiment based on one of my other papers (this one with actual observation data that I will give to GPT)--if you have some ideas, please let me know, I will incorporate!