AI’s scientific promise becomes easier to assess when a study defines both the questions and the success criterion. New research highlighted by WIRED examines whether models can reach conclusions found in recent theoretical papers. The result is striking, but narrower than a claim that automated systems can independently conduct every part of science.
The study concerns research problems with known human answers, not a demonstration that a machine can replace the entire research process.
A defined test produces a measurable improvement
MIT FutureTech’s October 13 research summary describes 254 recent arXiv papers spanning seven domains across mathematics, physics and economics. Gabriel Manso, Claudio Mayrink Verdun and Neil Thompson compare models released between December 2024 and April 2026 using human answers unavailable to the models when answering.
The reported share of problems on which models reached the authors’ conclusion rose from 2.8 percent for o1 to 34.6 percent for GPT-5.5. The researchers report approximately $5 per attempt and $15 per success for the latter model. Those are costs within this evaluation, not a budget for completing a scientific project.
WIRED describes internet access being blocked during testing and emphasizes theory-heavy areas that do not require running simulations or physical experiments. That helps explain what the design tries to isolate: reasoning toward an answer rather than retrieving it online.
It also explains why the outcome should not be expanded into a universal productivity estimate. Reaching a recorded conclusion, selecting an important research question and establishing a new result are separate achievements. A test can offer meaningful evidence about one without measuring the others.
Scientific usefulness depends on the alternative
A separate, earlier September preprint by Manso, Emma Fu and Thompson supplies relevant context. It assembles 2,507 comparisons between AI and other analytical techniques across 27 disciplines, drawn from papers published between 2000 and early 2025.
The authors find different patterns depending on the comparator. AI often performs better than traditional statistical methods at higher computational cost. In nearly a quarter of those comparisons, it is both more expensive and worse. Against scientific-computing approaches, AI has often been cheaper but less accurate, although its relative performance has improved.
This is a different study, with a different corpus and research question. Its findings are not extra trials in the new theory benchmark, and the two sets of results should not be combined into a single success rate.
The useful connection is economic as well as technical. A method that produces an answer at low cost may change which problems are practical to attempt, but the relevant comparison remains the existing method for that task. “AI for science” describes too broad a category to establish whether a particular replacement is worthwhile.
Verification remains work rather than a footnote
In WIRED’s report, the researchers stress that expert judgment remains necessary to interpret solutions and check their reasoning. That caution belongs alongside the improvement, not after it as an optional disclaimer. An answer can look persuasive before its intermediate argument has been examined.
The new benchmark’s published success criterion also sets a boundary on interpretation. Matching a paper’s conclusion is informative, but it does not establish that every proposed derivation has the same quality or that a system will recognize its own mistakes on a new problem.
For research teams, the sensible next assessment would separate several costs: generating candidate solutions, examining their reasoning, resolving errors and deciding whether a result is useful. A small generation bill would not show that the combined process is equally inexpensive.
The evidence therefore supports a substantial capability change on a defined set of theoretical tasks. It does not settle how quickly entire disciplines will reorganize, or whether the same performance extends to work requiring experiments. The more consequential prospect is that stronger, cheaper candidate reasoning becomes available to scientists, while selection, interpretation and verification remain explicit parts of the research workflow.










