Nature retracts study praising ChatGPT in education.

A study that once garnered attention for claiming AI significantly improves student learning outcomes has been officially retracted by the prestigious scientific journal Nature, raising concerns about the quality of evidence in the rapidly developing field of digital education.

The paper titled "The Impact of ChatGPT on Students' Learning Outcomes, Cognitive Development, and Thinking Skills," published in May 2025 by researchers Jin Wang and Wenxiang Fan (Hangzhou Normal University, China), was considered one of the first empirical studies demonstrating the positive impact of ChatGPT on education. The study used meta-analysis, synthesizing data from 51 studies conducted between 2022 and 2025, and concluded that ChatGPT had a moderate to high positive impact on students' learning outcomes, cognitive development, and higher-order thinking skills.

In a recent announcement, the scientific journal Nature stated that the decision to retract the article stemmed from "biases in the meta-analysis," undermining confidence in the validity of the conclusions. Notably, the authors have not responded to inquiries regarding this decision.

Immediately after its publication last May, the study spread rapidly on social media and attracted significant attention in the education community. According to Ben Williamson, a lecturer at the University of Edinburgh, the paper was considered "hard evidence" demonstrating the effectiveness of AI in learning, with hundreds of thousands of views in a short period of time.

Many scholars have already questioned the research methodology. A 2025 study by Ilkka Tuomi, a scientist at Meaning Processing Ltd., points out that meta-analysis studies in the field of educational AI often encounter problems when synthesizing data sources of uneven quality, even including works from less reputable journals. According to Tuomi, using studies conducted just weeks after ChatGPT launched is suspicious, as a serious study typically takes months to develop and at least a year to publish.

Williamson argued that the withdrawn study "aggregated a series of low-quality works and turned them into attention-grabbing claims," ​​thereby inadvertently contributing to the exaggeration of AI's role in education. Experts believe these unsubstantiated conclusions could have significant consequences if used as a basis for policy or technology investment decisions in schools.

Jake Baskin, CEO of the Computer Science Teachers Association, viewed Nature's retraction as an important warning: "We need to seriously evaluate whether generative AI truly improves teaching and learning, rather than just following a technological trend."

In fact, many teachers say that tools like ChatGPT are posing numerous challenges, from students using AI to complete assignments to difficulties in assessing real-world competence. Large language models also frequently contain errors, making their application in teaching more complex. Nevertheless, the trend of integrating AI into schools is still being strongly promoted by technology companies and policymakers.

This incident highlights a significant gap in the AI ​​research ecosystem for education: a lack of high-quality, reliable, and verifiable evidence. As Williamson pointed out, what educators need is not attention-grabbing conclusions, but "data and evidence reliable enough to guide them." In the context of AI reshaping how we teach and learn, this need is more urgent than ever.