Homework scores rose, exam scores fell in AI study

Three economists followed 26,811 secondary school students in one Chinese county for 30 months. They found that once students started using generative AI, their homework scores went up and their scores on tests taken without it went down. The paper, “The Generative AI Learning Penalty: Evidence from Chinese Secondary Education,” is by David Stromberg of Stockholm University and Victor Lei and Yanhui Wu of the University of Hong Kong. It came out on June 2 as CEPR discussion paper DP21577 and drew wider attention after The Economist analysed it on August 18.
After students adopted AI, their homework scores rose 18 percent and their homework took 30 percent less time. Within six months, their scores on the monthly closed-book exams fell 20 percent. Results on the high-stakes entrance exams fell 18 percent and 24 percent. That penalty took about two years to reach its full size.
The data are the county’s administrative records for grades 7 to 12 across nine subjects. They include homework marks, homework completion times and the monthly closed-book exams. They also include both entrance exams: the zhongkao for high school and the gaokao for university. Adoption was self-reported. It rose from near zero to about 80 percent over the period, and the tools named were Doubao and DeepSeek. Among AI users, average homework time fell from 64 minutes to 45.
This was not a randomised trial. Students took up AI at different times, so the authors used a staggered-adoption difference-in-differences design. Each student’s scores before and after they started using AI are compared with the scores of students who had not started yet. The method controls for a lot, but nobody assigned students to use AI. Some coverage described the study as a six-month experiment with a control group. It is an observational panel analysis.
The county has more than a million people and GDP per capita slightly below $6,000. The authors call it representative of Chinese counties outside the developed coastal belt. That still limits how far the results extend to other education systems. The paper has not been peer reviewed.
A second working paper, also not peer reviewed, points the other way. Zara Contractor and German Reyes of Middlebury College ran a randomised experiment in proctored sessions. Undergraduates learned an unfamiliar topic and wrote an essay, some with access to generative AI and some without. Students with AI access scored 0.27 standard deviations higher on an immediate test, and they kept the advantage when they repeated the assessments a week later. That paper is arXiv:2607.08849, also circulated as IZA discussion paper 18792.
In the Chinese data, the losses are not spread evenly. They concentrate among roughly 80 percent of AI users, whose behaviour the authors call homework outsourcing: they finished unusually fast and still got high homework marks. Heavier use went with bigger drops. Students who used AI zero to one hours a week saw an average score decline of 5 percent, against 30 percent for those using it five or more hours. Social sciences took the largest hit, followed by STEM and then languages. The penalty was also larger for younger students, high achievers and boys. AI users who spent about as long on assignments as non-users showed small learning losses.
Up next
Instant Cloud shuts down in 2027 as team joins OpenAI
“Existing users should migrate off of Instant Cloud within the next 12 months”: the Instant team is joining OpenAI, and every cloud app shuts down on August 31, 2027.
Related stories
- Claude learned proteins, not just papers about them
- Claude Fable 5.2 hits 69% on PCB layout test, up from 25%
- GPT-6 Astra hits 66.7% on wet-lab tasks, GPT-5.6 Sol 43.8%
- METR clocks Anthropic benchmark gains at 1.9x old pace
- GPT-6 Astra designed a Codex device in under a week
- Microsoft AI’s MAI-Base-2 trained on 16K GPUs with Muon
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

