Skip to content

Research

Homework scores rose, exam scores fell in AI study

3 min read

Written by an AI model from the linked sources and checked against them by a second model. How we work

Three economists followed 26,811 secondary school students in one Chinese county for 30 months. They found that once students started using generative AI, their homework scores went up and their scores on tests taken without it went down. The paper, “The Generative AI Learning Penalty: Evidence from Chinese Secondary Education,” is by David Stromberg of Stockholm University and Victor Lei and Yanhui Wu of the University of Hong Kong. It came out on June 2 as CEPR discussion paper DP21577 and drew wider attention after The Economist analysed it on August 18.

After students adopted AI, their homework scores rose 18 percent and their homework took 30 percent less time. Within six months, their scores on the monthly closed-book exams fell 20 percent. Results on the high-stakes entrance exams fell 18 percent and 24 percent. That penalty took about two years to reach its full size.

The data are the county’s administrative records for grades 7 to 12 across nine subjects. They include homework marks, homework completion times and the monthly closed-book exams. They also include both entrance exams: the zhongkao for high school and the gaokao for university. Adoption was self-reported. It rose from near zero to about 80 percent over the period, and the tools named were Doubao and DeepSeek. Among AI users, average homework time fell from 64 minutes to 45.

This was not a randomised trial. Students took up AI at different times, so the authors used a staggered-adoption difference-in-differences design. Each student’s scores before and after they started using AI are compared with the scores of students who had not started yet. The method controls for a lot, but nobody assigned students to use AI. Some coverage described the study as a six-month experiment with a control group. It is an observational panel analysis.

The county has more than a million people and GDP per capita slightly below $6,000. The authors call it representative of Chinese counties outside the developed coastal belt. That still limits how far the results extend to other education systems. The paper has not been peer reviewed.

A second working paper, also not peer reviewed, points the other way. Zara Contractor and German Reyes of Middlebury College ran a randomised experiment in proctored sessions. Undergraduates learned an unfamiliar topic and wrote an essay, some with access to generative AI and some without. Students with AI access scored 0.27 standard deviations higher on an immediate test, and they kept the advantage when they repeated the assessments a week later. That paper is arXiv:2607.08849, also circulated as IZA discussion paper 18792.

In the Chinese data, the losses are not spread evenly. They concentrate among roughly 80 percent of AI users, whose behaviour the authors call homework outsourcing: they finished unusually fast and still got high homework marks. Heavier use went with bigger drops. Students who used AI zero to one hours a week saw an average score decline of 5 percent, against 30 percent for those using it five or more hours. Social sciences took the largest hit, followed by STEM and then languages. The penalty was also larger for younger students, high achievers and boys. AI users who spent about as long on assignments as non-users showed small learning losses.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.