When Homework Scores Rise but Exam Scores Fall
A careful reading of new evidence on generative AI, homework productivity, and closed-book exam performance.
The study
The authors used 30 months of panel data from 26,811 Chinese students in grades 7–12, linking homework time and scores to monthly closed-book exams and entrance exams across nine subjects. They used staggered adoption in a difference-in-differences design.
The reported divergence
AI adoption raised homework scores by 18% and reduced completion time by 30%, while monthly exam scores fell by 20% within six months. High-stakes entrance-exam scores fell by 18% and 24%, with the full estimated penalty appearing after about two years.
What homework outsourcing means here
The researchers did not observe every student prompt. They called exceptionally short completion times paired with high scores behavior consistent with homework outsourcing. Roughly 80% of AI users fit that pattern, and losses were concentrated there. This supports an interpretation; it does not prove every student outsourced work.
Limits and the Knobee connection
This working paper studied a particular non-random adoption setting. Knobee uses active recall, independent quizzes, Daily Boost, curriculum grounding, and Socratic scaffolding to focus on unaided learning rather than polished completion. The paper did not evaluate Knobee.