Does studying with AI help you learn?

When AI help hurt

The clearest warning comes from a field experiment with about 1,000 high-school students learning maths, published in PNAS in 2025 [1]. During practice sessions some students could use a standard GPT-4 chat assistant. Their practice scores went up by 48%. On the exam afterwards, taken without the assistant, they scored 17% lower than students who had practiced with no AI at all. And they did not notice: they did not think they had learned less.

A second, smaller study found a similar pattern on a surprise test 45 days later: 57.5% for students who had studied with ChatGPT against 68.5% for those who had studied on their own (d = 0.68) [2]. We have read only its abstract.

The mechanism is not mysterious. The research on retrieval practice shows that the effort of recalling and working something out is what makes it stick. An assistant that hands you the answer removes that effort while making the session feel more productive, the same illusion of fluency that makes rereading so popular.

When AI help worked

The same PNAS study had a third group. Their assistant was set up as a tutor: it gave hints and guided students towards the answer rather than giving it. That version largely avoided the harm [1].

In a randomized trial with Harvard physics students, an AI tutor designed around good teaching practice produced median learning gains more than twice as large as an active-learning class [3]. The tutor asked questions, gave feedback and made students do the steps themselves.

What separates the two

Across these studies the line is consistent:

• AI that answers for you raises how well you do while it is there and lowers how well you do when it isn’t.
• AI that makes you think — asks you questions, gives hints, checks your reasoning, explains where an answer went wrong — can help a lot.

The quality of AI-made study material matters too. Cards and questions are only as good as their match to your course, so it pays to keep them tied to your own material and to check them against it.

How to use AI so it helps

1. Try first, then ask. Attempt the problem or the recall before you open the chat. Use the AI to check and explain, not to start.
2. Ask for questions, not answers. “Quiz me on this lecture” and “Ask me why this happens, then tell me what I missed” put the effort back on you.
3. Ask for hints. “Don’t solve it, give me the next step only.”
4. Keep it tied to your material. An answer grounded in your own lecture, with the slide or page it comes from, is easier to check than a confident general one.
5. Check yourself without it. The test is what you can do when the chat is closed.

Where the evidence stops

These are early studies, with different students, subjects and AI systems, and the field is moving quickly. The direction of the result is the part to trust: when AI does the thinking, you learn less; when it makes you think, you can learn more.

References

1. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. doi.org/10.1073/pnas.2422633122
2. Barcaui, A. (2025). ChatGPT as a cognitive crutch: Evidence from a randomized controlled trial on knowledge retention. Social Sciences & Humanities Open, 12, 102287. (Abstract.) doi.org/10.1016/j.ssaho.2025.102287
3. Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15(1), 17458. doi.org/10.1038/s41598-025-97652-6