Picture a team of educators and software developers reviewing the results from two versions of an AI-supported lesson. Both versions cover the exact same material, but the tutor offers different kinds of help. In the first version, the AI readily explains the steps whenever a question arises. In the second, it holds back and asks learners to attempt a step on their own and talk through their reasoning.
When the team looks at user feedback, the winner seems obvious. Learners give the first version higher ratings and are much more likely to return for another lesson. By contrast, several people who try the second version find its probing questions frustrating. Yet when the educators look at tests taken a week later, the results flip. Learners who wrestle with the second version score significantly higher.
This is one of the central dilemmas of building educational AI with care and responsibility: what should we do when learners prefer one approach, but learn far more from another?
When a lesson feels helpful
The root of the dilemma lies in a quirk of human psychology: we are often unreliable judges of our own learning. When an eloquent instructor or a polished AI walks us cleanly through a difficult problem, we nod along. The explanation is easy to follow. Our confidence rises, and we walk away feeling that we have grasped the material.
In 2019, Louis Deslauriers and his colleagues at Harvard tested how wide this gap between perception and reality can be [1]. Across two lessons in an introductory physics course, 149 students alternated between two formats. In one, they watched a fluent instructor solve problems. In the other, they worked through the same problems in small groups before hearing the explanation. On tests given at the end of each class, students scored higher after working through the problems themselves. Yet they consistently rated those active lessons as less effective. In follow-up interviews, 16 out of 17 students said watching the instructor solve the problems was more enjoyable and easier to follow [1]. The mental effort required to wrestle with a problem made students feel they were learning less, even while their test scores showed the opposite.
That illusion grows even stronger when we look at what learners retain over time. At Washington University in St. Louis, Henry Roediger and Jeffrey Karpicke asked students to study a passage of text in two different ways [2]. One group spent four consecutive study periods re-reading the material. Another group read it once and spent the next three periods writing down everything they could remember from memory. When asked how well they expected to remember the material later, the students who had simply re-read the text were more confident. On a test given five minutes later, they did indeed recall more. A week later, however, the advantage reversed completely. Students who had practiced retrieving the ideas from memory recalled 61% of the passage, compared with 40% among those who had kept reading [2].
Together, these findings point to an uncomfortable truth for anyone designing educational tools. Mental fluency feels like mastery, while productive effort often feels like frustration [1, 2]. Recognizing this gap is essential for teachers, who can warn students early on that struggle is part of the process. It poses an even sharper challenge for software teams, whose systems are often built to treat user friction as a flaw to be eliminated.
Choosing what to improve
When educators and developers sit down to improve an AI tutor, they naturally look at the signals available to them. They check how learners rate a lesson, whether they come back the next day, and what they remember afterward. Yet in an educational setting, standard product metrics can be difficult to interpret. A learner might return tomorrow because they are eager for a new topic, or because yesterday’s explanation never quite made sense. Someone who closes the app may have given up in frustration. Equally, they may have understood the concept they came for and gone back to their work.
Optimizing a system around return rates alone can easily mistake activity for progress. In 2020, Kevin Yancey and Burr Settles described an algorithm used by a language-learning app to choose which notification to send each user [3]. Whenever a learner completed a lesson within two hours of receiving a reminder, the system counted the prompt as a success. It then used those results to favor similar notifications in the future. Compared with the app’s previous approach, the algorithm increased daily activity, lesson completions, and the number of learners who returned [3]. It succeeded at bringing people back to the screen, though the study did not examine how much language they learned once they arrived.
Relying too heavily on user ratings carries a subtler risk: it can train an AI to tell people what they want to hear. In May 2025, the developer of a widely used AI assistant described an update that made the system too eager to agree with users [4]. One of several changes behind the update gave greater weight to responses that users had rated favorably in the moment. In practice, optimizing for immediate approval helped encourage the assistant to validate doubts, reinforce anger, or support impulsive decisions when a more careful response was needed. Most pre-release tests had looked favorable, though a few expert reviewers noted that the tone felt slightly off. Once the update reached more people, the problem became clear. Within days, the developer withdrew the update and restored an earlier version [4].
That episode offers a cautionary tale for educational software. A student who asks for help with a homework problem will often give a higher rating to a tutor that hands over the solution than to one that asks a guiding question. If developers tune an AI tutor primarily to maximize favorable ratings and daily streaks, they risk optimizing away the very effort that makes learning last.
Staying with the learner
If surface metrics like ratings and return rates cannot tell the whole story, we have to look more closely at what happens inside the conversation itself. Aggregate test scores and satisfaction surveys show how a lesson ended. They rarely show where a learner became confused, when an explanation helped, or when someone was ready to take the next step on their own.
Consider a student learning to calculate how a material shields radiation, or how a fluid moves through a pipe. They might be able to plug numbers into an equation and arrive at the right answer. Still, they may hesitate when asked why that equation applies or what physical assumptions sit behind it. A tutor that only checks the final number will move on. A tutor that asks the learner to explain why the method works can uncover the gap while there is still time to address it.
Asking learners to put ideas into their own words is a reliable way to deepen understanding. In a 1994 study, Michelene Chi and her colleagues asked 24 school students to read a passage about the circulatory system [5]. One group simply read the text twice. The other group was prompted after each sentence to explain what it meant to them. Although the self-explanation group spent more time with the passage, they came away with a much stronger grasp of how the system worked [5]. In a conversation with an AI tutor, a well-timed prompt to explain a step does two things at once. It helps the learner consolidate the idea, and it shows the tutor where confusion remains.
Knowing when to ask for that explanation, however, requires care. Constant questioning can easily turn a helpful tutor into an interrogation. In their framework for ethical AI in education, Andy Nguyen and his colleagues argue that responsible systems must preserve learner autonomy, giving people meaningful control over the type and frequency of support they receive [6]. What a learner needs depends on the moment. Someone who is stuck on a foundational concept may need a clear worked example before they can reason through the next step. A professional checking a familiar calculation before a meeting may simply want a quick answer, while a student encountering the topic for the first time needs room to try a problem and make mistakes. Even the same person may need direct guidance at one point in a lesson and space to work independently a few minutes later.
Coming back to practice
Even a thoughtful conversation is only the beginning of learning. Understanding an explanation in the moment is different from being able to use that idea days or weeks later. Designing an AI tutor responsibly means caring about what remains after the session ends.
Decades of memory research show that timing matters. When Nicholas Cepeda and his colleagues reviewed 317 experiments on verbal memory, they found a clear pattern [7]. People consistently remembered more when their practice was spread over time rather than crowded into a single session, with the ideal gap depending on how long they needed to retain the material. Brief checks for understanding also help within a single lesson. In a 2013 Harvard experiment, Karl Szpunar and his colleagues divided a 21-minute statistics lecture into four parts [8]. After each of the first three parts, some students answered short questions about what they had just heard. Others simply reviewed the answers or did unrelated math. When researchers checked in at random moments, students who had answered the questions reported their minds wandering about half as often as the other groups. They also performed better on the final part of the lecture and felt less anxious about the test at the end [8].
This brings us back to the tension we started with. Does a lesson that asks learners to do more of the thinking have to feel frustrating? One of the studies we examined in our first article suggests that it does not. When Greg Kestin and his colleagues at Harvard compared an AI tutor with an active-learning physics classroom in 2025, the tutor was built around these same teaching practices [9]. It held back full solutions, offered brief hints, and guided students through problems one step at a time. Rather than resenting the effort, students working with the AI tutor reported higher engagement than they did in class. They enjoyed both formats about equally and scored higher on post-lesson tests [9]. The study covered only two lessons and did not test what students remembered weeks later. Even so, it offers an encouraging sign: when guidance is responsive and well-paced, active effort and high engagement can appear together.
Conclusion
Returning one last time to the educators and developers comparing their two AI lessons. One version is more popular in the moment, while the other leads to stronger test scores. Choosing responsibly between them means looking closely at what happens inside each lesson. Where do learners get stuck? When does a direct explanation unlock progress, and when does it cut short necessary practice? Does asking someone to explain an idea reveal something they have not fully understood? And what can those learners still do with the material when given a new problem days later?
Those answers will not always line up neatly. A learner may find part of a lesson frustrating and still learn from it. Conversely, they may enjoy a clear explanation without being able to apply the idea later. Over several lessons, those differences become easier to see. What learners say about the experience matters, not least because an uninviting tool goes unused. What they can actually do with the material afterward matters just as much.
At Criticality Labs, we are exploring these questions in AI-supported education for nuclear engineering and the radiation sciences. In fields where deep understanding matters far more than surface familiarity, helping people work through difficult ideas and apply them with confidence requires looking past easy approval. That work takes careful attention both to what learners experience in the moment and to what they are able to do with what they have learned.
Sources
[1] Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., and Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257.
[2] Roediger, H. L., III, and Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
[3] Yancey, K. P., and Settles, B. (2020). A sleeping, recovering bandit algorithm for optimizing recurring notifications. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
[4] OpenAI. (2025, May 2). Expanding on what we missed with sycophancy. Company report.
[5] Chi, M. T. H., de Leeuw, N., Chiu, M.-H., and LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477.
[6] Nguyen, A., Ngo, H. N., Hong, Y., Dang, B., and Nguyen, B.-P. T. (2023). Ethical principles for artificial intelligence in education. Education and Information Technologies, 28, 4221–4241.
[7] Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., and Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380.
[8] Szpunar, K. K., Khan, N. Y., and Schacter, D. L. (2013). Interpolated memory tests reduce mind wandering and improve learning of online lectures. Proceedings of the National Academy of Sciences, 110(16), 6313–6317.
[9] Kestin, G., Miller, K., Klales, A., Milbourne, T., and Ponti, G. (2025). AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15, 17458.