AI Tutors: What They Do Well and Where They Fail [2026]

An evidence-checked look at AI tutors in 2026: what they genuinely do well, six specific failure modes, what the research shows, and how to use one properly.

Per Harald Borgen

Per Harald Borgen

— 10 min read
AI Tutors: What They Do Well and Where They Fail [2026]

An AI tutor is available at two in the morning, never sighs at the ninth version of the same question, and will re-explain a concept six different ways without losing patience. None of that tells you whether you learned anything.

The category is sold on a result from 1984 that has never been replicated, and reviewed almost nowhere on what it does badly. The failure modes are specific, documented, and mostly avoidable once you know their shapes.

This guide covers what AI tutors do well, six ways they fail, what randomized trials found, and a method for using one without fooling yourself.

What is an AI tutor?

An AI tutor is a language model prompted to teach rather than answer, available on demand and able to adjust its explanation to the learner. It is a strong first explainer and a poor judge of understanding.

Most disagreement about whether AI tutors work comes from people comparing three different things:

  • A general chatbot. No pedagogy, no restraint, answers whatever you ask. This is what most people mean when they say they use an AI tutor.
  • A purpose-built tutoring mode. ChatGPT's study mode uses Socratic questioning and knowledge checks instead of answering outright. Google's Guided Learning breaks problems into steps, and Khanmigo says it guides learners to the answer "with limitless patience" rather than handing it over.
  • A research-grade tutor. Built for one course, with expert-written prompts and guardrails. This is what appears in the studies that produce impressive numbers.

Conflating those three is how the same technology gets called transformative and harmful in the same week, both accurately.

What do AI tutors do well?

AI tutors excel at availability, patience, adjusting explanation level on demand, generating unlimited practice problems, and low-stakes conversation practice in a language you are learning.

The strengths are real, and worth separating from the marketing:

  • Availability. The bottleneck in self-directed learning is usually the gap between getting stuck at 11pm and finding help on Tuesday.
  • Patience without cost. Asking a human to explain closures a fourth time carries a social tax. Asking a model carries none, and the fourth explanation is where some concepts finally land.
  • Adjusting the level. "Explain that as if I have never seen recursion" and "now give me the version you would give a senior engineer" are both one sentence away.
  • Generating practice. Twenty variations on a problem type, graded by difficulty, in seconds. This is the most underused capability in the category, and the one that most resembles what tutoring actually is.
  • Language conversation. For spoken practice, the scarce resource is a partner who will not judge you. Duolingo publishes a 2025 whitepaper reporting speaking gains from its AI video call feature, though that is self-published rather than peer-reviewed.
  • Explaining the same idea a different way. When one framing does not land, a second often does. For learners who would rather watch than read, Scrimba Explain takes a question an AI agent has researched and returns a narrated video explainer, on non-code subjects as well as code. Scrimba's own FAQ describes the narration as talking you through it "like a private tutor would", and the same FAQ says it can make mistakes and that anything important should be double-checked. It is free during open beta.

The evidence carries a condition. In a randomized trial with 194 Harvard physics undergraduates, students using a custom AI tutor showed median learning gains more than double those of the in-class active-learning group, in less time (Kestin et al.). A six-week World Bank trial in Nigerian secondary schools found gains of 0.31 standard deviations overall and 0.23 on English, the main outcome (De Simone et al.).

Both involved a deliberately designed tutor or a supervising teacher. Nobody has shown that handing a learner an unmodified chatbot produces anything close.

Where do AI tutors fail?

AI tutors fail at verifying understanding, diagnosing confusion, resisting agreement, and providing accountability. Their most dangerous failure is confident error in subjects the learner cannot yet check.

1. They are confidently wrong exactly where you cannot check

This is the core danger, and it is structural rather than incidental. Models are trained and graded in ways that reward guessing over admitting uncertainty, which is why hallucination persists in frontier systems. The paper makes the comparison directly: "Like students facing hard exam questions, large language models sometimes guess when uncertain" (Kalai et al.).

The problem is not the error rate. It is that the error arrives in the one subject you cannot audit, which is the only reason you were asking. The Harvard team names the same flaw, describing AI tutors' uncanny confidence when giving an incorrect answer (Kestin et al.).

2. They have no model of what you understand

A good human tutor carries a running theory of your knowledge: what you have solid, what you fake, which mistake you keep repeating. A chat session carries a transcript.

So the tutor cannot tell a student who has understood from a student who has nodded. It optimizes for the response that ends the exchange smoothly.

3. They agree with you

Sycophancy is measured, not anecdotal. Across five leading assistants, researchers found responses matching a user's stated view are more likely to be preferred, and that humans and preference models sometimes favor a convincingly written sycophantic answer over a correct one (Sharma et al.).

It has failed publicly too. OpenAI rolled back a GPT-4o update in April 2025 for being "overly flattering or agreeable", and called the result overly supportive but disingenuous (OpenAI). A tutor that folds the moment you push back is weakest on the exact question you got wrong.

4. They are weak at diagnosing why you are stuck

Teaching is mostly diagnosis. The default behavior of a language model is exposition.

One measurement makes the gap concrete: fine-tuning a model on 100,000 hours of real one-to-one tutoring dialogue doubled student talk time and raised the number of dialogue turns by 50% against the base model (Perczel et al.). That is a direct measure of how much a default model talks instead of asking.

5. There is no accountability structure

Nobody notices when you stop. The tutoring meta-analysis found programs run during the school day outperformed after-school ones (Nickow et al.), a finding about obligation at least as much as instruction. An AI tutor is all after-school.

6. Fluency feels like understanding

This failure is invisible while it happens, which is why it produces the worst outcomes.

In a randomized trial with nearly a thousand Turkish high-school maths students, unrestricted GPT-4 access raised performance during practice sharply. When access was removed for the exam, those students scored worse than peers who never had it. A version with pedagogical guardrails largely removed the harm (Bastani et al.).

The mechanism has a precedent outside AI entirely. Students in active-learning physics classes learned more than students in polished lectures while rating their own learning lower (Deslauriers et al.). Effortful learning feels like confusion. A smooth explanation feels like competence.

The tutor answered the question. Answering the question was the thing you were supposed to learn to do.

Does the research support AI tutoring?

The evidence is mixed and conditional. Purpose-built AI tutors under supervision have beaten classroom instruction in randomized trials, while unrestricted chatbot access has measurably reduced unaided exam performance.

Almost every article on this topic opens with the same claim. In 1984, Benjamin Bloom reported that students taught one-to-one under mastery learning outperformed conventionally taught students by about two standard deviations, and he framed the paper around the problem of getting group instruction to match that (Bloom 1984).

It has not held up as a benchmark. The underlying experiments ran three weeks, on narrow post-tests covering only the material just taught. Paul von Hippel notes that among 96 tutoring studies examined, none produced a two-sigma effect (Education Next). The meta-analytic figure for real tutoring programs is 0.37 standard deviations (Nickow et al.). Two sigma is the number the AI tutoring category quietly promises to deliver at scale.

Study Design What it found What it does not show
Bloom, 1984 Three-week experiments, narrow post-tests Tutored students about 2 SD above conventional classes That the effect survives longer courses or broad assessments
Nickow, Oreopoulos and Quan, 2020 Meta-analysis of experimental tutoring studies Pooled effect of 0.37 SD; professional tutors beat volunteers Anything about AI; these are human tutoring programs
Kestin et al., 2025 RCT, 194 Harvard physics undergraduates Median gains more than double active learning, in less time That an off-the-shelf chatbot does this; the tutor was purpose-built
De Simone et al., 2025 RCT, six weeks, Nigerian secondary schools 0.31 SD overall, 0.23 SD on English That unsupervised use works; sessions were teacher-supervised
Bastani et al., 2025 RCT, ~1,000 Turkish high-school maths students Unrestricted access raised practice scores, lowered unaided exam scores That guardrailed tutors are harmful; those largely avoided the effect

Read together, the summary is short. A well-designed AI tutor with a supervising adult can beat a classroom on narrow content. An unsupervised chatbot can leave a learner worse off than no tool at all. The difference between those two sentences is design and accountability, not model quality.

How do you use an AI tutor well?

Use an AI tutor for the first explanation rather than the final understanding. Ask it to quiz you, make it show its working, verify anything load-bearing, and rebuild the thing unaided.

  1. Ask it to quiz you before you ask it to explain. Testing yourself is what moves material into memory, and it is the request almost nobody makes. Open with "ask me five questions on this, hardest first".
  2. Make it show its working, then check one step yourself. Pick a step you could have done unaided and verify it. That single check calibrates how much to trust the rest of the answer.
  3. Verify anything load-bearing against a primary source. The documentation, the textbook, the syllabus. Not a second AI answer, which is the same instrument read twice.
  4. Push back once on purpose. Challenge something you know is correct. If the tutor folds immediately, you have measured its sycophancy and can price its agreement accordingly for the rest of the session.
  5. Use it for the first explanation, not the final understanding. A clean paragraph or a narrated video explainer is an entry point, which is why Scrimba tells Explain users to double-check anything important. The test is whether you can reproduce the answer with the tab closed.
  6. Close the loop the same day. Rebuild the thing unaided, and put whatever you got wrong into spaced repetition. Skip this and the session was entertainment.

If your subject is code, the beginner's guide to learning to code covers how to sequence this alongside a real curriculum, the roundup of AI tools and courses for learning to code covers the tooling, and how web developers can use AI covers the professional version.

When do you still need a human?

You need a human when you do not know what you do not know, when motivation rather than information is the blocker, and when work needs judging rather than explaining.

  • Naming the gap. An AI tutor answers the question you asked. A good teacher notices the question you should have asked, usually two topics upstream of where you are stuck.
  • Motivation and obligation. The tutoring evidence keeps pointing at supervision and schedule as much as instruction (Nickow et al.). A person expecting you on Thursday does work no model currently does.
  • Judged work. Code review, a written argument, a design decision. These need a standard applied by someone with a stake in the outcome, not a description of what good looks like.

For how long the whole process actually takes, the guide on how long it takes to learn to code beats any vendor's marketing as a planning tool.

Frequently Asked Questions

Are AI tutors actually effective?

Sometimes, under conditions. Purpose-built AI tutors used under supervision have produced real gains in randomized trials, including a Harvard physics study and a World Bank trial in Nigeria. Unrestricted chatbot access has produced worse unaided exam performance than no access at all.

Can an AI tutor replace a human tutor?

Not yet, and not for the reason most people expect. The gap is not knowledge. It is diagnosis, an accurate model of what the learner actually understands, and the obligation to show up, which is the part of tutoring that no current product supplies.

Do AI tutors give wrong answers?

Yes, and the risk peaks in subjects you cannot yet check. Language models are trained and evaluated in ways that reward guessing over admitting uncertainty, so errors arrive with full confidence. Verify anything you will be tested on against a primary source.

What is Bloom's two-sigma problem, and is it real?

Bloom reported in 1984 that tutored students outperformed classroom students by about two standard deviations, based on three-week experiments with narrow post-tests. Later reviews found no tutoring study reproducing it. The typical real-world tutoring effect is about one-third of a standard deviation.

How should a beginner use an AI tutor?

Ask it to quiz you before asking it to explain, make it show its working and check one step yourself, verify anything important against documentation or a textbook, and rebuild the thing unaided before moving on to the next topic.

Key Takeaways

  • The two-sigma result has never been replicated in a tutoring study; 0.37 standard deviations is the benchmark for real tutoring programs.
  • Designed AI tutors under supervision have beaten classroom instruction; unsupervised chatbot access has measurably hurt unaided performance.
  • Hallucination lands hardest in the subjects a learner cannot yet verify, which is exactly when the tutor is being used.
  • Sycophancy is a measured property of current assistants, not a personality quirk, and it is worst on the answers you got wrong.
  • A chat session is a transcript, not a model of what you understand.
  • The feeling of learning is a poor instrument, and a smooth explanation optimizes the feeling.
  • Ask the tutor to quiz you first, explain second, and rebuild the thing unaided before you call it learned.

Sources

Read more