AI in Education Technology

A good tutor knows the answer and does not give it. Everything difficult about educational applications comes from that sentence, because a model has been trained specifically and successfully to do the opposite.

The tutor’s job conflicts with the training

Ask a model a homework question and it answers it, completely, with working. That is the behaviour it was tuned toward, and it is the behaviour users rate highly. It is also precisely what destroys the educational value of the interaction: the student who receives a worked solution has learned that the tool produces worked solutions.

Prompting alone holds this back poorly. “Do not give the answer” survives two or three turns and then collapses under a student who says they have already solved it and just wants to check, or who asks the same question in a way that does not look like the original. This is an instruction-following limit rather than a prompt-writing failure — the underlying pull is strong and it reasserts itself as the conversation lengthens. Related reading: instruction-following limits and conversation degradation.

Solve privately, tutor publicly

The architecture that holds is to separate the two jobs into two calls with different visibility.

item + student's attempt
   │
   ├─▶ SOLVER  (never shown to the student)
   │     · correct answer, with working
   │     · classify the student's error against a taxonomy for this topic
   │     · which step first went wrong
   │
   └─▶ TUTOR   (shown)
         input:  the error class and the first wrong step — NOT the answer
         policy: one hint, at the level below where they failed
         output: a question, or a smaller sub-problem

The tutor call never receives the solution, so it cannot leak it. What it receives is a diagnosis, and the diagnosis is the part that makes the hint useful: a student who wrote 3/4 + 1/2 = 4/6 has made a specific, named, extremely common error — adding numerators and denominators — and the right response addresses that error rather than restating the method. Error taxonomies for arithmetic, algebra and elementary mechanics have been studied for decades and are published; a model given the correct answer and the student’s work is good at picking the matching class from a supplied list, and poor at inventing the class.

One more component is not optional: the solver must not do arithmetic itself. Give it a calculator or a symbolic tool, because a tutoring system whose own answer is wrong will confidently correct a correct student, which is the single most damaging thing it can do. Why models get arithmetic wrong and tool calling cover the mechanics.

Grading, and the unconventional correct answer

Automated grading of free-text responses fails in a characteristic direction: it marks down correct answers that took an unexpected route. A student who solves a geometry problem by a similar-triangles argument when the rubric expected Pythagoras, or who writes a valid proof in a different order, is marked wrong by a system matching against an exemplar. That error is worse than a missed mark, because a student who is right and is told they are wrong learns to distrust their own reasoning, and because the students most likely to take an unusual route are not randomly distributed.

What helps: grade against a rubric of criteria rather than against a model answer, require the grader to cite the span of the student’s response satisfying each criterion, and give it an explicit refer_to_teacher output for responses it cannot map. A system that refers eight per cent of scripts to a human is usable; one that guesses on all of them is not. And measure agreement against human markers on double-marked scripts, reporting where the disagreements fall rather than only the rate — see human evaluation and abstention.

What would count as evidence

Education technology has a long history of products that increase engagement and change nothing about what students know, so the evaluation question deserves to be answered precisely. Session length, messages exchanged and satisfaction ratings are all improvable by making the system more willing to hand over answers, which means they are anti-metrics for a tutor.

The design that would actually settle it is unglamorous and standard: a pre-test, a period of use, and a post-test on unseen items from the same construct, against a comparison group doing an equivalent-time alternative rather than nothing at all. Two details decide whether the result means anything — the post-test items must not be items the tutor drilled, or you have measured memorisation of those items; and randomisation has to be at whatever level the intervention is delivered, usually the class, with the sample size computed accordingly. That is a real study, it takes a term, and it is the only thing that distinguishes a tutoring product from a well-designed answer key.

Building for people who are not adults

Two constraints apply here that do not apply elsewhere in this cluster, and they are architectural rather than legal fine print.

The first is that a conversational system used by children will, given enough conversations, receive disclosures of harm — self-harm, abuse, neglect. That is not a hypothetical to be handled by a refusal template. It needs a detection layer independent of the tutoring model, a defined route to a named human at the institution, and a decision made in advance about what the system says in the interval. Designing this after launch means designing it during an incident.

The second is that data about children is treated as a special category almost everywhere, that schools and districts contract on behalf of students who did not consent, and that retention, secondary use and access are correspondingly restricted. The practical effect is that the questions in provider data-use terms and zero data retention get asked by a procurement officer before a pilot, and a product that cannot answer them does not get one. None of this is legal advice; the requirements differ by jurisdiction and by age band.