Classification Pipeline — 02

Four stages,
one classification.

Each student step moves through a fixed pipeline before it is labelled — from structured input, through deterministic checks, to model escalation only when needed.

01.

Schema Definition

Each submission is parsed into a structured schema: the mark scheme's criteria, and the student's step-by-step working, validated before any classification begins.

02.

Deterministic Resolution

A rule layer checks each step against the mark scheme's expected forms and a library of known error patterns, resolving clear cases for free with full interpretability.

03.

Escalated Classification

Steps the rule layer can't resolve confidently are batched and passed to a prompted language-model classifier, which uses the full submission to judge intent.

04.

Continuous Fine-Tuning

As labelled submissions accumulate, a smaller open model is fine-tuned on real data, moving the production path toward a faster, self-contained classifier.

§ 02—CLASSIFICATION TAXONOMY

Three outcomes, one taxonomy.

Every step of a student's working is classified into exactly one of three outcomes — a shared vocabulary that ties raw working back to mark-scheme logic.

§ 02.1—PRACTICE

Correct

The step satisfies the relevant mark-scheme criterion outright — full credit, no ambiguity.

§ 02.2—PRACTICE

Process Error

The method is right, but execution slipped — an arithmetic mistake, a sign error, a mis-substitution. Method is credited even where a mark is lost.

§ 02.3—PRACTICE

Knowledge Gap

The method itself is wrong — the step shows the student doesn't understand what this part of the question requires.

System Architecture

Engineered for learning.

F.01

Rule-First Resolution

A deterministic layer checks every step against the mark scheme's expected forms and a library of known error patterns before anything else runs — resolving clear cases for free, with full interpretability.

Explore the rule layer

RULE LAYER

LIVE
// classifier/extraction.py — rule tier
› MATCH student_step AGAINST MarkScheme criteria
› LABEL correct | process_error | knowledge_gap
// source: "rule" — zero-cost, deterministic

F.02

Model Escalation

Only steps the rule layer can't resolve confidently are escalated to a prompted language-model classifier, which uses the student's complete working to judge whether a step is a procedural slip or a genuine gap in method.

View escalation logic

ESCALATION

LIVE
// classifier/extraction.py — LLM fallback tier
› ESCALATE steps unresolved by the rule layer
› classify() falls back to classifier.baseline
// source: "llm" — full student working evaluated

F.03

Fine-Tuning Roadmap

Once enough labelled submissions exist, a smaller open model is fine-tuned on real classified data, moving the production path toward a faster, self-contained classifier.

See the fine-tuning path

FINE-TUNING

LIVE
// classifier/finetune/train_lora.py
› FORMAT gold examples -> SFT training pairs
› TRAIN open-weights model (e.g. Qwen2.5-7B) via LoRA
› GATE: ships only once it beats engine=hybrid on held-out data

Early internal testing on a small validation set showed encouraging results — presented as illustrative preliminary findings, not a verified benchmark.