Where Synthetic Stops: training on expert corrections from professional work
TL;DR
Frontier models look strong on professional benchmarks, yet professionals still edit the output. Those edits (AI draft, expert override, reason, final work product) are dense training signal that mostly disappears into redlines and ticket logs. Synthetic data works where outputs can be checked automatically. In legal, finance, and similar judgment work, correctness is graded with expert rubrics, not unit tests. This note defines a correction trace, reviews what published work says about synthetic-only loops and process supervision, shows how corrections become rubrics and rewards, and proposes a matched-budget measurement protocol anyone can run. We have not run that protocol at scale yet. When we do, we will publish the results whichever way they come out.
1. The gap
Frontier models can now pass the bar, summarize a 10-K in seconds, and resolve a large share of routine support tickets. Spend a week watching professionals use them, though, and you see something the benchmarks mostly miss: the expert edits the output. A paralegal rewrites a clause that was technically correct but wrong for that client. An analyst moves a number from one quarter to another because the model misread a restatement. A support lead overrides a refund the policy allowed but the account history did not.
Those edits are the most information-dense signal produced anywhere in the AI stack, and almost all of them disappear. They live in redline histories, ticket logs, and version diffs inside the products professionals already use. Nobody trains on them.
This note argues that they should be trained on, and lays out how we would test whether they are worth it. Pillars is building toward that test: scoping real professional work into tasks with hidden graders and reference answers, then measuring whether expert corrections beat matched synthetic traces. Nothing in this note claims we have already run that experiment.
OpenAI's GDPval makes the gap concrete. The evaluation spans 44 occupations across the nine U.S. industries that contribute most to GDP. The full set has 1,320 specialized tasks (220 in the open gold set), each built from real work products by professionals with an average of about 14 years of experience, and each reviewed through multiple expert rounds (about five on average) [1]. Deliverables are not short exam answers. They are legal briefs, engineering designs, customer-support conversations, nursing care plans, spreadsheets, and slide decks. Graders are industry experts who compare model and human deliverables head to head, using occupation-specific rubrics rather than a single unit-test key.
Early GDPval results show frontier models approaching expert quality on a substantial share of one-shot deliverables. On the gold set, Claude Opus 4.1 produced outputs rated as good as or better than the human expert deliverable on just under half of tasks (wins plus ties) [1]. Performance has risen quickly across OpenAI model generations. The same paper and release note an important limit: current GDPval is one-shot. It does not capture the multi-draft loop where a lawyer revises a brief after client feedback, or an analyst iterates after spotting an anomaly [1]. That loop is exactly where correction traces live.
The default answer to "we need more domain data" in 2026 is synthetic data: prompt a strong teacher model, filter, and train. That approach has earned its place. Where outputs can be checked automatically (code that compiles, math with an answer key), synthetic pipelines and RL with verifiable rewards have driven much of the recent progress. But professional work mostly is not like that. Judging a legal memo or a credit analysis takes expert-written rubrics, not a unit test. And a teacher model cannot produce corrections for mistakes it does not know it makes.
2. What a correction trace is
A correction trace is a structured record of:
- an AI draft produced inside a live professional workflow,
- the expert's edit or override,
- the reason for the change (when captured),
- the final accepted work product.
01AI draftThe model attempt
What a model or tool produced inside a live professional workflow, before anyone corrected it. The record starts from that attempt, so it tracks a real draft rather than an idealized answer.
02Expert overrideWhat changed
The edit or override an accountable professional made. This part stores the difference between the draft and what they would accept. It is not a label written from a blank page.
03ReasonWhy it changed
The reason for the change, when the workflow captured one. It says why the expert edited the draft. A trace can omit this part.
04Final workThe accepted output
The work product the professional accepted. In the method, that accepted version is what a later grader can treat as the reference.
Read left to right. The name is the part of the record, and the line under it says what that part is. Open a step to read the method note for that category: click it, or tab to it and press Enter. The notes name categories. They are not sample traces, and the figure reports no measurement.
It is not a free-text preference pair, and it is not a dump of raw documents. It is the delta between what the model did and what an accountable professional accepted.
If your product already records these, read our note on licensing expert correction data from your AI product.
That differs from two common substitutes.
Annotators labeling from scratch. Crowd or contractor labels start from a blank or near-blank slate. They are useful, and InstructGPT-style human feedback showed how far preference data can go [2]. They are also expensive to scale into every professional niche, and they sit outside the stakes of live client work. LIMA later showed that a small, carefully curated set of high-quality demonstrations can teach style and format when the base model already holds the knowledge [3]. Correction traces sit closer to live work: the expert is fixing a real draft under real constraints.
Teacher-model synthetic data. A strong teacher can generate plausible answers, critiques, or trajectories. That works when the domain has automatic verifiers. It fails quietly when the teacher's blind spots match the student's: the imitation copies style and confidence more than capability [4]. Gudibande et al. found that imitating proprietary models with ChatGPT outputs closed little of the factual gap on tasks poorly covered in the imitation data, even when human raters liked the style [4].
A correction trace is different in three ways.
On-policy. It starts from what a model (or tool) actually produced in the workflow, so it targets real failure modes rather than idealized answers. The imitation-learning literature has made this point for more than a decade. Ross, Gordon, and Bagnell's DAgger aggregates data under the learner's own state distribution so errors do not compound the way they do under pure expert demonstrations [5]. Agarwal et al.'s on-policy distillation (GKD) extends the idea to language models: train the student on its own generations with teacher token-level feedback, reducing train-inference mismatch [6]. Thinking Machines Lab's 2025 note on on-policy distillation makes the same case for post-training: sample from the student, grade densely with a teacher, and get a denser signal than sparse RL rewards [7]. Expert corrections are a human-graded version of that on-policy loop.
Process-level. The edit shows where the reasoning went wrong, not only whether the final answer was right. Lightman et al. compared process supervision to outcome supervision on MATH and found process-supervised reward models substantially more reliable for searching over solutions (78.2% solve rate best-of-1860 on their held-out subset, versus 72.4% for the outcome-supervised baseline) [8]. They also released PRM800K, 800,000 step-level human labels. Professional edits are coarser than MATH step labels, but they play a similar role: credit assignment to the part that broke.
Grounded. The person who made the edit was accountable for the result. That does not make the data automatically clean or rights-ready. It does mean the label is tied to a real outcome, not a synthetic preference.
None of this means human data is free of problems or that synthetic data is a dead end. Correction traces are messy, unevenly distributed across tasks, and full of sensitive content. Anyone who uses them has to take rights and privacy seriously before a single example reaches a training run. The real question is narrower and testable: at a matched budget, in domains where correctness cannot be verified automatically, do expert corrections from real work buy more improvement per example than synthetic traces?
3. What the literature says about synthetic-only data
Synthetic data is not a slogan. It is a family of methods with a clear sweet spot and a clear failure mode.
Where synthetic data clearly works. Code and math have automatic checkers. Reinforcement learning with verifiable rewards, rejection sampling against unit tests, and teacher distillation on checkable trajectories have all produced large gains in those domains. That is not in dispute. Rubrics-as-rewards work (Gunjal et al.) also shows how checklist-style criteria can extend RL-style training into domains without a single binary key, which is useful context for professional work [9].
Where recursive synthetic loops degrade. Shumailov et al. showed in Nature that training generative models on recursively generated data produces "model collapse": tails of the original distribution disappear, and later generations mis-perceive reality [10]. Alemohammad et al. study related autophagous ("self-consuming") loops and describe Model Autophagy Disorder (MAD): without enough fresh real data each generation, quality (precision) or diversity (recall) degrades [11]. The practical takeaway is not "never use synthetic data." It is "do not close the loop without a fresh human source," especially when the tails (rare edge cases, unusual client facts, uncommon clause structures) are the part of the distribution that matters.
Where imitation of a stronger model is a false promise. Gudibande et al. showed that finetuning weaker open models on proprietary-model outputs improves style and instruction following far more than factual capability on unsupported tasks [4]. That result matters for professional domains: sounding like a confident memo is not the same as getting the restatement quarter right.
Where small curated human data is enough for alignment style. LIMA's result (strong behavior from 1,000 curated examples on a strong base model) supports a related point: alignment format can be cheap if pretraining already holds the knowledge [3]. Professional correction traces are aimed at a different lever. They are not primarily teaching chat style. They are teaching the last-mile edits that separate a fluent draft from an accepted work product.
The balanced reading is: synthetic data is the right default where verification is automatic; human corrections are the scarce resource where verification is expert judgment. The open question is how much that scarce resource is worth per example.
4. Corrections as rubrics and rewards
Expert edits are not only training targets. They are a way to build graders.
GDPval already grades with expert-written rubrics and pairwise expert comparison [1]. LegalBench organizes legal reasoning into issue-spotting, rule-recall, rule-application, rule-conclusion, interpretation, and rhetorical understanding, with tasks such as CUAD-derived clause classification [12]. FinanceBench tests open-book financial QA against public filings, where many failures are numerical or retrieval errors rather than style failures [13]. τ-bench tests tool-using agents against domain policy documents in retail and airline settings, where success is a database state match plus required user-facing information [14]. Across these benchmarks, the grading object is structured: a rubric, a policy checklist, a clause category, or a state assertion.
A correction trace maps onto that structure. The expert's final accepted version becomes the reference. The changed spans become rubric items. Adversarial non-solutions (shortcut answers that satisfy a shallow check) become the tests you run before a grader ships. In our planned method language: we scope real work into tasks with hidden graders and reference answers; graders are built from what the expert finally accepted; for contract-redline style tasks, graders check the specific clauses that changed; and we try to break every grader with shortcut answers before it ships.
Illustrative example (public task, not partner data)
The following is illustrative. It is built from the public LegalBench / CUAD "Audit Rights" task family, which asks whether a clause gives a party the right to audit the counterparty's books, records, or physical locations for compliance [12]. It is not drawn from any seller, partner, or private corpus.
Task. Given a contract clause, decide whether it grants audit rights, and if so draft a one-paragraph memo stating scope and limits.
Model draft (illustrative).
The clause grants Party A unrestricted audit rights over Party B's books and operations. Party A may audit at any time, for any reason, including operational systems and customer data.
Expert correction (illustrative).
Rewrite. The clause grants audit rights only over books and records, not operational systems or customer data, and only upon thirty days' written notice during normal business hours. Strike "unrestricted" and "any time, for any reason." Add that audits may not unreasonably interfere with Party B's business.
Reason (illustrative). The draft flattened a limited inspection right into a blanket operational audit. That overstates the client's leverage and misstates the counterparty's residual privacy and operational constraints.
Rubric items derived from the correction.
| ID | Criterion | Weight |
|---|---|---|
| R1 | Correctly identifies that the clause is an audit-rights clause | Essential |
| R2 | Limits scope to books and records (excludes operational systems / customer data unless stated) | Essential |
| R3 | Includes notice requirement (e.g., thirty days' written notice) | Important |
| R4 | Includes timing / non-interference constraint (normal business hours; no unreasonable interference) | Important |
| R5 | Avoids overclaim language ("unrestricted", "any time", "any reason") | Essential (pitfall) |
Reward sketch. Score a candidate memo by the weighted share of satisfied rubric items (explicit aggregation), or pass the same checklist to a judge model for an implicit scalar (as in Rubrics as Rewards) [9]. The expert's final accepted memo is the reference. Shortcut attacks (a memo that says "audit rights: yes" with no scope limits) should fail R2 through R5.
The point of the example is mechanical, not empirical. One expert edit produced five checkable criteria. Multiply that across real professional workflows and you get both training targets and evals that are harder to contaminate than public exam questions, because the underlying work is non-public and the graders stay held out of training packs.
5. A measurement protocol (an invitation, not a result)
We propose a matched-budget comparison. Anyone with access to a teacher model, a student model, and a professional-work eval suite can run a version of it. We have not run it at scale.
Protocol table
| Item | Spec |
|---|---|
| Question | At matched example budget N, do expert correction traces beat teacher synthetic traces on professional-work evals? |
| Train A | SFT or on-policy distillation on N correction traces (AI draft → expert final, optionally with reason) |
| Train B | SFT or on-policy distillation on N synthetic traces from a fixed teacher, same task prompts |
| Budget | Matched N, matched tokens, matched train steps; report wall-clock and $ cost |
| Student | Same base checkpoint for A and B |
| Teacher (for B and for any OPD variant) | Fixed; report model ID and decoding settings |
| Evals | GDPval gold (or subset), LegalBench interpretation / rule-application slices, FinanceBench sample, and a τ-bench-style policy task if tool use is in scope [1][12][13][14] |
| Primary metrics | Pairwise win / tie / loss vs. the other arm; sign test on W/L; exact match or rubric score where defined |
| Secondary metrics | Rubric-item pass rates; human expert spot-check on a fixed subsample |
| Regression panel | Hold GPQA / MMLU-Pro / a coding eval flat within noise (report 95% CIs). The point is to check that domain gains did not quietly break general capability |
| Contamination controls | Graders and held-out splits are never shipped in training data; eval sets carry canary strings; training samples come from non-public work where applicable |
| Reporting | Full config table, seeds, and raw W/T/L. Publish whichever arm wins |
Rights and privacy handling
Correction traces contain client facts, credentials-adjacent text, and privileged material. They cannot be treated like public web text.
Before any sample moves into a training or eval pack, we commit to the following method language (and only this language until counsel and systems exist to support stronger claims):
- Rights are reviewed with counsel before anything moves.
- We check terms of service and privacy policy for the whole period, customer contracts, and model-provider terms before anything moves.
- Sessions are redacted with two independent passes and a human spot-check, and the source approves every sample before anyone sees it.
- Each dataset ships with a datasheet: source period, fields, outcome coverage, redaction method and measured redaction error rate, known limits, license scope.
- Samples go to named teams under evaluation-only terms, with a per-team copy we can trace.
Residual re-identification and privilege risk remain, and the datasheet is where measured error rates belong once they exist.
What we will publish
This note is a framework and a protocol. It is not a results post. Pillars has not run the matched-budget experiment at scale. When we do, we will publish the config, the W/T/L table, the regression panel, and the outcome whichever way it comes out: corrections win, synthetic wins, or a tie within noise. An open protocol is only useful if negative results are allowed to surface.
References
- Patwardhan, T., Dias, R., Proehl, E., et al. (2025). GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. arXiv:2510.04374. Also: OpenAI. (2025, September 25). Measuring the performance of our models on real-world tasks. https://openai.com/index/gdpval
- Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155.
- Zhou, C., Liu, P., Xu, P., et al. (2023). LIMA: Less Is More for Alignment. arXiv:2305.11206.
- Gudibande, A., Wallace, E., Snell, C., et al. (2023). The False Promise of Imitating Proprietary LLMs. arXiv:2305.15717.
- Ross, S., Gordon, G. J., & Bagnell, J. A. (2011). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS 2011. arXiv:1011.0686.
- Agarwal, R., Vieillard, N., Zhou, Y., et al. (2024). On-policy Distillation of Language Models: Learning from Self-Generated Mistakes. ICLR 2024. arXiv:2306.13649.
- Lu, K., & Thinking Machines Lab. (2025, October 27). On-Policy Distillation. Thinking Machines Lab: Connectionism. https://thinkingmachines.ai/blog/on-policy-distillation
- Lightman, H., Kosaraju, V., Burda, Y., et al. (2023). Let's Verify Step by Step. arXiv:2305.20050.
- Gunjal, A., Wang, A., Lau, E., et al. (2025). Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains. arXiv:2507.17746.
- Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024). AI models collapse when trained on recursively generated data. Nature, 631, 755-759. https://doi.org/10.1038/s41586-024-07566-y
- Alemohammad, S., Casco-Rodriguez, J., Luzi, L., et al. (2023). Self-Consuming Generative Models Go MAD. arXiv:2307.01850.
- Guha, N., Nyarko, J., Ho, D. E., Ré, C., et al. (2023). LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models. arXiv:2308.11462. CUAD Audit Rights task: https://hazyresearch.stanford.edu/legalbench/tasks/cuad_audit_rights.html
- Islam, P., Kannappan, A., Kiela, D., Qian, R., Scherrer, N., & Vidgen, B. (2023). FinanceBench: A New Benchmark for Financial Question Answering. arXiv:2311.11944.
- Yao, S., Shinn, N., Razavi, P., & Narasimhan, K. (2024). τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045.