Synthetic Data for AI: What the Collapse Papers Actually Say

You have probably seen both headlines. One says models trained on their own output degrade until they are useless. The other says this was overstated and synthetic data is fine. Both are reporting real experiments and the experiments disagree because they were not testing the same thing.
The difference is one word: replace or accumulate.
Shumailov and colleagues, writing in Nature in July 2024 (volume 631, pages 755 to 759), trained each generation of a model on the previous generation's output and watched quality degrade. Their term for it is model collapse, a degenerative process in which one model's output pollutes the next model's training set until the models misperceive reality. The mechanism is specific and worth remembering: the tails go first. Rare events are under-represented in any sample of a model's output, so each generation learns a slightly narrower world than the last, and the paper describes the resulting defects as irreversible.
Gerstgrasser and thirteen co-authors tested the same loop with one change. Instead of replacing the real data with each generation's synthetic output, they added the synthetic data to the original real data and kept both. Collapse did not happen. In their analytically tractable setup, the test error under accumulation has a finite upper bound independent of the number of iterations, and the result held across model sizes, architectures and hyperparameters, as well as for diffusion models and variational autoencoders.
So the question is not whether synthetic data is safe. The question is whether your pipeline throws the real data away. If it does, the Nature result applies to you. If it does not, the accumulation result does.

And there is a third case, which is the one most teams are actually in. You cannot grow a training set without bound, because compute is finite. A follow-up by Kazdan and colleagues tested exactly that compromise, and the answer is in between: degradation that is slow and gradual rather than explosive. Not safety, and not collapse. A slope.
Does training on synthetic data cause model collapse?
Only if the synthetic data replaces the real data. Shumailov et al. found progressive, irreversible degradation when each generation trained on the previous one's output. Gerstgrasser et al. found no collapse when synthetic data was accumulated alongside the original real data, with a test error bounded independently of iteration count. Keep the real data and the loop is stable.
That is the whole finding, and it is easy to get wrong in practice because nobody decides to throw real data away. It happens by attrition: the original dataset is expensive to store, or belongs to a team that has moved on, or was collected under a consent that has expired, and the generated version is right there and clean. Three generations later nobody can say what proportion of the training set came from a model.
Write down the ratio and keep it measurable. It is the one number in this entire subject that predicts the outcome.
The fixed-compute case, which is the one you are actually in
Accumulation has an obvious problem. If each generation adds to the set, the set grows forever, and at some point you cannot afford to train on all of it.
Kazdan and colleagues tested the honest version of this. Real and synthetic data accumulate, but each generation trains on a fixed-size subset of what has accumulated. They tested three settings: estimating a multivariate Gaussian, kernel density estimation, and fine-tuning a language model. In all three the result was slow and gradual rather than explosive degradation of test loss across generations.
Read that carefully, because it is neither good news nor bad. Capping the data per generation does not prevent degradation. It converts a cliff into a slope.
Which means the practical question is how many generations you will run. Most products are not running a recursive self-training loop at all; they fine-tune once on a mix and ship. One generation of synthetic augmentation on top of real data is not the experiment these papers are warning about. A quarterly retraining cycle that feeds on its own outputs for three years is.
A related finding from the accumulation work is worth sitting with: under accumulation the proportion of real data in the set tends towards zero as generations pile up, and the models stayed stable anyway. The real data's value is not its share. It is that it is still there, anchoring the distribution.
Ask whether you have a data problem or a labelling problem
Before any of this matters, there is a cheaper question, and it disqualifies a good proportion of synthetic-data projects.
Do you lack examples, or do you lack labels on the examples you already have? These feel similar and have nothing in common. A support team with four years of tickets has plenty of data and no labels; what it needs is a few hours of a domain expert's time and a labelling interface, not a generator. A startup three weeks from launch has neither, and generation is one of the few options available. Teams reach for synthetic data in the first situation far more often than the second, because generating feels like engineering and labelling feels like admin.
And if you lack labels, synthetic data cannot supply them honestly. A model labelling your historical records is making predictions, not observations, and training on those predictions is the recursive loop in miniature. It is sometimes a reasonable shortcut and it should be recorded as one, with a sample checked by a person.
The third version of the question is the one that actually ends projects. Do you lack outcomes? Inputs are usually recoverable from the systems you already run. Outcomes, meaning which prediction turned out right and which recommendation the customer took, are frequently never recorded anywhere, and no generator can manufacture them. That specific gap is why data readiness fails more projects than modelling does.
Where synthetic data genuinely earns its place
The research above is about training foundation models recursively. Most teams are not doing that, and the ordinary uses are better founded than the debate suggests.
Rare classes. You have 40,000 normal transactions and 60 fraudulent ones. No amount of clever modelling fixes 60 examples. Generating plausible variants of the minority class is standard practice, long predates language models, and is usually measured by whether recall on real held-out fraud improves. It either does or it does not, and you can tell.
Edge cases you cannot collect ethically or safely. The failure modes you need a system to handle are frequently the ones you cannot wait for. A triage model needs examples of the presentation that happens twice a year. A content filter needs examples of the abuse you are not going to commission.
Test fixtures that look real. Development and staging environments need data with the shape of production and none of the people in it. This is the least glamorous and most reliably valuable use, and it sidesteps most of the arguments in this post because nothing is being trained.
Load and cost rehearsal. A realistic corpus of the right size and shape lets you measure what a feature will cost and how it will behave at volume before real users generate either. Response length, retrieval depth and context size are the levers that set the bill, and modelling what inference costs at scale is far easier against a synthetic corpus than against a guess. Nothing is trained, so none of the collapse argument applies.
Bootstrapping an evaluation set before you have traffic. You cannot evaluate what has never run. Generated cases get you to a first measurement, which is better than opinion, provided you replace them as real traffic arrives. We have argued elsewhere that an evaluation set should start from fifty real traces rather than invented cases, and that remains true. Generated cases are the scaffolding, not the building.

Where it fails quietly
The failures are not dramatic. That is the problem with them.
It cannot invent a distribution it has not seen. A generator trained on your data reproduces what is in your data, including the gaps. If your historical records under-represent a customer segment, the synthetic version under-represents it too, with more confidence and a larger sample size. The bias is not removed. It is laundered.
It flatters your evaluation. Generating test cases from the same model family you are evaluating produces cases that model finds easy. The score goes up and the product does not improve. This is the single most common way a team ends up with an eval suite that says everything is fine while support tickets rise, and it is why evaluating retrieval against generated questions is worth less than it looks.
The tails go first, and the tails are the point. The Nature mechanism applies at small scale too. Whatever is rare in the source is rarer in the generated version, and rare is usually where your product fails.
And the provenance disappears. Generated records enter a dataset looking exactly like collected ones, and six months later nobody can separate them. That matters in two ways nobody budgets for. You cannot answer the ratio question the research says decides your outcome, and you cannot answer a regulator or a customer asking which of their data trained what. A single origin column, written at generation time, costs nothing and is impossible to reconstruct afterwards.
Is synthetic data good enough for an evaluation set?
As a starting point, yes. As the permanent set, no. Generated cases let you measure something before you have traffic, which beats arguing. But they are drawn from a model's idea of what your users do, and users are stranger than that. Replace them with real traces as they arrive, and never let a generated case be the last word on a release.
A discipline that makes this safe: label every case with where it came from, and report the score separately for generated and real cases. The moment those two numbers diverge, the generated half has stopped measuring your product.
There is a sharper version of the same trap. If you generate the cases with the model you are about to evaluate, or with anything in the same family, you have asked a student to set their own exam. The questions land inside what the model already handles well, the score comes back high, and nothing has been learned. Where generated cases are unavoidable, generate them with a different model from the one under test, and say so in the report.
Generating it is the easy part; judging it is not
Any model will produce as much plausible data as you ask for. The work is deciding whether it is any good, and there are only three honest checks.
The three honest checks
- Does the downstream metric move on real held-out data? This is the only test that matters and it ends most arguments. Train with and without the synthetic set, evaluate both on real data the model never saw, and compare. If the number does not move, the synthetic data did nothing, however realistic it looks.
- Does it cover what the real data does not? Compare the distribution of the generated set against the real one on the dimensions you care about. If the generator has simply produced more of what you already had, it has added volume and no information.
- Can a person who knows the domain tell the difference, and does it matter? For test fixtures, realism is enough. For training data in a regulated field, a domain expert reading a sample is a cheap control and catches the confident nonsense that statistics will not.
Each of those is work, and it is the work people skip, which is how a project ends up with 100,000 generated examples and no evidence they helped. The same discipline that applies to getting your data ready at all applies here with more force, because generated data arrives without the provenance real data carries.
The privacy argument, and its limit
Synthetic data is often proposed as the way to use sensitive records without using them, and it is a real benefit with a real boundary.
Generated records that carry no individual's actual values remove a whole category of conversation about moving data to a model vendor, which matters when existing agreements constrain where data can go. For test environments this is close to unarguable.
There is a second, quieter benefit. A generated corpus can be moved to a device, a laptop or a test rig without the approvals a production extract would need, which makes it the practical way to experiment with a model running on the user's own hardware before anyone signs off on real records leaving the building.
The limit is that synthetic does not automatically mean anonymous. A generator trained on a small dataset can reproduce individual records closely enough to be identifying, particularly for outliers, who are exactly the people most identifiable in the first place. If the privacy claim is load-bearing, it needs testing rather than asserting, and the test is whether any generated record is suspiciously close to a real one.
Can synthetic data replace the real data you do not have?
Not in the way people hope. It can amplify a weak signal, balance a class, and fill a gap you can describe. It cannot create information about a population you never observed, and the output will be fluent about it anyway.
The useful reframing is that synthetic data is a transformation of the data you already hold, not a substitute for data you do not. If you have 60 fraud cases it can help you use those 60 better. If you have none, it has nothing to work from, and the honest answer is to go and collect some. This is the same conclusion the choice between retrieval and fine-tuning keeps producing: the asset is the data, and the technique is downstream of it.
What we would do
Keep the real data, always, whatever the storage costs. That single rule is what separates the Nature result from the accumulation result, and it is cheap.
Decide the ratio deliberately and revisit it on a schedule, the way you would revisit any other dependency. A number written once in a design document and never checked is how a set drifts from mostly real to mostly generated without a decision being taken. Put it on the same quarterly review as the model version, because the two change together.
Use synthetic data where the mechanism is clear and the measurement is available: rare-class balancing checked against real held-out recall, test fixtures for environments where nothing is trained, and bootstrap evaluation cases that get retired as real traffic arrives. Label every generated record as generated, permanently, so that three model generations from now somebody can still answer the ratio question.
Be most careful where it is most tempting. A quarterly retraining loop that quietly consumes its own output is the experiment the Nature paper ran, and the fixed-compute work says a capped version of it degrades slowly rather than not at all. Slow degradation across a product's lifetime is still degradation, and like a model retirement you did not plan for, it is only a problem because nobody wrote down the date.
Frequently asked questions
Data generated by a model or a simulation rather than collected from the real world, used to train, test or evaluate another system. It ranges from simple augmentation of a minority class to full generated datasets that mirror the structure of production records without containing any real person's values. The important distinction is not how it was produced but whether it is added to real data or used in place of it.
Only when it replaces real data. Shumailov and colleagues, writing in Nature in July 2024, found progressive and irreversible degradation when each model generation trained on the previous generation's output, with rare events disappearing first. Gerstgrasser and colleagues found no collapse when synthetic data accumulated alongside the original real data, with a test error bounded independently of the number of iterations. The deciding factor is whether the original data stays in the set.
You get slow degradation rather than none. Kazdan and colleagues tested accumulation under a fixed compute budget, training each generation on a fixed-size subset of the accumulated data, across multivariate Gaussian estimation, kernel density estimation and language-model fine-tuning. The reported result was slow and gradual rather than explosive degradation of test loss across generations. Capping the data converts a cliff into a slope, which matters over many generations and rarely over one.
Not on its own, and it can make the problem harder to see. A generator trained on biased data reproduces the bias with a larger sample size and more apparent confidence, which launders it rather than removing it. Synthetic data can balance a class you can identify and describe, but it cannot supply information about a group that was never adequately represented. Fixing that requires collection, not generation.
Not automatically. A generator trained on a small dataset can reproduce individual records closely enough to identify someone, and outliers are both the most likely to be reproduced and the easiest to re-identify. If the privacy argument is doing real work in your compliance position, test it: check whether any generated record sits suspiciously close to a real one, and treat the result as evidence rather than treating the word synthetic as a guarantee.
Train with it and without it, then evaluate both on real data the model has never seen. If the metric does not move on real held-out data, the synthetic set added volume and no information, however convincing it looks. Two supporting checks: compare the generated distribution against the real one to see whether it covers anything new, and have someone with domain knowledge read a sample for confident nonsense.
Yes, as scaffolding. Generated cases get you to a first measurement, which is better than opinion, and they are the only option before traffic exists. But they come from a model's idea of what users do, and real users are stranger. Label every case with its origin, report generated and real scores separately, and retire the generated ones as traffic arrives. When the two scores diverge, the generated half has stopped measuring your product.

Vikas Choudhary
Vikas has around fifteen years of experience building software and now builds generative AI systems at Zyneto. His work covers retrieval augmented generation, agentic AI, knowledge graphs, AI memory, and the evaluation and guardrails that decide whether any of it is safe to put in front of customers. He has shipped enterprise copilots, document AI, chatbots and predictive analytics for e-commerce, fintech and marketing teams, and works day to day in Python, JavaScript and SQL. He follows multimodal models, business process automation and enterprise AI security closely, and mentors engineers moving into AI. He writes about architecture, inference cost and the failure modes that only show up at production scale.
Zyneto builds custom software, AI and automation
Tell us what you are building and we will reply with a clear next step.









