Every model learns from a record of the world. For text, that record is large and nearly spent. For almost everything else that matters, from the rare failure of a machine to the sound of a hall that has not been built, the record was never made. The next gains will go to whoever can produce trustworthy data where none exists, and can say exactly how far that data can be trusted.
The stock of text is finite and nearly drawn down
Epoch AI estimated the stock of quality public human text at around 300 trillion tokens, with a 90% interval of 100 to 1,000 trillion. Its central finding: on current trends, that stock will be fully used at some point between 2026 and 2032, with 80% confidence. The window has opened. We are inside it.
Text is the best-documented case, and still the record runs out. Every word people chose to write down and publish is a single, one-time endowment. Training runs grow faster than people write. The gap closes from one side only.
For the physical world, the record is thinner still
Text at least was written in abundance before anyone needed it. Data about how things behave in the physical world was mostly never written at all.
The clearest measure is robotics. The Open X-Embodiment collaboration assembled more than a million real robot trajectories across 22 robot embodiments, by pooling 60 existing datasets from 34 research labs. It is the largest open robot dataset there is, and it exists only because no single lab could gather it. A million episodes sounds like a lot until it is set against the variety of the world: every object, every surface, every lighting condition, every way a grasp can go wrong.
The same holds for rooms, garments, bodies and grids. Nobody has measured how a stadium sounds at every seat before it is poured. Nobody has a drape record for a dress that has not been cut. Nobody has a year of fifteen-minute readings for a feeder that never had a meter. For anything new, the data cannot have been collected, because the thing did not yet exist.
Feeding models their own output wears away the world
The tempting answer is to let models make more data for models. Shumailov and colleagues tested that answer and published the result in Nature in 2024. Training each generation on the unfiltered output of the last produced what they called model collapse: irreversible defects in which the tails of the original distribution disappear. It happened in large language models, in variational autoencoders and in Gaussian mixture models alike.
The tails are exactly where the value sits. The rare failure, the unusual body, the extreme load, the room configuration nobody tried: these are the cases a model most needs and least often sees. A loop that copies copies loses them first, and does not get them back.
Later work sharpens the point rather than overturning it. Kazdan and colleagues, in a paper revised in March 2025, found that collapse comes when real data is replaced, and that models stay stable when real data is kept and accumulated alongside what is generated. Real measurement anchors everything else. The question is how to make each measurement count for more.
Represent the thing itself
Livgens starts from a different place. Before anything can be simulated, it has to be said exactly what it is. Livgens represents anything real, in any matter, held as the thing itself: how a room carries sound, how a fabric falls, how a part takes a load, how light lands on a street.
The site sets out what that covers. Sound: how a room or a venue carries it, seat by seat, as impulse responses you can listen to. Light and sight: what a camera sees from where it is mounted, with the truth behind every pixel. Matter and load: parts, structures and circuit boards from FreeCAD and KiCad, with their loads and checks. Materials: how a fabric drapes and fits on a measured body, before it is cut. Places: a room, a building or a city, from a drawing, photographs or a scan, carried as OpenUSD, IFC and CityJSON. Bodies: your own measurements, and bodies fitted to any size. Populations: households, customers and whole countries, living their lives over decades.
Every thing carries its measurements and the range they hold for. That range is the guard against the failure Shumailov described. A representation that knows its own limits cannot quietly drift past them, and a simulation built on it stays inside what is known.
Run it, then hold it to the world
Representation is the first half. Living Worlds Lab takes it from there and runs it forward: one life or a billion, over a second, a season or a lifetime. What comes out is data a model can learn from and be tested against, in places where none was ever collected.
The second half is the check, and it is where generated data earns trust or loses it. Each result is registered before its data is read and scored once on measurements no model has seen. The misses are published too.
That discipline produces a specific, testable claim in place of a general promise. In one registered study, a model of power through low-voltage feeders that learned from a checked simulation first, then from a handful of real measurements, predicted 30 feeders no model had measured, over a year it had not seen, better than the same model trained on real measurements alone. With five measured feeders its normalised error was 0.712 against 0.880. With eighty, the two were level. The same model taught by a deliberately wrong simulation did worse than either. Verdict: Met.
Each part of that result matters. The gain is largest where real data is scarcest. It closes as real data grows, which is what an honest simulation should do. And a wrong simulation is punished, which is the evidence that the checked one carried real information about the world.
What this means
The data wall is real, and it is wider than text. Copying model output cannot climb it. Real measurement remains the anchor, and it will stay scarce. The work that remains is to make each measurement go further: represent the thing exactly, run it forward, and score the result against the world before anyone relies on it.
What this does not show is that any simulation, of anything, is good enough. Each one has to be checked on its own measurements, in its own range, and some will fail. The feeder study is one region and one operator. Its value is that it was registered, scored once and reported as it came out.
Where Livgens fits
Livgens represents the thing exactly, in sound, light, matter and motion, and Living Worlds runs it forward through its lifecycle: Livgens knows what a thing is, and Living Worlds knows what it will do. Livgens's applications and tools sit on one API that represents a thing, runs it and pulls the data out from your own software, under the same rules: the same checks, and the same person asked before anything real happens. Health is live today; Looper, Render, Twin and Programmer are in early access. The aim is plain: make the data the world never gave you, and show exactly how far it holds.
Sources
- Epoch AI: Will we run out of data? Limits of LLM scaling based on human-generated data (2024)
- Open X-Embodiment: Robotic Learning Datasets and RT-X Models
- Shumailov et al., AI models collapse when trained on recursively generated data, Nature 631, 755-759 (2024)
- Kazdan et al., Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World (rev. March 2025)
