AI Makes the Average Too Cheap

The rruxandra.github.io essay, "Regression to the Mean — On LLMs and the Quiet Death of the New," argues that large language models can gently pull thought toward the familiar. The page frames the promise of a collaborator on every desk against a sharper worry: a system trained on the past may reward what sounds typical while treating the strange new thing as a correction target.

My read is that the essay is directionally right and technically too clean. LLMs are not literally doomed to emit only the statistical center. Prompts, sampling, retrieval, fine-tuning, review, and workflow design all matter. But the 2024 and 2025 research context makes the warning practical: AI-assisted writing and brainstorming can improve individual outputs while narrowing collective diversity. When the average answer becomes instant and cheap, the scarce skill is deciding when to keep the awkward deviation.

Answer Snapshot

Question	My read
What happened?	A personal essay argues that LLMs can regress creative work toward the mean by returning familiar continuations and sanding down outliers.
Why it matters	Research on AI-assisted writing and brainstorming has found a similar tradeoff: individual outputs can look stronger while the collective set becomes more alike.
Who benefits if this is handled well?	Writers, researchers, engineers, product teams, and educators who want AI to widen exploration instead of standardizing the first draft.
My thesis	The practical answer is not to avoid LLMs. It is to use them in workflows that preserve evidence, disagreement, and deliberate human choice.
What I would not claim	This is not proof that every AI-assisted work is mediocre, or that novelty cannot be elicited from models. It is a warning about defaults.

The Essay Names A Real Failure Mode

The strongest part of the source is its inversion of scarcity. If everyone can ask a model for a polished answer, polish gets less valuable. The essay's phrase "guard the tail" is the part I would keep. New work often starts as something that looks wrong to the current consensus. If the assistant is always nudging the sentence, concept, naming, or architecture back toward what it has seen before, the user has to notice that pressure and sometimes reject it.

There is a technical foundation under that intuition. The GPT-3 paper describes GPT-3 as an autoregressive language model. That does not mean modern chat products simply choose the single most likely next word, and it does not erase reasoning-like behavior learned at scale. It does mean the default interaction is still built around continuations that fit a learned distribution. The model is powerful because it has absorbed so many patterns. The risk is that pattern fluency can feel like judgment.

That distinction matters because the lazy version of the critique is easy to dismiss. "LLMs predict text" is true but incomplete. "Therefore they cannot help with original work" is too broad. The more useful claim is narrower: if the user delegates taste, framing, and final judgment to the model, the work will tend to inherit the model's center of gravity.

The Evidence Is A Tradeoff, Not A Slogan

The best public context I found does not say "AI kills creativity." It says the effect can be two-sided. A Science Advances study on creative writing found that access to generative AI ideas improved how stories were rated for creativity, writing quality, and enjoyment. But the same study also found that AI-assisted stories became more similar to one another.

That is the exact tradeoff the essay is pointing at. A tool can help an individual get unstuck and still compress the collective range of outputs. In a classroom, newsroom, product team, or software organization, that matters because the group result is not just the average quality of one artifact. It is the spread of hypotheses, metaphors, designs, edge cases, and disagreements that survive long enough to be tested.

A Wharton summary of a Nature Human Behaviour brainstorming paper points in the same direction. Across five experiments, ChatGPT-assisted brainstorming produced narrower idea sets, with significant drops in 37 of 45 statistical comparisons. I would not overread one research line as destiny, but it is enough to make the warning practical instead of merely aesthetic.

Model Collapse Is Related, But Not Identical

The phrase "regression to the mean" also echoes the model-collapse literature. In Nature's 2024 paper on recursively generated data, researchers found that indiscriminate training on model-generated content can make models lose information about the true distribution, with tails disappearing first and learned behavior converging toward a low-variance point estimate.

That finding is important, but I would keep it separate from the source essay. Model collapse is about training data and recursive learning over generations. The essay is more about everyday use: people asking models for answers, feeding those answers into more work, and letting the default phrasing or framing become the next prompt. The mechanism is different. The intuition rhymes.

The practical overlap is preservation. Model builders need high-quality human and real-world data so the training distribution does not eat itself. Teams using LLMs need human judgment, primary sources, dissenting drafts, domain review, and non-AI references so their work does not become a loop of acceptable summaries.

The Pushback Is Also Correct

The common objection I find persuasive is that "the model returns the average" is not the whole product. A Hacker News thread about whether LLMs are "regression to the mean machines" shows the useful split. Some developers complain about repetitive, locally plausible output. Others point out that context, review, prompts, documentation, and feedback can make LLM-assisted work better than a raw model sample.

That pushback matters. There are real techniques for widening output. A Harvard research project on sustained creativity and diversity in LLMs argues that decoding schemes can produce more conceptually diverse results without access to the model's internal vector space. Even ordinary product choices such as asking for multiple competing hypotheses, forcing citation to primary sources, or separating generation from critique can change the shape of the output.

So I do not buy the deterministic version of the source's claim. The model is not a sealed box that can only hand back the mean. But I do buy the default-risk version. Most users do not run diversity-aware decoding schemes. Most teams do not measure idea spread. Most draft workflows reward speed, readability, and consensus long before they reward a hard-to-defend outlier.

A Better AI Workflow Protects Deviation

If I were turning the essay into an operating rule, I would make it boring and concrete. First, start with unaided notes before opening the model. Second, ask the model for alternatives that disagree with the initial frame, not just a cleaner version of it. Third, preserve the rough draft so the model's edits do not erase the original shape of the thought. Fourth, require source links or evidence for factual claims. Fifth, make a human decide which weird edges to keep.

That is especially true in technical work. An AI assistant can produce the obvious abstraction, the common error-handling pattern, the familiar API shape, or the standard product narrative. Sometimes that is exactly what you want. But if the problem is novel, the safe-looking middle may be the wrong place to stand. The review process has to ask whether the model removed useful friction.

The same principle applies to writing. I do not think every rough sentence should be preserved as an act of authenticity. Editing is good. Clarity is good. But there is a difference between tightening an argument and sanding off the reason it existed. A model can help with the first. It should not be allowed to quietly perform the second.

My Bottom Line

I like the essay because it makes the cheap thing visible. The average answer now arrives instantly, politely, and with enough polish to pass as thought. That is useful. It is also dangerous when the user forgets that plausible fluency is not the same as discovery.

The answer is not nostalgia for unaided work. The answer is better taste under automation pressure. Use the model to compress background, generate candidates, surface counterarguments, and stress-test a draft. Then make a deliberate choice about the tail. The new thing will often look awkward before it looks right. If the model keeps correcting it, that may be the first sign worth paying attention to.