LLMs & Information Decay

LLMs have become a powerful productivity tool, ones that are best utilized by those who already understand the subject matter well enough to validate their outputs and course-correct mistakes. However, there’s a problem rearing its head when LLM-generated text is being treated an authoritative figure by those who are unable to validate its accuracy. In these cases, plausible-sounding inaccuracies don’t get caught; they get repeated, copied, forwarded, and spread as canonical unless someone takes the time to validate their authenticity.I’ve started thinking of this as ‘Information Decay’ - the gradual loss of fidelity to the original source of truth as information passes through LLM-generated summaries, rewrites, and ‘helpful’ LLM-created additions. The content looks polished, but it drifts further from the source of reality with each iteration, however it feels correct at each step of the chain. Two recent examples from my peers have stuck with me.<h2 id="gpt-writing-requirements">GPT writing requirements</h2> A business analyst sent them, the team lead, an email containing explicit technical requirements. The requirements were odd, they were using non-standard endpoints, naming conventions inconsistent with their internal standards, and would take more effort than implementing the same feature using the standard workflow. The team lead assumed they were particularities that the client required and began implementing them as written. After more than a week of work from the team, the lead discovered that the ‘requirements’ did not come from the client. The analyst had taken the client’s high-level requests, fed them into a GPT, then forwarded the output as if it were an explicit specification from the client. Because the analyst didn’t understand the stack well enough to spot issues, and because the output looked as though it had been defined by an engineer, the engineer team treated them as authoritative. The result was that the week of work from that team was a total waste and a week delay in delivering to the client.<h2 id="llms-talking-to-llms">LLMs talking to LLMs</h2> A friend of mine working at separate company noticed a pattern of LLM-written emails being skimmed, then parsed by another LLM, then replied to with another LLM. At a small scale, this wasn’t catastrophic. The issue was the subtle drift.If you’ve ever asked an LLM to write against an inconsistently named API, you’ve seen this behavior: Given endpoints like<ul><li><code>writeX(var)</code>, <code>readX(var)</code>, <code>updateX(var)</code></li><li><code>writeZ(var)</code>, <code>readZ(var)</code> </li></ul>The model will assume that <code>updateZ(var)</code> exists because it “should” - fundamentally they are pattern matching machines, and ,truthfully, they’re extremely good at it. However, these same kind of assumptions were creeping within internal communication. Minor inaccuracies weren’t caught because the email looked clean, and people weren’t proofreading the generated text as carefully as they would had they manually written those messages.The inaccuracies weren’t always huge, but each pass had the potential to harden them. What started as a small plausible assumption became a fact and later shows up in a follow-on discussion and implementation details.<h2 id="information-decay">Information Decay</h2>This is Information Decay in practice, minor but plausible inaccuracies creeping in as verified fact.None of this is new in spirit, miscommunications and poorly captured requirements have always existed. The difference is throughput and credibility. LLMs make it easy to generate authoritative sounding information at scale, and that increases both the volume of unverified text and the probability that something wrong slips into the system. In theory, this is an easy fix, just tell people to check their GPT-generated output with the same level of detail as they would their own work, but we’re humans, we aren’t going to do that. We’re a species that trends towards convenience even if the convenience of the solution is harmful long term. All of this has me questioning the nature of communication within companies. AI/LLMs are here to stay, meaning this problem is something that will continue to happen, and as we have generations raised with these tools they will only expand and become more pronounced. Markets will be made on the solutions to these sorts of problems; right now we're using tools that are designed for human-to-human communication and throwing an LLM in the middle, will the solution be something that's designed to have an LLM in the middle of our communication layers? Absurdist to think of right now and definitely something that's much further out, plus, we'll have to go through our LLM-flavored dotcom crash first, then we're back in the wild west.The future will surely be fascinating.