This week, the developer community is locked in a familiar debate: do you really need a framework to build an LLM application? The question has resurfaced with unusual force, and the answer is coming from an unexpected source. Jerry Liu, co-founder and CEO of LlamaIndex — the most widely used RAG framework in the ecosystem — told a podcast audience that the scaffolding layer is collapsing. "The need for frameworks to help users compose deterministic workflows in a lightweight way is diminishing," he said. The indexing layer, the query engine, the retrieval pipeline, the finely tuned agent loop — all of it, in his view, is being eroded from underneath.
The three forces dismantling the stack
LlamaIndex is the dominant framework for retrieval-augmented generation, the technique that connects LLMs to external data. Yet Liu openly acknowledged that the category itself is becoming less relevant. He pointed to three structural shifts. First, every new model generation improves at reasoning over unstructured data faster than humans can adapt their tooling. Second, protocols like MCP (Modern Context Protocol) and plugins such as Claude Agent Skills let models discover and use tools without per-tool integration code. Third, coding agents are replacing the act of writing code itself, reducing the developer's reliance on libraries. Liu noted that roughly 95% of LlamaIndex's own code is now AI-generated. "Engineers don't write the actual code," he said. "Everything is in natural language." The boundary between programmer and non-programmer is dissolving.
What used to require hands-on orchestration
A few years ago, building an LLM application meant reading API docs, writing integration code, and manually tuning retrieval pipelines. Today, pointing Claude Code — Anthropic's coding agent — at a file path is often sufficient. Liu recalled that "three years ago, this kind of task would have been extremely inefficient, or the agent would have just broken." Now, he said, "you can build relatively advanced retrieval with very simple primitives." The core change is in how tools are connected. Agent patterns have converged into what Liu calls a "managed agent diagram." Instead of writing custom orchestration for every workflow, developers attach tools, MCP connectors, and skill plugins to a single harness layer that wraps the agent. The bespoke middleware is gone.
What replaces the scaffolding: context extraction
If the scaffolding layer disappears, what becomes the differentiator? Liu's answer is "context." The ability for an agent to decode a file format and extract precise information becomes the critical capability. Higher accuracy and cheaper parsing — the process of analyzing documents to pull out needed data — become core competitive advantages. LlamaIndex, Liu argued, holds an edge in agentic document processing through OCR (optical character recognition, converting text in images to machine-readable text). "We've confirmed that core data is trapped inside every file format container," he said. At the same time, concerns are growing that vendors like Anthropic might lock down session data. Liu emphasized modularity and agnosticism. Developers should not bet on a single frontier model or overcomplicate their stack. "Every time a new model comes out, the winner changes," he said. "You need to preserve flexibility."
Retrieval has evolved into an "agent + sandbox" pattern. Enterprises must eliminate technical debt in their codebases and adapt to shifting patterns. Liu's advice is blunt: acknowledge that parts of the stack will eventually need to be discarded. The scaffolding was never the point. The data underneath it always was.
