Developers have long assumed that providing an AI agent with a Language Server Protocol (LSP) is the gold standard for code navigation. After all, LSPs provide semantic understanding, allowing an agent to jump to definitions and find references with surgical precision, much like a human developer does in VS Code. However, a recent series of experiments reveals a surprising disconnect between the theoretical power of a tool and how a Large Language Model actually utilizes it in the wild. When given the choice, the most advanced coding agents often ignore the sophisticated semantic map of the LSP in favor of the blunt, text-based search of grep.

The Performance Gap in Tool Adoption

The study focused on three primary models—Opus 4.8, Sonnet 4.6, and Haiku 4.5—tasked with navigating Python and TypeScript repositories. The goal was to evaluate how these models handled code location discovery, caller exploration, and editing tasks using either grep or an LSP. The results were stark. In simple code location tasks, the adoption rate for LSP was nearly non-existent. Opus 4.8 chose the LSP 0% of the time, while Sonnet 4.6 and Haiku 4.5 only utilized it 4% and 6% of the time, respectively.

The most telling data point emerged when the researchers forced the models to use the LSP first. In these controlled groups, the task success rate plummeted from 100% down to 89%. Even in more complex multi-file renaming operations, Opus 4.8 relied on the LSP in only 3% of its attempts. This suggests that the models do not perceive the LSP as the most efficient path to a solution, despite its ability to understand the underlying structure of the code.

However, the preference shifted when the task required reference completeness, such as finding every single caller of a function. In these scenarios, LSP adoption rates climbed to 45% for Opus 4.8, 50% for Sonnet 4.6, and 57% for Haiku 4.5. In terms of precision, the LSP was flawless, recording a score of 1.00 by effectively eliminating false positives. In contrast, grep's precision lagged at 0.76. Interestingly, the recall—the ability to find all actual instances—remained roughly equal for both tools at approximately 0.66, meaning the LSP did not necessarily find more references than grep, it just found them more accurately.

The Context Trap and the Harness Effect

The discrepancy in tool choice is not a failure of the model's reasoning, but rather a consequence of the return context. The initial LSP implementation returned only the file path, line number, and column. To actually see the code, the agent had to perform a second, separate action: reading the file. Grep, by contrast, returns the matching line of code immediately, such as `src/auth.ts:42: return validateToken(token)`. This provides the agent with immediate visual confirmation and context, collapsing two steps into one.

To test this hypothesis, the researchers modified the LSP to return the source text along with the location. The impact was immediate and dramatic. In experiments using Opus 4.8 with a pyright index, the version of the LSP that returned only locations had a first-attempt success rate of 0.67 and required an average of 15.2 subsequent file reads. Once the LSP was updated to return the code context, the success rate jumped to 0.83, and the number of subsequent reads crashed to 3.2. Furthermore, token consumption dropped from 4,131 to 3,336, proving that providing immediate context makes the agent both more accurate and more efficient.

This behavior is further explained by the concept of the harness. An agent's performance is defined as the product of the Model and the Harness (Model × Harness). The harness encompasses the entire execution environment: the system prompts, the tool input schemas, the error formats, and the loop structure. Because many models undergo post-training on trajectories involving `read`, `grep`, `edit`, and `bash`, they develop a behavioral policy heavily biased toward these interfaces. Even when a superior tool like an LSP is introduced via a prompt, the model defaults to the patterns it encountered during training.

There is also a functional advantage to grep that LSPs cannot replicate: comprehensiveness. While `find_references` in an LSP identifies actual code symbols, a refactoring task often requires updating comments, documentation strings, and configuration files. Grep captures these non-code areas, making it a more robust tool for holistic repository maintenance. This was evident in repository-specific tests. In the TypeScript library remeda, grep achieved a precision of 1.00, and using an LSP actually increased token usage by 16% without improving the F1 score. Conversely, in the hono repository, where grep's precision was low at 0.51, the LSP increased the F1 score by 0.246 and reduced token usage by 12%. In the Python requests library, the LSP raised the F1 score by 0.072 but increased token usage by 19%.

Designing effective AI agents requires moving beyond the assumption that a more precise tool is always a better tool. The priority must be the design of the output format to ensure the model can make an immediate decision without unnecessary round-trips. By structuring tool outputs as `path:line:content`, developers can align the tool's utility with the model's learned behavioral patterns, treating semantic search and text search as complementary paths rather than competitors.