The dream of the autonomous AI agent often begins with a simple premise: give a model a URL and a goal, and let it structure the chaos of the web into a clean database. For many developers, this feels like a solved problem. With the advent of high-context, fast-reasoning models, the industry has shifted its focus toward prompt engineering and agentic loops, assuming that the primary hurdle is the model's ability to understand unstructured text. However, when this theory meets the messy reality of public infrastructure data, the bottleneck is rarely the intelligence of the AI, but rather the existence of the data itself and the rigidity of the systems consuming it.
The Architecture of Automated Collection
To test the limits of automated data extraction, a pipeline was constructed to gather free-swimming schedules from over 600 public swimming pools across South Korea. The technical stack relied on `gemini-2.0-flash-lite` to transform raw web text into structured JSON. To provide a baseline of truth, the system integrated the KSPO OpenAPI from the Korea Sports Promotion Foundation, which provided essential metadata including pool names, geographic coordinates, and pricing tiers. The goal was to automate the most tedious part of the process: finding and parsing the actual time slots for swimming sessions.
During the initial rollout, the pipeline encountered a hard technical ceiling in the form of HTTP 429 errors. The sheer volume of requests triggered rate limits, causing the system to skip critical data points despite the implementation of standard retry logic. The solution required a shift in execution strategy, moving away from aggressive scraping toward a batched approach. By dividing the 600-pool workload into smaller units and introducing strategic delays between requests, the pipeline stabilized. While the API limits were a solvable engineering hurdle, a more fundamental problem emerged: the void of source data. A significant number of facilities either lacked official websites or failed to publish their schedules online. In these instances, the model faced a vacuum. Without a text source to ground the query, the LLM began to hallucinate, inventing plausible but entirely fictional schedules to satisfy the prompt.
The Schema Contract Paradox
Even when the model successfully extracted data from a valid website, a second, more subtle failure occurred. The data was being extracted, but it was not being reflected in the application. This discrepancy revealed a gap between extraction success and system utility. The application was designed to read data through a strict schema contract, requiring specific day codes, a precise `HH:mm` time format, and predefined price grades. If the LLM output deviated by a single character or used a slightly different time format, the application rejected the entire entry as invalid.
This realization shifted the project's core metric. The team stopped measuring success by whether the LLM could extract information and started measuring it by the schema pass rate. This led to the implementation of a defensive pipeline where the primary KPI was the schema contract validation. Instead of spending resources on prompt tuning to make the model smarter, the focus shifted to building a rigorous validation layer that acted as a gatekeeper. If the output did not perfectly match the required JSON structure, it was flagged as a failure, regardless of how accurate the information seemed to a human reader.
This pivot highlighted a critical truth about LLM ROI: the value of automation is not determined by the model's reasoning capabilities, but by the existence of the source data and the ability to verify the output against a system contract. By narrowing the scope to 72 facilities with confirmed official homepages, the number of fully completed schedules jumped from 32 to 82. This targeted approach allowed for the successful integration of 283 sessions across 54 facilities into the live service.
For the remaining 500 facilities where data was missing or malformed, the strategy evolved into a hybrid model. AI was used to generate initial drafts based on fragmented data, which were then reviewed by human administrators or supplemented by user-submitted reports. This hybrid loop acknowledged that no amount of model intelligence can replace a missing source document.
Practical LLM automation is not a test of how well a model reads unstructured text, but how reliably it adheres to a strict technical contract. The transition from measuring extraction rates to measuring schema pass rates is the only way to ensure that AI-generated data is actually usable in a production environment.
