Anyone who has spent significant time with a voice assistant knows the precise moment the illusion of intelligence breaks. It happens in the silence between the end of a user's sentence and the start of the AI's response. That awkward gap, often punctuated by a misplaced interruption or a delayed realization that the user has stopped talking, is the hallmark of the turn-based interaction model. For years, the industry has treated voice AI as a series of discrete packets: listen, process, speak. But as the demand for truly fluid, human-like interaction grows, the industry is realizing that the bottleneck isn't just the model's intelligence, but the very architecture of the conversation itself.
The Death of the Turn Detector and the Rise of Full-Duplex Audio
To solve the latency dilemma, OpenAI has fundamentally redesigned the audio pipeline for GPT-Live. The most significant change is the complete removal of the turn detector. In traditional voice systems, a dedicated turn detector model acts as a gatekeeper, attempting to predict exactly when a user has finished speaking. This creates a systemic trade-off: if the detector is too aggressive, it cuts the user off mid-sentence; if it is too conservative, the response feels sluggish. By eliminating this component, GPT-Live moves to a full-duplex voice model, allowing the system to send and receive audio streams simultaneously.
This shift represents a departure from the legacy cascaded architecture. In a cascaded system, the pipeline operates linearly: Speech-to-Text (STT) converts audio to text, a Large Language Model (LLM) generates a text response, and Text-to-Speech (TTS) converts that text back into audio. This serial process not only accumulates latency at every stage but also strips away critical non-verbal cues—tone, inflection, and pacing—that are lost during the text conversion. While subsequent speech-to-speech models reduced transcription loss, they remained shackled to turn-based interactions.
GPT-Live solves this by giving the voice model direct control over the conversation. Audio flows continuously into the model without the need for buffering or blocking operations. By minimizing the wait time for data accumulation, the system achieves a response latency of under one second, mirroring the natural rhythm of human speech. The model no longer waits for a signal to start; it exists in a constant state of listening and reacting.
Engineering the Fast Path: Go, WebRTC, and Asynchronous Reasoning
Achieving sub-second latency required more than just a model change; it required a total overhaul of the backend infrastructure. OpenAI replaced the media frontend and inference logic, previously built on Python asyncio, with the Go programming language. The impact of this migration was immediate and measurable. The p95 latency—the delay experienced by the slowest 5% of requests—was reduced to the level of the previous system's p50 median. Go's superior concurrency primitives allowed for a more stable and seamless delivery of audio frames in a high-pressure production environment.
To ensure that the audio stream never stutters, the architecture now employs a dual-track system. The first is the Media Fast Path, a dedicated lane where only audio data travels between the client and the voice model. The second is the asynchronous RPC (Remote Procedure Call) boundary, which handles tool calls, delegations, and complex application logic. By decoupling the media flow from the business logic, OpenAI ensures that even if a backend tool or an external API call lags, the real-time media loop remains uninterrupted.
This stability is further reinforced by the adoption of WebRTC for the transport layer. WebRTC is designed to handle the volatility of real-time internet connections, including packet loss and clock drift. When packets arrive late, WebRTC subtly stretches the audio playback to prevent audible gaps. Once the stream stabilizes, it temporarily accelerates the playback speed to catch up to the real-time flow. This prevents the jarring audio artifacts and pauses that typically plague cloud-based voice interfaces.
Beyond transport, the system manages the heavy lifting of long-term memory through stateful inference and a non-blocking context compaction mechanism. As a session grows, the context window fills, necessitating a handoff between model instances. OpenAI implements a warm-up and prefill process where a new instance is prepared in the background. The system runs inference on both the old and new instances in parallel, switching control to the new instance only when it is fully synchronized.
When the conversation exceeds the model's context limit, the system performs context compaction. Normally, this process invalidates the KV (Key-Value) cache and triggers a prefill phase that can cause delays of several hundred milliseconds—a death sentence for real-time audio. GPT-Live avoids this by offloading compaction to a separate background instance. While the primary instance maintains the live conversation, the background instance compresses the context and prepares the new state. Once ready, the system swaps the instances instantaneously, ensuring the audio stream remains continuous regardless of the computational load.
Finally, the architecture separates the act of speaking from the act of thinking. The GPT-Live model prioritizes the maintenance of the real-time media loop, while complex reasoning tasks are delegated asynchronously to frontier models like GPT-5.5. If a query requires deep analysis, the voice model can maintain the flow of conversation with fillers or brief acknowledgments while the frontier model processes the answer in the background. Once the result is ready, it is fed back into the media loop.
This separation of concerns provides the foundation for advanced features like computer control and agent orchestration in the ChatGPT desktop app. By keeping the real-time path lean and predictable, developers can customize backend tools and operational policies without risking the stability of the audio frontend. The result is a system where the intelligence of a frontier model is delivered with the immediacy of a local process.
