Every AI engineer eventually hits the cost wall. The initial excitement of deploying a multi-agent system using fully managed services often evaporates the moment the first monthly cloud bill arrives. Managed models offer an intoxicating level of convenience, but when a system scales to thousands of repetitive, low-complexity tasks, the premium paid for that convenience becomes a liability. The industry is currently shifting toward a more surgical approach to model deployment, where the goal is no longer just performance, but the precise alignment of model capability with the cost of the task at hand.

The Hybrid Blueprint for Managed and Self-Hosted Intelligence

To solve the tension between operational ease and financial sustainability, a hybrid architecture utilizing Amazon Bedrock AgentCore as a unified runtime is the most viable path forward. In this design, the Bedrock AgentCore runtime acts as the single point of entry, serving as an orchestrator that determines the most efficient execution path for every incoming request. Rather than locking the entire pipeline into a single model provider, this structure allows developers to route tasks based on their complexity and cost sensitivity using the agents as tools pattern from Strands Agents.

In a practical implementation, this looks like a tiered intelligence system. For high-stakes tasks requiring deep reasoning, such as budget management or complex financial planning, the orchestrator routes the request to Amazon Bedrock's Claude Sonnet 4.6. This ensures stability and high-tier performance without the overhead of managing underlying GPU clusters. Conversely, for repetitive, data-heavy tasks like routine financial analysis, the system routes the request to a Qwen 3.5 9B model hosted on a SageMaker AI real-time endpoint. Because SageMaker AI endpoints can be exposed via OpenAI-compatible APIs, these self-hosted models integrate seamlessly into the existing pipeline without requiring extensive SDK modifications.

Deploying the Qwen 3.5 9B model requires a specific infrastructure stack to maintain performance. The recommended configuration utilizes the ml.g6e.2xlarge instance type paired with the vLLM DLC v0.22.1-gpu-py312-cu130 image. This specific image provides a pre-configured environment with CUDA 13.0 and Python 3.12, which is critical for maximizing token generation speeds through GPU acceleration. The deployment pipeline is managed through the bedrock-agentcore-starter-toolkit and the deploy_agentcore.ipynb notebook, which handles everything from loading model weights to configuring the API Gateway.

To set up the necessary environment for this hybrid communication, the following installation is required:

pip install sagemaker-core openai httpx strands-agents[otel] yfinance pydantic bedrock-agentcore

One of the primary technical hurdles in this setup is authentication persistence. SageMaker AI's OpenAI-compatible API uses Bearer tokens that expire, which can crash long-running agent sessions. To prevent this, the architecture implements an automatic renewal logic by utilizing an Auth subclass from the httpx library. This ensures that the system validates the token before every request and fetches a new one if the current token has expired, maintaining a seamless workflow across the hybrid boundary.

The Observability Gap and the Ghost of Missing Tokens

While the hybrid architecture solves the cost problem, it introduces a critical visibility problem. Bedrock AgentCore features auto-instrumentation based on OpenTelemetry, but this automation is designed with a narrow scope. It primarily recognizes generative AI tasks that are called via boto3 through Bedrock's standard inference paths. When the orchestrator routes a request to a self-hosted model on SageMaker AI, the call bypasses the standard Bedrock path, causing the request to vanish from the automated traces.

This creates a dangerous blind spot in production. The Strands OTEL integration successfully captures the agent's lifecycle spans and tool calls, but it fails to generate the gen_ai.chat span for calls made through the OpenAIModel provider. The gen_ai.chat span is the only place where input and output token counts are recorded. For the Qwen 3.5 9B model on SageMaker, the agent receives the correct answer, but the telemetry reports zero tokens used.

Without token-level observability, operational management becomes impossible. Developers cannot calculate the actual cost per request, making it impossible to track infrastructure spend against business value. More importantly, this gap prevents regression testing. If a prompt is modified to improve accuracy, there is no way to see if that change caused a spike in token consumption or a decrease in response length. The inability to separate pure model inference time from overall response latency also makes it nearly impossible to identify where the actual bottlenecks exist in a complex multi-agent chain.

This failure is rooted in the default behavior of vLLM. When vLLM is configured for streaming responses, it sends text chunks as they are generated but does not include the final usage object by default. Since Strands relies on the data returned by the model provider to track usage, the absence of this object results in an accumulated_usage value of zero. To fix this, the vLLM configuration must be explicitly updated to include the stream_options parameter:

{"include_usage": True}

Adding this option forces vLLM to send a final chunk containing the total token count after the text stream is complete. However, simply enabling this in vLLM is not enough to fix the telemetry. Because the auto-instrumentation still ignores the SageMaker path, developers must manually implement the gen_ai.chat span. By wrapping the SageMaker agent call in a manual span, the system can directly extract values from the AgentResult.metrics.accumulated_usage dictionary. By explicitly recording the inputTokens, outputTokens, and totalTokens keys, the system finally gains the quantitative data needed to calculate the exact cost of every single inference call.

This specific observability workflow has been validated using the Strands Agents framework and the vLLM DLC v0.22.1-gpu-py312-cu130 environment. It transforms the self-hosted endpoint from a black box into a transparent component of the infrastructure, allowing for precise efficiency calculations.

Strategic resource management is the final piece of the puzzle. While self-hosting on SageMaker AI reduces the per-token cost, it introduces a different financial risk: the real-time endpoint cost. Unlike Bedrock's pay-as-you-go model, SageMaker real-time endpoints charge for the instance as long as it is running, regardless of whether it is processing requests. In a testing environment, an abandoned endpoint can quickly consume a budget. A rigorous workflow must include automated scripts to delete endpoints and associated resources immediately after validation is complete.

For organizations with strict data residency requirements or those operating in highly regulated environments, the hybrid approach is not just a cost-saving measure but a necessity. It allows sensitive data to remain within a controlled SageMaker environment while leveraging the power of managed models for general reasoning. The engineering overhead of implementing manual OpenTelemetry spans is a small price to pay for the combination of infrastructure control, cost efficiency, and operational visibility.

This architecture proves that the future of AI agents is not about finding one single model to rule them all, but about building a sophisticated routing layer that treats different models as specialized tools in a larger, cost-aware ecosystem.