Developers have long faced a frustrating binary when sourcing GPU power: either pay a premium for managed serverless APIs or wrestle with the operational overhead of bare-metal instances. For those choosing the latter, the inefficiency is glaring. A dedicated GPU server often sits idle during development cycles or underutilized during low-traffic periods, yet the cost remains fixed. This friction has created a gap in the market for a provider that can bridge the divide between raw infrastructure and seamless API access, particularly within the European data sovereignty landscape.

The Mechanics of the Hetzner Experiment

Hetzner is currently testing a solution to this friction through an experimental inference API designed for maximum compatibility. By adopting an OpenAI-compatible interface, the provider allows developers to migrate existing workloads without rewriting their client-side integration. The workflow is streamlined: users generate an API token via the Hetzner Experiments dashboard and simply redirect their base URL to the Hetzner servers. To lower the barrier to entry, the company has paired the release with an OpenCode connection tutorial, enabling users to test the API without writing a single line of boilerplate code.

The engine powering this experiment is the `Qwen/Qwen3.6-35B-A3B-FP8` model. This specific choice highlights a trend toward efficiency via Mixture-of-Experts (MoE) architecture. While the model possesses 35 billion total parameters, it only activates 3 billion parameters during any single inference pass. This allows the model to maintain a high level of intelligence while significantly reducing the computational cost per token. To further optimize memory, the weights are quantized to FP8 (8-bit floating point), which reduces the memory footprint without catastrophic loss in precision. The model is capable of processing both text and image inputs and supports a substantial context window of 262K tokens.

Technical testing conducted on July 23, 2026, with a single client showed rapid response times, though these results remain anecdotal as the service lacks official data on concurrency, latency under load, or a formal Service Level Agreement (SLA). In qualitative assessments, the model demonstrates strong adherence to formatting and retrieval instructions and handles image processing with accuracy. However, it exhibits the typical weaknesses of smaller-scale models, failing on a few basic arithmetic problems during testing.

The Hardware Gap and the Utilization Gamble

Beyond the API's surface functionality, the real story lies in the underlying hardware strategy. The FP8-quantized Qwen model has a file size of approximately 38GB. When accounting for KV cache and context length, this model fits comfortably within the 96GB VRAM provided by a single NVIDIA RTX 6000 Ada server. By selecting a model of this size, Hetzner has ensured that they can serve the API using their existing hardware fleet without needing to engineer complex multi-GPU clusters for a single request.

This move represents a fundamental shift in how Hetzner views its GPU assets. Traditionally, a bare-metal GPU server is a silo; one customer occupies the hardware regardless of whether the GPU is running at 10% or 100% capacity. By transitioning toward an inference API, Hetzner can implement a shared-tenant model. This allows multiple users to draw from the same GPU pool, maximizing hardware utilization rates and transforming idle silicon into a revenue-generating stream. It is a pivot from selling a box to selling a token.

However, this strategy hits a ceiling when facing the industry's largest open-weight models. For instance, a model like GLM-5, with 754 billion parameters, requires hundreds of gigabytes of VRAM even with aggressive quantization. Serving such a model requires a recipe involving at least eight GPUs linked by high-speed interconnects. To support this, a provider needs B200 or B300 class hardware and a fabric like NVLink or InfiniBand to prevent the interconnect from becoming a bottleneck. Currently, Hetzner's public lineup does not include these high-end clusters, meaning their current API offering is limited to the "small-to-medium" model class.

For the AI engineer, the current performance of the `Qwen/Qwen3.6-35B-A3B-FP8` model is a secondary detail. The primary signal is the direction of Hetzner's capital expenditure. If this experiment leads to the deployment of B200/B300 clusters and an expanded model catalog, it would position Hetzner as a formidable, low-cost alternative to the hyperscalers for European inference workloads.

If Hetzner successfully scales this shared-inference model to high-end clusters, they will effectively commoditize the European LLM hosting market.