The modern LLM development cycle is often strangled by the GPU dance. Developers spend hours renting raw compute, wrestling with Docker daemon configurations, updating NVIDIA drivers, and manually mapping ports just to see if a specific model version performs better on a niche dataset. This infrastructure overhead creates a physical bottleneck that separates a hypothesis from a result, turning what should be a five-minute test into a half-day engineering project.

The Mechanics of Instant Inference

Hugging Face Jobs solves this friction by abstracting the entire environment setup into a single entry point: `hf jobs run`. Instead of building a custom image or managing a virtual machine, developers can deploy a vLLM server immediately using the official `vllm/vllm-openai` image. The system handles the container orchestration and hardware provisioning in the background, allowing the user to specify their hardware requirements through the `--flavor` option. These flavors represent specific GPU configurations and memory tiers, and the full list of available hardware can be retrieved using the `hf jobs hardware` command.

Once the command is issued, the system pulls the model weights and boots the container. The deployment is considered active when the logs display the Application startup complete message. To make the server accessible to the outside world, the `--expose 8000` flag routes the container's internal port 8000 through the Hugging Face Public Jobs Proxy. Upon successful execution, the system provides a unique URL that serves as the gateway to the model. Because vLLM adheres to the OpenAI API specification, this URL can be dropped directly into existing OpenAI-compatible libraries without changing the underlying application logic.

Cost efficiency is handled through a granular, per-second billing model. For instance, the `a10g-large` flavor is priced at 1.50 dollars per hour. This allows developers to optimize spend by selecting the smallest possible flavor that can fit the model's memory footprint. The complete deployment command for a standard setup looks like this:

hf jobs run --flavor a10g-large --expose 8000 vllm/vllm-openai

From Simple Hosting to Full-Stack Debugging

The real shift occurs when moving from simple hosting to an integrated development environment. Hugging Face Jobs integrates authentication and gateway management into a single platform token. Every request sent to the deployed vLLM server must include the Hugging Face token as a Bearer token in the header, ensuring that the endpoint remains gated and secure.

For a quick connectivity test, a standard curl request can be used to verify the chat completions endpoint:

bash

curl https://<job_id>--8000.hf.jobs/v1/chat/completions -H "Authorization: Bearer $(hf auth token)" -d '{"model": "...", "messages": [{"role": "user", "content": "Hello!"}]}'

In a Python environment, the integration is seamless. By setting the `base_url` of the OpenAI client to the HF Jobs URL and providing the HF token as the API key, the developer gains full access to the remote model:

client = OpenAI(base_url="https://<job_id>--8000.hf.jobs/v1", api_key=hf_token)

To verify the server health and check which models are currently loaded into VRAM, the `v1/models` endpoint provides a real-time inventory:

curl https://<job_id>--8000.hf.jobs/v1/models -H "Authorization: Bearer $(hf auth token)"

While API logs provide basic visibility, deep debugging often requires direct system access. By adding the `--ssh` flag during deployment, developers can open a shell directly inside the running container. This requires registering a public key at huggingface.co/settings/keys. Once connected, the `nvidia-smi` command becomes an essential tool for monitoring GPU memory utilization in real-time, allowing developers to spot memory leaks or process hangs that would be invisible in standard API logs. This SSH capability, supported in `huggingface_hub` version 1.20.0 and above, drastically shortens the debugging cycle for runtime errors.

hf jobs ssh <job_id>

Choosing between HF Jobs and Hugging Face Inference Endpoints depends entirely on the goal of the deployment. HF Jobs is a Docker-centric tool designed for maximum control and ephemeral tasks. It is the ideal choice for one-off experiments, batch generation on specific datasets, or pre-production evaluations where the developer needs to tweak vLLM flags manually. In contrast, Inference Endpoints is a managed service built for long-term production stability. It offers sophisticated access control (Public, Protected, Private) and a Scale-to-zero feature that automatically kills compute resources during inactivity to save costs. HF Jobs prioritizes the developer's control over the container, while Inference Endpoints prioritizes operational automation.

This flexibility becomes critical when deploying massive models like Qwen3.5-122B. Because Qwen3.5-122B utilizes a hybrid architecture combining Mamba and Attention with a massive 256K context window, it is prone to Out of Memory (OOM) errors if run with default vLLM settings. To stabilize such a model, developers must implement distributed processing via the `--tensor-parallel-size` flag, splitting the model weights across multiple GPUs. On an `h200x2` server, setting this value to 2 allows two GPUs to handle the computation in parallel, leveraging the higher memory bandwidth of the H200 hardware.

Furthermore, strict memory constraints must be enforced to prevent the KV cache from consuming all available VRAM. For Qwen3.5-122B, limiting the context length via `--max-model-len 32768` and capping the maximum number of sequences via `--max-num-seqs 256` ensures the server remains stable under load. The optimized command for a high-performance deployment is as follows:

hf jobs run --flavor h200x2 --expose 8000 vllm/vllm-openai --tensor-parallel-size 2 --max-model-len 32768 --max-num-seqs 256

For those building autonomous agents, vLLM can be extended to support tool calling. By enabling `--enable-auto-tool-choice` and specifying the correct parser—such as `hermes` for the Qwen3 family—the model can transition from generating text to issuing function calls.

hf jobs run --flavor h200x2 --expose 8000 vllm/vllm-openai --enable-auto-tool-choice --tool-call-parser hermes

This setup allows the model to be integrated into agentic frameworks like Pi. By registering the custom provider in the `~/.pi/agent/models.json` configuration file, the Pi agent can route requests to the HF Jobs endpoint. Once configured, running the agent allows it to interact with the local system, reading and writing files or executing Bash commands based on the model's tool-call requests.

pi agent run

The transition from manual infrastructure management to a single-command deployment model removes the final physical barrier to rapid AI iteration. By treating GPU clusters as ephemeral, programmable jobs rather than static servers, the focus shifts from the plumbing of LLMOps to the actual performance of the model.