The current era of generative AI is defined by a frustrating dichotomy. On one side, we have proprietary giants like GPT-4o and Gemini that seamlessly blend text, vision, and audio, but remain locked behind APIs. On the other, the open-source community has a wealth of text-centric models, yet deploying a truly massive, native multimodal system on private infrastructure remains a logistical nightmare for most engineering teams. The industry has been waiting for a model that doesn't just wrap a vision encoder around a language model, but one that treats multiple modalities as first-class citizens at a trillion-parameter scale.
The Architecture of a Trillion-Parameter Giant
Thinking Machines has stepped into this gap with the release of Inkling, a multimodal large language model (LLM) now available on Hugging Face. The scale of the model is immense, boasting a total of 975 billion parameters, effectively placing it in the 1 trillion parameter class. This capacity is backed by a massive training regimen involving 45 trillion tokens encompassing text, images, audio, and video. To make such a behemoth computationally feasible, Inkling utilizes a Mixture-of-Experts (MoE) architecture. While the total parameter count is nearly a trillion, the model only activates 41 billion parameters during any single inference pass, significantly reducing the compute overhead per token.
Beyond its raw size, Inkling is designed for extreme context handling, supporting a context window of 1 million tokens. This allows the model to ingest massive documents, long-form video, or extensive audio files without losing the thread of the conversation. The model follows a decoder-only structure and is engineered specifically for native multimodal input, meaning it does not rely on separate, disconnected modules for different data types. Instead, it is optimized for fine-tuning and domain adaptation, allowing developers to tailor its agentic capabilities to specific industrial or creative workflows. The weights are hosted on Hugging Face, enabling direct deployment on custom infrastructure.
Engineering the Multimodal Twist
What separates Inkling from previous open-weight attempts is not just the scale, but the fundamental shift in how it handles spatial and temporal data. Most modern LLMs rely on Rotary Positional Embedding (RoPE) to understand where a token sits in a sequence. Inkling rejects this in favor of Relative Attention. In this setup, each attention layer learns positional information directly from the attention logits. By introducing a fourth projection layer alongside the standard query, key, and value projections, the model generates a relative feature R for each token and head. This tensor is then adjusted based on the distance between key and query vectors, allowing the model to encode position more fluidly across its massive 1 million token window.
To manage the computational cost of such a wide window, Thinking Machines implemented a hybrid attention scheme. The model alternates between global attention, which looks at the entire context, and sliding window attention, which focuses on a fixed local neighborhood. These are deployed in a 5:1 ratio, where every five sliding window layers are punctuated by one global layer. The final layer always employs global attention to ensure the model can synthesize a rich, holistic representation of the input. This hybrid approach prevents the quadratic memory explosion typically associated with long-context models while maintaining long-range dependency.
Local representation is further offloaded from the MoE and attention modules to a specialized component called SConv (Short 1D Convolution). SConv processes the current token and the previous W-1 hidden states, where W represents the sliding window size. By delegating local pattern recognition to SConv, the MoE routing and attention mechanisms are freed to focus on high-level reasoning and cross-modal synthesis. The MoE structure itself consists of six routing experts and two shared expert sinks, the latter of which act as a stabilizer to ensure general-purpose knowledge is preserved regardless of which specialized expert is triggered.
The modality-specific pipelines are equally specialized. For vision, Inkling uses a hierarchical MLP patchifier that progressively merges pixels through linear layers to create a single embedding per patch. Audio is handled by breaking signals into 100ms chunks, converting them into mel spectrograms, and then classifying them into discretized mel bins. These bins pass through a dedicated audio embedding tower before being summed into the final input. For video, the model simply adds a temporal dimension to the image input, allowing it to track continuity and motion across frames.
Deploying a model of this magnitude requires a strategic approach to VRAM. For those with enterprise-grade clusters, the BF16 checkpoint requires 2TB of VRAM. The NVFP4 (Nvidia FP4) version reduces this requirement to 600GB. For developers with more limited hardware, Unsloth has provided a 1-bit quantized version that slashes VRAM consumption by 95% compared to the original.
Inkling is compatible with several high-performance inference engines, including transformers v5.14.0, SGLang, vLLM, and llama.cpp. When using SGLang with an 8-GPU distributed setup, the model provides an OpenAI-compatible API on port 30000. Users can configure the GPU count via the `--tp-size` argument and manage KV cache headroom using `--mem-fraction-static`. In vLLM, tensor parallelism is handled via `--tensor-parallel-size`, while the context window is capped using `--max-model-len` to prevent out-of-memory errors. A unique feature of Inkling is the `reasoning_effort` parameter, which allows users to tune the intensity of the model's internal processing across seven levels: none, minimal, low, medium, high, xhigh, and max. This is implemented via the any-to-any pipeline in the transformers library.
To move from a raw model to a functional tool, Thinking Machines provides Pi, a coding agent harness. Pi connects the model to external tools and environments. To activate it, users must add their llama.cpp server endpoint or Hugging Face Inference Provider address to the configuration file located at `~/.pi/agent/models.json`. Once configured, running the `pi` command in a project directory allows the model to execute complex mathematical reasoning and coding workflows by interacting with the local file system and external APIs.
To get started with the underlying library, users can update their environment with the following command:
pip install -U transformersThe release of Inkling signals a shift toward open-weight models that no longer compromise on modality or context length, effectively bringing the capabilities of the most advanced proprietary labs into the hands of the developer community.
