Inkling: The trillion‑parameter multimodal model that…

Discover Inkling, a 975B‑parameter Mixture‑of‑Experts model with native image, text, and audio input, 1M token context, and day‑zero support for transformers, SGLang, vLLM, and llama.cpp.

🚀 Meet Inkling: the trillion‑parameter model that sees, hears, and reads all at once.

⚡ TL;DR: Inkling is a 975B‑parameter Mixture‑of‑Experts model with native image, text, and audio input, a 1M token context, and day‑zero support for transformers, SGLang, vLLM, and llama.cpp

The Simple Version

Inkling packs image, text, and audio understanding into one giant brain.

🎯 Think of it this way: It’s like a Swiss‑army knife that can cut wood, open bottles, and screw nuts — all in one tool

How It Actually Works

Mixture‑of‑Experts (MoE)

Only 41B of the 975B parameters fire up for each token.

🎯 Think of it this way: Imagine a kitchen with 256 chefs, but only 41 work on any single dish

Hybrid Attention & SConv

It mixes relative and sparse attention to keep speed high.

🎯 Think of it this way: Like using a wide net for fish and a fine net for shrimp — both at once

Vision & Audio Towers

Separate encoders for pictures and sounds feed into a shared language backbone.

🎯 Think of it this way: Different senses funnel into a single mind that speaks

Running the Model

Use the transformers pipeline with any‑to‑any mode.

🎯 Think of it this way: One command tells the model what to do, no matter the input type

Serving & Quantization

Serve with SGLang or vLLM, and run 1‑bit GGUFs on CPU.

🎯 Think of it this way: Deploy the giant on a tiny laptop by compressing it like a zip file

Technical Deep Dive

The model uses BF16 checkpoints and calibrated NVFP4 for inference.

Speculative decoding with MTP layers speeds up generation.

Context length reaches 1M tokens, trained on 45T tokens.

MoE setup includes shared expert sinks for balanced load.

Vibe‑eval results show strong multimodal reasoning.

Why This Matters

Inkling lets developers build apps that read documents, transcribe audio, and answer visual questions in one step.

Its efficient MoE design lowers hardware costs for startups.

Open‑source release fuels research in multimodal reasoning and fine‑tuning.

Key Takeaways

Needs ~2TB VRAM in BF16, but 1‑bit GGUFs cut size by 86% with only 26% accuracy loss
Reasoning effort can be tuned; medium (0.7) is the sweet spot
Transcribe/OCR first, then reason — prompt accordingly
Great for document processing, multimodal agents, and fine‑tuning
Free inference via Inference Providers for two hours at launch
Supported frameworks include transformers, SGLang, vLLM, llama.cpp
Vibe‑eval shows strong performance on vision and audio tasks

Inkling: The trillion‑parameter multimodal model that understands text, images, and audio