What is AI model routing?

Model routing is the process of using a central logic layer to analyze a prompt and send it to the most appropriate LLM based on cost, speed, or capability.

Why use local models instead of cloud APIs?

Local models allow you to process sensitive data on your own hardware, ensuring that private information is never sent to a third-party company's servers.

Is a multi-model system harder to maintain than one subscription?

It requires more initial setup to build the routing logic, but it provides long-term flexibility and significant cost savings.

Moving Beyond Claude: Building a Multi-Model AI Routing…

Learn why relying on a single AI vendor is a risk and how to build a routing system using DeepSeek, Kimi, and local models to cut costs and increase privacy.

Imagine paying a premium monthly fee for a superpower, only to realize that your entire business is now hostage to a single company's pricing whims and terms of service. This is the "subscription trap" that many AI power users fall into when they rely solely on one frontier model.

⚡ TL;DR: To avoid vendor lock-in and high costs, Nick Puru moved from a $200 Claude subscription to a multi-model routing architecture. By using a central router to distribute tasks among GLM, Kimi, DeepSeek, and local models, he reduced expenses while improving data privacy for client work.

The Simple Version

For a long time, the trend in AI has been to find the "one model to rule them all." Whether it is Claude, GPT-4, or Gemini, users tend to put all their prompts and workflows into one interface because it is convenient.

However, relying on one vendor is like hiring one person to be your accountant, your lawyer, your marketing head, and your janitor. While that person might be brilliant, they are expensive, and if they quit or raise their rates, your entire business grinds to a halt.

🎯 Think of it this way: Instead of hiring one overpriced "super-employee," you are building a specialized agency. You have a manager who listens to the request and then assigns the task to the cheapest, most efficient specialist who can actually do the job.

How It Actually Works

The shift from a single subscription to a multi-model system involves moving from a Chat Interface to a Routing Architecture. Instead of typing into a window, the user interacts with a system that decides where the prompt should go.

The Router (The Dispatcher)

At the heart of this system is a routing file. This is essentially a set of rules or a piece of code that analyzes an incoming request and maps it to a specific model based on the task's complexity and the required privacy level.

If a task is simple—like formatting a list—the router sends it to a fast, cheap model. If the task requires deep reasoning or complex coding, it might go to a frontier model like Claude. This prevents the user from "wasting" expensive tokens on trivial tasks.

The Specialized Staff

Nick mentions utilizing a variety of models to handle different roles. He incorporates GLM, Kimi, and DeepSeek. These models often excel in specific areas or offer significantly better price-to-performance ratios than the big American frontier models.

He also uses Claude Code and Codex as the "bodies" or execution layers. These tools handle the actual writing and implementation of code, while the routing logic handles the high-level decision-making.

🎯 Think of it this way: The router is like a switchboard operator in an old office. They don't do the work themselves; they just make sure your call gets to the right department so you don't spend an hour waiting on hold with the wrong person.

The Local Vault

One of the biggest drivers for this change is client data privacy. When working with sensitive business information, sending data to a cloud-based API (like Anthropic's or OpenAI's) can be a compliance nightmare.

By integrating local models, the system can process sensitive data on the user's own hardware. This ensures that private client information never leaves the local environment, providing a layer of security that no cloud subscription can offer.

Technical Deep Dive

From a developer's perspective, this architecture moves the intelligence from the Model Layer to the Orchestration Layer. Instead of relying on the inherent capabilities of a single LLM, the developer builds a system of "agents" and "routes."

Implementing the Routing Logic

A routing file typically functions as a mapping object. In a simplified version, it looks like a series of if/else statements or a JSON configuration that assigns a task_type to a model_endpoint. For example, a request tagged as "Data Scrubbing" would be routed to a local Llama-3 instance, while "Architectural Review" would be routed to a high-reasoning API.

This allows the developer to swap models instantly. If a new model like DeepSeek releases a version that is 2x faster and 50% cheaper, the developer only needs to change one line in the routing file to update their entire business infrastructure.

The Manager-Worker Pattern

The system uses a Manager Pattern (referred to as "Fable" in the video). The manager model is usually a high-reasoning model that doesn't do the grunt work but instead breaks a complex prompt into smaller sub-tasks.

Once the manager has decomposed the problem, it assigns those sub-tasks to the "worker" models. This is significantly more efficient because you only pay the "premium price" for the reasoning phase, while the execution phase is handled by cheaper, specialized models.

Hardware and Local Deployment

To make the local model component work, users typically employ tools like Ollama or LM Studio. This allows them to run quantized versions of open-source models on their own GPUs. By doing this, the "cost per token" for local processing drops to essentially zero, minus the cost of electricity and hardware.

Why This Matters

This approach solves the problem of Vendor Lock-in. When a business builds its entire workflow around one tool, they lose all bargaining power. If the vendor changes their API, increases prices, or suffers an outage, the business is paralyzed.

By diversifying the model stack, a developer creates a resilient system. If one API goes down, the router can automatically failover to a secondary model, ensuring that the business keeps running regardless of a single company's stability.

Ultimately, this represents the professionalization of AI usage. Moving from a "chatbot subscriber" to an "AI architect" allows for better cost control, superior data privacy, and the flexibility to use the best tool for every specific job.

Moving Beyond Claude: Building a Multi-Model AI Routing System

Key takeaways