What is a semantic ID in recommendation systems?

A semantic ID is a compressed token that represents the meaning of content, allowing an LLM to understand and reason over video or text without processing every single raw bit of data.

Why are feed-based apps cheaper than AI chatbots?

Chatbots must generate every single token of text at inference time, whereas recommenders only need to output a small 'pointer' (a semantic ID) to content that already exists.

What does it mean to have a 'steerable' feed?

It means users can use natural language to give direct instructions to the algorithm, such as 'show me more cooking videos' or 'stop showing me sports,' rather than relying on implicit signals like likes.

Why LLM Recommenders Will Be AI's Biggest Consumer App

Devansh Tandon from Meta explains how LLM-native recommendation systems use semantic IDs to scale efficiently, potentially outperforming AI chat apps.

Most people think the biggest AI win will be a chatbot that writes your emails, but the real goldmine might be the algorithm that decides which video you watch next.

⚡ TL;DR: Devansh Tandon from Meta argues that LLM-based recommenders will become AI's largest consumer application. By using semantic IDs to turn content into tiny tokens, these systems can be up to 100x cheaper to run per hour of engagement than generative AI chat apps.

The Simple Version

Imagine you are at a massive library. A traditional recommendation system is like a librarian who looks at a list of books you've checked out and tries to find similar titles based on keywords.

An LLM-native recommender, however, is like a genius scholar who has actually read every book and can explain why they think you'll love a specific chapter. Instead of just matching tags, it understands the nuance of the story.

🎯 Think of it this way: Traditional recommendation is like a grocery store clerk recognizing you buy milk and suggesting eggs. An LLM recommender is like a chef who knows you like spicy food, understands your dietary restrictions, and can explain exactly why a specific dish fits your palate.

How It Works

The Four S-Curves of Recommendation

The industry isn't just making one type of recommender; it is climbing four distinct evolutionary stages, or "S-curves."

First is Traditional RecSys, which relies heavily on manual feature engineering. Next is LLM-inspired, where models scale using techniques borrowed from language models. The third is LLM-native, where a base model is specifically adapted to reason about recommendations. Finally, we are seeing the rise of Agentic systems, where AI agents plan, retrieve, and even critique recommendations in a loop.

The Three-Step Recipe

Building an LLM recommender isn't just about feeding data into a model; it requires a specific architectural approach. As Devansh Tandon describes it, the process follows a "three-step recipe":

Tokenization via Semantic IDs: Converting content into a language the model understands.
Bilingual Pre-training: Training the model to understand both natural English and your specific content catalog.
Post-training for Ranking: Teaching the model to use reasoning to order content for a user.

🎯 Think of it this way: Creating semantic IDs is like giving every book in the library a unique, descriptive code that tells you its genre, author, and mood, rather than just a random barcode.

Technical Deep Dive

The core innovation that makes this scalable is the move from hash-based IDs to Semantic IDs (SIDs). In traditional systems, a piece of content might have a random ID that the model has to memorize. This is inefficient and doesn't scale.

By using SIDs, you can compress massive amounts of data. Tandon notes that a three-minute Instagram Reel, which might normally represent about 10,000 tokens, can be compressed into just 10 semantic tokens. This allows the model to reason over much longer user interaction histories without hitting context window limits.

The Five-Layer Cake Architecture

For developers looking at the stack, a modern LLM recommender is structured like a five-layer cake:

Semantic ID Layer: The foundation that converts the content corpus into manageable tokens.
Base LLM Foundation: An open-weights or first-party model (like Llama) that provides the reasoning engine.
Pre-training: The stage where the model learns to bridge the gap between English and the semantic IDs.
Post-training: Where the model learns to rank and use Chain-of-Thought (CoT) reasoning.
Surface Fine-tuning: Light adjustments to deploy the model on specific products like Reels or Ads.

One of the most powerful outcomes of this architecture is steerable feeds. Because the model uses CoT, it can actually explain its logic. As Tandon points out, users can eventually "talk to" their algorithm—for example, by telling Instagram, "I want to follow the FIFA World Cup right now"—and the model can immediately adjust the feed based on that linguistic command.

Why This Matters

The economic argument for LLM recommenders is staggering. While AI chat apps like ChatGPT or Claude must generate every single word (token) from scratch at inference time, a recommender system is simply decoding pointers to existing content.

Since creators provide the content, the "supply" is essentially free. This structural difference means that content feeds can be up to 100x cheaper per hour of consumer engagement than generative chat apps.

With four of the top ten most-used apps in the world being content feeds, the transition to LLM-native systems represents one of the largest potential shifts in how AI is monetized and consumed at scale.