Most AI developers treat PDFs like simple text files, but in the real world, a single misread table cell can crash an entire autonomous workflow. When we move from simple chatbots to agents that make decisions, the cost of a 'bad parse' doesn't just ruin one answer—it compounds across every subsequent step.
The Simple Version
Imagine you're trying to follow a complex IKEA manual, but someone has shredded the pages and taped them back together slightly off-center. You might still figure out where the screws go, but you'll spend way more time thinking about the map than actually building the furniture.
This is exactly what happens when an AI agent reads a PDF. PDFs were designed in the 1990s for printers, not for AI reasoning. They care about where a pixel sits on a page, not whether a piece of text is a header or a value in a table.
Reducto's approach is to stop asking the AI to 'just read the PDF' and instead provide it with a perfectly structured map. By cleaning the data first, the AI spends less energy trying to figure out the layout and more energy solving the actual business problem.
How It Actually Works
The Hybrid Vision Strategy
You don't always need a massive Vision Language Model (VLM) to understand a page. Reducto uses a tiered approach: small, traditional Computer Vision (CV) models handle the layout, while VLMs handle the semantics.
For context, VLMs (like GPT-4o or Claude 3.5 Sonnet) are powerful but expensive and sometimes non-deterministic. By using sub-one-million parameter models that run on a CPU, they can detect document regions and layout at scale before the 'heavy hitter' models ever see the data.
Agentic OCR and Token-Level Edits
Standard OCR often fails on the 'long tail'—messy handwriting or weird fonts. However, simply asking a VLM to 'rewrite' the text is dangerous. As Abraham notes, intelligent models often try to be too helpful; if a human made a math error in a table, a VLM might 'correct' the total, which is a hallucination of the source text.
To fix this, Reducto uses agentic OCR. Instead of regenerating the whole page, they apply token-level corrections—changing a '0' to an 'O' or a comma to a period. As Abraham puts it, "VLMs can now present this sort of idea of what we call agentic OCR," mimicking a human editor rather than a ghostwriter.
The Table Dilemma: Markdown vs. HTML
Tables are the final boss of document intelligence. Simple tables work great in Markdown, but complex ones with merged cells lose their meaning. For these, Reducto uses HTML, which preserves the structural relationships that LLMs need for reasoning.
However, HTML is a nightmare for embedding models used in RAG. Embedding models struggle to correlate a natural language query (like "How did revenue change?") with a blob of HTML tags. Reducto solves this by creating a dual representation: a natural language summary for retrieval (the embedding) and the raw HTML for reasoning (the LLM input).
Technical Deep Dive
The core technical challenge in agentic document processing is the Precision-Recall Trade-off. According to Abraham, frontier models with high reasoning capabilities are extremely precise—if they extract a row, it's usually correct. However, they suffer from poor recall, meaning they silently drop rows or entire sections of a document.
Conversely, dedicated document processing services often have high recall but lower precision. To bridge this gap, Reducto employs agent harnesses. These are iterative loops where an agent uses a toolset—including a code interpreter—to extract data, visualize the result, and self-correct.
For example, when converting a line chart to a data table, no model can do it in one shot. The harness works like this: 1) The VLM approximates the data points, 2) the code interpreter plots those points back onto the image, 3) the agent compares the plot to the original chart, and 4) it repeats the process until the error margin is minimized.
This architectural shift also impacts token economics. By providing structured inputs (like the HTML/Markdown mix), the model doesn't have to use its internal 'reasoning tokens' to mentally reconstruct the document layout. This reduces latency and increases the accuracy of the final output.
Why This Matters
For industries like finance, insurance, and healthcare, this is the difference between a demo and a production-ready product. In these fields, 'mostly correct' is a failure. A missing row in a CBP form or a misread digit in a hedge fund's line chart can lead to catastrophic financial errors.
We are seeing a move away from deterministic, linear pipelines. Instead of a fixed flow (PDF $ ightarrow$ OCR $ ightarrow$ LLM), the future is an agentic file system. Here, the agent is given a CLI and a directory of documents and decides which tools to call and which files to read based on the goal.
Ultimately, the lesson is that the 'intelligence' of the model is capped by the quality of the data. As Abraham warns, "The risk of bad inputs becomes really, really pervasive in your pipeline" once you move from simple Q&A to multi-step autonomous agents.