Imagine having a digital assistant that doesn't just answer questions, but actually knows your schedule, reads your meeting notes, and even tracks your daily focus levels to give you personalized productivity hacks. This isn't science fiction; it's what happens when you move beyond simple chatbots and start building specialized AI agents.
The Simple Version
Most people use AI like a search engine: you ask a question, and it gives an answer. But an AI Agent is different; it's more like an employee that has access to your tools, your history, and even your physical hardware.
Think of a standard LLM like a brilliant professor who has read every book but has no idea what your life looks like. An agent built with Hermes is like that same professor, but they've also been given access to your calendar, your notebook, and your smart home devices.
How It Actually Works
The Agent Profiles
The core idea behind Hermes is the ability to create different "profiles" or specialized agents tailored to specific roles. Tina Huang demonstrates this with Tacobot, a team coordinator on Discord connected to Notion and GitHub, and Lifebot, a personal assistant that tracks food intake and health metrics from an Oura Ring.
Beyond these, Hermes allows for highly specialized bots: Sensei turns non-fiction books (via PDFs or audio) into personal coaches, and Revelbot acts as an always-on research agent. Tina also highlights the possibility of a "Jarvis situation," where agents act as deal watchers to alert you of sales or group chat digests to summarize massive amounts of WhatsApp or Discord messages.
As Tina puts it, "Hermes is freaking pretty magical," because it allows you to build a suite of agents that handle everything from autonomous coding to health journaling. She notes that you can have as many Hermes profiles as you want, each potentially connected to its own Telegram bot.
The Data Engine and LM Wiki
For an agent to be useful, it needs a place to store and retrieve information. Hermes uses a dual-memory approach: Honcho shared memory for short-term context and Obsidian as a long-term AI database.
To manage this, Tina uses an LM Wiki—an LLM-managed library of sources. While Obsidian is the tool used for visualization, the LM Wiki is the implementation that allows an AI to scan, prune stale data, and update information over time. This prevents "knowledge rot" and allows agents like Wikbot to ingest new links and cross-reference them with existing build specs.
Technical Deep Dive
From an architectural standpoint, Hermes is an agent harness that decouples the LLM from the interface. It is accessible via a desktop app for non-coders or a terminal for developers. For those running local models, Tina recommends Llama CPP over Ollama for those seeking more control and faster speeds.
The system is entirely model agnostic. Users can route requests to Gemini 1.5 Flash for speed or use local models like Qwen 2.5 35B (or smaller versions like 27B) for privacy. Hardware requirements are flexible: while a Mac Studio is used for heavy lifting, a laptop with 16GB of RAM is sufficient. For extreme edge cases, models as small as 3MB can run on ESP32 microcontrollers to control IoT hardware.
The Execution Pipeline: The Runbook
The most critical part of the building process is the Runbook. Tina describes this as a combination of a PRD (Product Requirements Document) and an implementation plan. Instead of coding manually, the workflow follows this path: Idea → LM Wiki (Research) → Claude/OpenAI (Runbook Generation) → Coding Agent (Execution).
By spending days or weeks refining the runbook, the actual implementation becomes trivial. Tina uses coding agents like Claude Code for linear implementation and Codeex for builds requiring visual elements. Because LLM-generated text can be unstable, she suggests compiling skills into code to ensure workflows are deterministic and stable over time.
Why This Matters
The shift from "chatting with AI" to "building with agents" represents a massive leap in productivity. We are moving away from manually moving data between apps toward building systems that move the data for us. This allows individuals to automate the "lazy" parts of life—like tracking exercise or food—without the friction of manual entry.
For professionals, this enables autonomous coding and research. Instead of a human writing every line of code, they manage a runbook that a coding agent executes. This transforms the development cycle, allowing complex agents to be deployed in hours rather than weeks, provided the foundational logic is sound.
Finally, the move toward local-first agents solves the privacy dilemma. By using tools like Open Router or local Qwen models, users can build a "Jarvis" that has deep access to their private lives and home IoT devices without sending that sensitive data to a centralized cloud provider.