How Morningstar Mastered AI Agent Debugging with LangSmith

Discover how Morningstar’s AI team eliminated blind debugging by adopting LangSmith for tracing, prompt collaboration, and metrics.

Ever feel like you’re debugging AI agents in the dark?

Morningstar’s engineers solved it with LangSmith.

⚡ TL;DR: Morningstar’s AI team adopted LangSmith to trace, debug, and measure agent performance, turning blind debugging into shared observability and boosting every metric.

The Simple Version

Think of an AI agent as a robot chef.

LangSmith is the kitchen camera that shows every step the robot takes.

How It Actually Works

Tracing Agent Flows

Each request through an agent creates a trace.

LangSmith captures every step, from prompt selection to tool use.

🎯 Think of it this way: It’s like a flight recorder for your AI pilot

Technical Deep Dive

They run a LangSmith server that stores full traces.

Traces include prompt text, model calls, tool responses, and timing.

Custom metrics such as time to resolution are logged automatically.

Prompt versioning lives inside the same UI, enabling team collaboration.

Why This Matters

Before LangSmith, debugging relied on scattered Splunk logs.

Now every engineer sees a common language for traces.

Result: faster issue resolution and continuous metric improvement.

Key Takeaways

Observability turns guesswork into data.
Shared tracing language improves team alignment.
Prompt versioning becomes collaborative.
Metrics improve without building custom infra.
Debugging time drops dramatically.
Agent performance can be measured end‑to‑end.
Adopting a tracing tool pays off quickly.