Polly is Now Generally Available Across LangSmith

LangChain has expanded Polly, its AI debugging assistant, to be available across all LangSmith pages and workflows. Previously confined to trace pages, thread views, and the playground, Polly is now accessible in every page via a bottom-right corner interface that can be opened with Cmd+I (Mac) or Ctrl+I (Windows/Linux).

Key Capabilities

Persistent Context Across Navigation Polly now maintains conversation history as you navigate between different LangSmith views. Start debugging a trace, switch to experiments to compare runs, return to a previous view, and Polly remembers the full context. This eliminates friction when moving between analysis tasks and debugging workflows.

Action-Oriented Assistance Beyond answering questions, Polly can now:

• Update and refine prompts directly
• Create datasets from failing runs
• Filter and organize project views
• Write and improve evaluator code
• Compare experiment results and provide recommendations

New Use Cases

Thread Analysis: For production agents, Polly can analyze entire user conversations to assess sentiment, identify issues, and determine if problems were resolved—without requiring manual review of every message.

Evaluator Development: Teams can now ask Polly to generate evaluator code, improve accuracy, and handle edge cases, reducing scaffolding time and letting developers focus on what matters most.

Experiment Comparison: After running evaluations, Polly can recommend which experiment performed best based on actual data, helping teams make informed decisions about prompt changes, model selections, and architectural improvements.

Getting Started

LangSmith users can access Polly immediately in the bottom-right corner of any page. To use Polly, add an API key for your model provider as a workspace secret (a 2-minute setup). New LangSmith users should first set up tracing to get data flowing into the platform.