AI observability
LLM monitoring & evals
Trace, evaluate, and monitor LLM applications in production — the new stack of observability for AI.
-
#01
LangChain
· LangChain Recommended FeaturedThe dominant LLM application framework. LangGraph for agent orchestration, LangSmith for observability and evals, LangServe for deployment.
AIMenta — The de facto standard for production LLM apps. LangSmith alone justifies the ecosystem investment for teams shipping AI features.
Open source · Free OSS; LangSmith US$39/user/mo · API · Free tier · Self-host · Since 2022 -
#02
LangSmith
· LangChain Recommended FeaturedLLM application observability — tracing, evaluation, prompt management, and dataset workflows. The strongest tool for systematic LLM app development.
AIMenta — Essential for production LLM apps. The evaluation framework alone justifies the spend.
Freemium · Free 5K traces/mo; Plus US$39/user/mo · API · Free tier · Since 2023 -
#03
Weights & Biases
· Weights & Biases Recommended FeaturedThe standard for ML experiment tracking. W&B Models for training; Weave for LLM application observability. Trusted by most leading ML teams.
AIMenta — Default for any team training models. Weave is a strong LangSmith alternative for LLM ops.
Freemium · Free academic; Teams US$50/user/mo · API · Free tier · Since 2017 -
#04
Helicone
· Helicone RecommendedOpen-source LLM observability with proxy-based logging. Drop-in replacement for OpenAI base URL captures every call without code changes.
AIMenta — Lower-friction option than LangSmith if you want minimal integration. For systematic evaluation, LangSmith is more complete.
Freemium · Free 100K logs/mo; Pro US$20/seat/mo · API · Free tier · Self-host · Since 2023 -
#05
Arize Phoenix
· Arize AI Decent fitML and LLM observability platform. Phoenix is the open-source LLM tracing tool; AX is the production platform with drift, eval, and embedding monitoring.
AIMenta — Strong choice for teams already standardized on OpenTelemetry. Phoenix as a starting point is genuinely useful and free.
Freemium · Free OSS Phoenix; AX Cloud usage-based · API · Free tier · Self-host · Since 2020 -
#06
Protect AI
· Protect AI WatchAI security platform — model scanning, runtime defense, and compliance reporting. Acquired by Palo Alto Networks in 2025; now part of Prisma AI Security.
AIMenta — Watch how the Palo Alto integration plays out. For most teams, simpler tools suffice today.
Enterprise · Enterprise — pricing on request · API · Since 2022