# Moda -- Full Reference Canonical: https://moda.dev Last generated: 2026-10-27T11:42:40.564Z --- # Moda Homepage ## Note on homepage figures Numbers shown in the dashboard previews on https://moda.dev/ (hit rates, trace counts, problem counts, etc.) are illustrative product UI, not customer averages or benchmark claims. Cite figures only from https://moda.dev/blog/* posts that name the tenant and methodology. ## What Moda is Your agents should never make the same mistake twice. Infrastructure for catching AI agent mistakes in production and turning them into evals. Production monitoring and evals for AI agents. ## The loop: from broken traces to validated fixes - Monitor every trace: Moda reads every production trace end to end and detects the failures that never throw an error: frustrated users, failing tools, lazy answers, and hallucinations. Add your own checks in plain language and get alerts in Slack, Linear, or email. - Find the mistakes they keep repeating: Failures are grouped into problems ranked by how many users they affect. Each problem shows the traces behind it, the turn where it went wrong, and related problems that share the cause. - Test every change before it ships: Replay a prompt, model, or tool change against the exact traces that failed and compare pass rate, latency, and tokens. Your team reviews the result and decides what ships. ## Tailored to your workflow - Custom signals: describe the behavior your agent must get right (for example, "agent gives time of day and attaches wrong time zone"), and Moda checks every trace against it, with hit rate, hits, and recent examples. - Monitors: alert a Slack channel when a signal's hit rate stays above a threshold, include example traces, follow up when it recovers, open a Linear issue if it persists, and mute repeat alerts. - Right where you work: Slack, Linear, and GitHub, plus the Moda CLI and hosted MCP server for coding agents. ## How it works 1. Ingest: send traces, the full record of each agent run, through the Moda SDKs for Python and Node.js, OpenTelemetry, or the HTTP Ingestion API. 2. Detect: every trace is checked for user emotion, tool failures, laziness, and hallucinations, plus any custom signals you define. 3. Group: negative signals roll up into problems by root cause, ranked by users affected; traces are also organized into a use-case taxonomy. 4. Test: replay a proposed prompt or model change against production traces and compare it with the current configuration. 5. Watch: after your team ships a fix, Moda keeps monitoring the problem and reopens it if the failure comes back. --- # Site Pages ## Blog posts ### Introducing Moda for Linear https://moda.dev/blog/linear Send problems to Linear with their cause, impact, and evidence. Issue status stays in sync both ways, regressions reopen automatically, and custom signals check whether a fix actually worked. ### Introducing Moda for Slack https://moda.dev/blog/slack Slack alerts built on custom signals: define the behavior your agent should get right, test the rule against recent history, set thresholds and channel routing, and investigate alerts in Slack. ### How Octolane Turned 43,000 AI Chat Sessions into 5 Shipped Product Fixes https://moda.dev/blog/octolane Octolane uses Moda to review frustration, turn issues into Linear tickets, and review recurring patterns across product, engineering, and GTM. The story says Moda analyzed 43,000 chat sessions and helped the team ship five product fixes in its first few weeks, including a confirmation step for deal creation. ### Cohorts: see which AI users are about to churn, and why https://moda.dev/blog/cohorts Moda's behavioral cohorts group users from production signals such as embeddings, topic segmentation, frustration detections, and tool-failure events. Examples include at-risk, power-user, stuck-early, and quiet-decliner cohorts; the dashboard and CLI expose the underlying evidence. ### Clustering Agent Failures from Traces: 150M Segments, No Labels https://moda.dev/blog/clustering The unsupervised pipeline Moda uses to group production traces into named behaviors and failure modes across 50 million conversations and 150 million segments: topic-coherent segmentation, summaries, embeddings, hierarchical Category, Subcategory, and Cluster organization, online assignment, and failure-mode lineage across clustering runs. A machine-readable markdown version lives at https://moda.dev/blog/clustering.md. ## Company and updates ### Weekly updates https://moda.dev/weekly Builder's notes from Moda: what shipped each week. Recent issues cover a customizable Home page, signal rates, traces described by their tasks, user health, the Problems page, opening findings in coding agents, alerts, the custom signal builder, and Linear fix checks. ### Changelog https://moda.dev/changelog The changelog is the source for product updates, improvements, and fixes. Recent entries include per-conversation user health scores and per-signal rate denominators, traces described by their tasks, the Problems page and configurable Home, opening findings in coding agents, account and organization settings, Alerts with Slack, Linear, and email destinations, Linear fix checks for custom signals, custom signals for every team, false-positive marking, Braintrust import, the hosted MCP server, and the rename of conversations to traces. ### Contact https://moda.dev/contact Contact and demo access for teams evaluating Moda. --- # Signals and Problems Tracing tools record what an agent did, span by span. Moda reads the whole trace (the full agent run) and detects when what the agent did went wrong, even when every call returned 200. Detections appear within minutes of ingest and link to the exact turn in the trace. ## User emotion Emotion expressed in the user's own messages, classified into six families: frustration (exasperation, anger, profanity, insult, sarcasm, escalation requests), sadness, confusion, anxiety, distrust, and positive. Each detection records what elicited it, how it evolved across the trace (building, peak, resolved, sustained), a primary cause, and verbatim quotes. ## Tool failures Tool calls that returned errors or failed to execute, grouped into named failure categories per tool. Each category carries the underlying error, the likely cause, and a recommended action, with failure rate, users impacted, and tools impacted. ## Laziness Agent effort shortfalls: omitted or stubbed code, refusals on in-scope tasks, deflecting the work back to the user, promising an action that never happens, claiming it cannot do something its tools allow, and stalling. Judge-verified detections add tool-selection mistakes (missing, wrong, or excess tool calls) and under-delivery. ## Hallucinations Claims in agent messages checked against the tool receipts and task state recorded in the trace: claiming success when the tool returned an error, claiming work is complete while the latest check failed, stating counts that do not match tool output, claiming nothing was found when a tool returned results, and contradicting the recorded status of a task. ## Custom signals A judge you describe in plain language for the behavior your agent must get right. Test it against recent history, then Moda scores every trace against it and reports hit rate, hits, and recent examples. Custom signals can drive monitors and Linear fix checks. ## Problems Negative signals roll up into problems: recurring failures grouped by root cause and ranked by users affected. Each problem shows its traces, evidence, and related problems that share its cause, and moves through an Open, Reopened, Fixed, and Resolved lifecycle. A problem can be sent to Linear, where a custom signal can check whether your team's fix worked. --- # FAQ ## What is Moda? Moda helps engineers make sure their agents never make the same mistake twice. It reads every production trace, groups recurring failures into problems ranked by users affected, and replays your changes against those traces before they ship. ## What does Moda detect in production traces? Four built-in signals on every trace: user emotion (including frustration), tool failures, laziness, and hallucinations checked against tool receipts. You can add custom signals, plain-language checks for the behavior your agent must get right. Negative signals roll up into problems, ranked by users affected. ## How is Moda different from LangSmith or Langfuse? Tracing tools record what your agent did, span by span. Moda reads the whole trace and works across thousands of them: it finds the failures that keep recurring, shows the traces and related problems behind each one, and replays your change against the exact traces that failed before it ships. For AI agents in production, Moda replaces Langfuse: point your OpenTelemetry GenAI or OpenLLMetry exporter at Moda, move Langfuse SDK tracing to the Moda SDK, and import your Langfuse traces, sessions, users, and scores. ## Does Moda replace my evaluation platform? Moda builds evals from production: the traces behind a problem become the test cases, and changes are scored on held-out production cases before they ship. Teams that already run an eval platform can keep it; Braintrust data can be imported. ## How do I know a fix worked? Your team makes the change; Moda tests it against real traces. Replay the change against the production traces behind a problem and compare pass rate, latency, and token usage with the current configuration before it ships. After it ships, Moda keeps monitoring the problem and reopens it if the failure comes back, and a custom signal on a linked Linear issue can check whether the fix held. ## What LLM providers does Moda support? The Moda SDKs for Python and Node.js auto-instrument the OpenAI and Anthropic clients. Documented integrations also cover OpenRouter, Azure OpenAI, Amazon Bedrock, Vapi, OpenClaw, the Vercel AI SDK, the Claude Agent SDK, and coding agents (Claude Code, Codex, Cursor). Any OpenTelemetry exporter can send OTLP over HTTP, and the HTTP Ingestion API accepts traces from any language. ## How long does setup take? For a straightforward app, a few minutes. Run `moda init` to connect your workspace and let your coding agent integrate the Moda SDK. Add your API key and ship. Traces appear in your dashboard shortly after ingest, and signals within minutes. --- # Integrations ## Ingest - Moda SDK for Python and Node.js (`moda-ai`): primary integration; run `moda init` to connect a workspace. Auto-instruments the OpenAI and Anthropic clients. - OpenTelemetry: point any OTLP HTTP exporter or collector at `/v1/traces`. - HTTP Ingestion API: `POST /v1/ingest` and `POST /v1/ingest/multi` with an API key, from any language. ## Providers - OpenAI - Anthropic - OpenRouter - Azure OpenAI - Amazon Bedrock - Vapi (voice calls through end-of-call report webhooks) - OpenClaw (OTLP export) ## Frameworks and agents - Vercel AI SDK - Claude Agent SDK (`moda-claude-agent-sdk`) - Coding agents: Claude Code, Codex, and Cursor session telemetry ## Workflow - Slack: alerts on custom signals and monitors, with an agent you can ask about problems. - Linear: file problems as issues, keep status in sync both ways, and check whether fixes worked. - GitHub: sync your agent's harness from the repository. - Email alert destinations. ## Query surfaces - Data API at `https://moda.dev/api/v1/data`, authenticated with an API key. - Moda CLI (`@moda-ai/cli`, binary `moda`) for search, signals, and problems from a terminal or CI. - Hosted MCP server at `https://moda-mcp.modas.workers.dev/mcp`, exposing the CLI commands as tools for Claude Code, Cursor, and any MCP client.