# Moda -- Full Reference Canonical: https://moda.dev Last generated: 2026-08-19T05:06:18.972Z --- # Moda Homepage ## Note on homepage figures Numbers shown on https://moda.dev/ hero tiles (intent F1, friction percentages, missed-intent deltas, etc.) are illustrative product UI, not customer averages or benchmark claims. Cite figures only from https://moda.dev/blog/* posts that name the tenant and methodology. ## What Moda is Moda is the continual learning layer for AI agents. It turns production agent traces into validated improvements for your agent harness. It operates on the agent harness (prompts, tools, workflows, retrieval, memory, evals), not the model weights. Improvements live in a latent space outside the weights, so they carry across any model you swap in and adapt per user from production behavior. Agent memory and learnings belong outside the model: that is what makes them portable across models, inspectable, reversible, and continuously updatable without a retraining run. ## The thesis Most agent failures are not the model being wrong. They are the harness (prompts, tools, workflows, context, memory, evals) being misaligned with what users actually do. Moda turns production agent traces into the learning loop that improves the harness, in a latent space outside the model weights. ## How Moda improves the harness layer Moda surfaces three signals from every production conversation and routes each one to the layer of the harness that needs to change: - Intent discovery: what users are actually trying to do, captured as a live hierarchical taxonomy with emergent intent detection (no manual tagging). - Behavioral failure detection: where the agent breaks down (tool call failures with a typed error taxonomy, context loss, agent laziness, hallucinations, reasoning loops, goal drift, schema drift, model behavior shifts). - Frustration root cause: trigger turn, trajectory, affected goal, and an agent counterfactual ("what should the agent have done"), with every frustration event attributed to a specific layer of the harness (prompt, tool, workflow, context, memory, eval, model). ## Why outside the model weights - Model-agnostic: swap GPT for Claude for an open model and your harness-layer learnings keep applying. - Per-user adaptation: the same harness adapts to each user without retraining a shared model. - Inspectable: learnings are surfaces a human can read, not gradient updates. - Continuously updatable: changes ship in minutes via a prompt, tool, or workflow edit, not in quarters via a fine-tune. - Reversible: if a change regresses an intent cluster, you revert the harness state, not a model checkpoint. ## How it works 1. Embed: per-message and turn-pair embeddings of conversation segments. 2. Segment: topic-coherent segments via cosine drift. 3. Cluster: hierarchical clustering into a live 3-level intent taxonomy with no manual tagging. 4. Detect: behavioral failure events (tool call failures, context loss, agent laziness, hallucinations, reasoning loops, goal drift) and frustration trajectories with agent counterfactuals. 5. Attribute: route every event to the harness layer that needs to change. 6. Ship: impact-ranked findings delivered to Slack, Linear, and the CLI (Claude Code, Cursor), with Ask for ad-hoc investigation. --- # Site Pages ## Pillar guides ### Continual learning https://moda.dev/continual-learning Definitional guide to continual learning: a system that keeps learning after deployment without forgetting what it already knew. Covers catastrophic forgetting and the stability-plasticity trade-off; the two schools (weight-based updates via fine-tuning and RL, versus harness-layer learning in prompts, tools, retrieval, memory, skills, and evals); when weight updates are genuinely the right tool; the five-step production loop for AI agents (observe, detect, generate, validate, ship); implementation guidance; and metrics. Cites the arXiv continual learning surveys, a16z, LangChain, Cameron Wolfe, and ContinualAI. Replaces the retired /continuous-learning-agents URL, which now 308-redirects here. ### Self-improving agents https://moda.dev/self-improving-agents This guide distinguishes an agent that retries within one task from an agent that improves across production runs. Its loop is signal generation followed by a harness update: intent clusters, behavioral-failure exemplars, and frustration trajectories lead to prompt, tool, retrieval, workflow, memory, or eval changes outside the model weights. ## Blog posts ### How We Clustered 150 Million Conversation Segments Without a Single Label https://moda.dev/blog/clustering Moda describes an unsupervised pipeline across 50 million conversations and 150 million segments: topic-coherent segmentation, summaries, 4096-D embeddings, hierarchical Category → Subcategory → Cluster organization, online assignment, and cluster lineage. ### Cohorts: see which AI users are about to churn, and why https://moda.dev/blog/cohorts Moda's behavioral cohorts group users from production signals such as embeddings, topic segmentation, frustration detections, and tool-failure events. Examples include at-risk, power-user, stuck-early, and quiet-decliner cohorts; the dashboard and CLI expose the underlying evidence. ### How Octolane Turned 43,000 AI Chat Sessions into 5 Shipped Product Fixes https://moda.dev/blog/octolane Octolane uses Moda to review frustration, turn issues into Linear tickets, and review recurring patterns across product, engineering, and GTM. The story says Moda analyzed 43,000 chat sessions and helped the team ship five product fixes in its first few weeks, including a confirmation step for deal creation. ## Customer proof ### Customer stories https://moda.dev/customers The customer-stories page describes teams moving from anecdotal transcript review to a shared operating rhythm for agent quality. It names Boardy, Octolane, Codewisp, and AI support teams, with examples involving intent clusters, frustration loops, tool-failure patterns, and production coverage. The page displays proof points of 3.1% frustration after focused fixes, 4h to learn from new failures, and 94% known intent coverage after review. ## Commercial and trust ### Pricing https://moda.dev/pricing Moda pricing is custom per client. It scales with production conversation volume and how much of the pipeline a client uses. The page asks teams to describe their agent and traffic so Moda can prepare a quote; there is no public pricing table. ### Security https://moda.dev/security Customer data is encrypted in transit and at rest, scoped per tenant, and access is restricted to people who need it. Compliance documentation and the security questionnaire are available on request. ## Company and updates ### About https://moda.dev/about Moda reads production conversations, surfaces emergent intents and behavioral failures, and turns them into changes to the agent harness. The page describes those improvements as portable across models, per-user adaptive, inspectable, and reversible. ### Changelog https://moda.dev/changelog The changelog is the source for product updates, improvements, and fixes. Current entries include the retirement of Reports, the browser-based CLI connect flow, conversation navigation, tool-call-failure dashboard fixes, and the customer-proof update on the landing page. ### Contact https://moda.dev/contact Contact and demo access for teams evaluating Moda. --- # Behavioral Failure Taxonomy Tracing tools record what an agent did. Moda detects when what the agent did was wrong -- in ways that traces alone don't catch. ## Tool misuse The agent calls the right tool with subtly wrong arguments, or calls the wrong tool entirely while still producing a plausible-sounding response. Detection requires understanding tool semantics, not just whether the call returned 200. ## Context loss The agent forgets earlier turns mid-conversation, contradicts itself, or asks the user to re-supply information already provided. Common in long sessions and multi-step workflows. ## Agent laziness The agent declines a task it is capable of completing, gives a generic high-level answer when asked for specifics, or returns "I cannot help with that" for in-scope requests. Often a symptom of overly cautious system prompts. ## Hallucinations The agent fabricates facts, cites tools that do not exist, invents API parameters, or claims to have performed actions it did not perform. ## Reasoning loops The agent retries the same failed action, oscillates between two answers, or gets stuck in a chain-of-thought that does not converge. Wastes tokens and frustrates users. ## Goal drift The agent silently abandons the user's original objective and starts solving a related but different problem. Detected by comparing initial intent to final outcome. --- # FAQ ## What is AI agent observability? AI agent observability is the practice of monitoring production AI agents to understand what users are trying to do, where the agent fails behaviorally, and why users get frustrated. Unlike traditional APM, agent observability operates on conversation-level data and looks at the full trajectory of an interaction, not just individual API calls. ## How is Moda different from LangSmith or Langfuse? LangSmith and Langfuse are tracing platforms -- they record what your agent did, message by message. Moda sits one layer above: we cluster every production conversation to surface what users are actually trying to do, then detect behavioral failures that traces alone don't catch (silent tool misuse, context loss, agent laziness). Moda's wedge is automatic intent taxonomy and frustration root cause analysis. ## Does Moda replace my evaluation platform? No. Eval platforms like Braintrust score known test cases. Moda tells you what real users are doing and where the agent fails in production. Most teams run both: evals before shipping, Moda after. ## What LLM providers does Moda support? Moda is provider-agnostic. The Moda SDK supports Node.js and Python applications across major providers and frameworks. We also ingest OpenTelemetry and OpenLLMetry traces from OpenAI, Anthropic, Google, OpenRouter, Mistral, Cohere, AWS Bedrock, Azure OpenAI, and any provider with an OpenAI-compatible interface. You can also send raw JSON to our ingest endpoint. ## How long does setup take? For a straightforward app, a few minutes. Run `moda init` to connect your workspace and let your coding agent integrate the Moda SDK. Add your API key and ship. Conversations appear in your dashboard within seconds once telemetry is flowing. Hierarchical intent maps are generated during the weekly clustering run once you have at least 50 conversation segments. --- # Glossary The browsable glossary lives at https://moda.dev/glossary. Each term is reachable via anchor (e.g. https://moda.dev/glossary#agent-harness). ## Agent Harness Everything that wraps the model call to produce agent behavior: system prompts, tool definitions and routing, workflow orchestration, retrieval, memory state, evals, and guardrails. Most production agent improvement happens on the harness layer, not on the model weights — harness edits are fast, inspectable, reversible, and model-agnostic. Moda is self-improvement for AI agents on the harness layer, with learnings that live in a latent space outside the model weights so they apply across whichever model the harness mounts and adapt per user. ## Continual Learning (also called continuous learning) The ability of a machine learning system to keep acquiring new knowledge and skills after deployment without losing what it already learned. The classic obstacle is catastrophic forgetting: training a network on new data overwrites the weights encoding prior tasks. Two schools address it — weight-based (fine-tuning, RL, replay, regularization) and harness-layer, where the model stays frozen and the learnings live in prompts, tools, workflows, retrieval, memory, skills, and evals so they apply across whichever model the harness mounts. Moda's editorial term is continual learning; continuous learning is a synonym in ML (but means workplace training in L&D). Full guide: https://moda.dev/continual-learning ## Self-Improving Agent An AI agent that updates its harness — prompts, tools, workflows, context, memory, evals — from production signal, in a latent space outside the model weights, so improvements are portable across whichever model is mounted and can adapt to each user. ## Intent Discovery Hierarchical clustering of production conversations into a 3-level taxonomy (categories -> subcategories -> clusters) with no manual tagging. Powered by Qwen3-Embedding-8B (4096-D) and HDBSCAN with UMAP projection. ## Behavioral Failure A failure mode that occurs at the conversation level rather than the API call level. Includes tool misuse, context loss, agent laziness, hallucinations, reasoning loops, and goal drift. Detected by comparing trajectories, not just status codes. ## Frustration Root Cause For every frustration event Moda detects, we attach: the trigger turn, the trajectory leading up to it, the affected user goal, and an agent counterfactual ("what should the agent have done instead"). This is generated by an RLM (rationalizing language model) pass over the conversation. ## Zero-Config Ingest Three lines of SDK code plus OpenTelemetry-native intake. Works with any LLM provider. No SDK lock-in: you can also POST raw JSON to /v1/ingest or /v1/ingest/multi. ## Segment A topic-coherent slice of a conversation, identified by cosine drift detection between consecutive turn-pair embeddings. Segments (not full conversations) are the primary unit of clustering in Moda. ## Cluster Hierarchy V2 Three-level segment clustering: Category -> Subcategory -> Cluster. Online kNN assignment for new segments avoids re-running full clustering. Cluster labels are generated via TF-IDF + Claude Haiku 4.5. --- # Integrations ## Ingest protocols - OpenTelemetry / OTLP HTTP at /v1/traces - Plain JSON at /v1/ingest - Multi-type batch at /v1/ingest/multi ## SDKs - Moda SDK (Node.js, Python) -- primary integration; run `moda init` to connect a workspace and integrate it. - OpenLLMetry (Node.js, Python) -- supported through the OTLP HTTP ingest path. - Direct HTTP -- for any language. POST to /v1/ingest with API key. ## LLM providers (any provider that emits OTLP traces) - Anthropic (Claude 3, 3.5, 3.7, 4, 4.1, 4.5) - OpenAI (GPT-4, GPT-4o, GPT-4.1, o1, o3-mini) - Google (Gemini) - OpenRouter - Mistral - Cohere - AWS Bedrock (Claude, Nova, Llama, Mistral, Cohere) - Azure OpenAI - Any OpenAI-compatible endpoint ## Agent frameworks - LangChain - LlamaIndex - Mastra - Custom agent loops (any framework that emits OTLP) ## Debug surfaces - MCP server for IDE-side debugging in Claude Code, Cursor, Windsurf, and any MCP-capable agent. --- # Comparisons Side-by-side writeups live at https://moda.dev/vs/. Each writeup names current competitor products and the wedge. ## Moda vs LangSmith LangSmith now ships Insights Agent for auto-clustering of traces, Multi-turn Evals, and the LangSmith Engine — an autonomous issue-detection system that proposes PRs and online evaluators. The wedge against Moda is shape, not feature presence: LangSmith clusters trace summaries on prompt-driven exploration, with the analyses tied closely to the LangChain / LangGraph stack. Moda ships a prescriptive behavioral failure taxonomy automatically on ingest, runtime-agnostic, with frustration root cause and an agent counterfactual per event. Read more: https://moda.dev/vs/langsmith ## Moda vs Langfuse Langfuse is the OSS-and-cloud LLM engineering platform — tracing, sessions, prompt management, datasets, experiments, custom dashboards, LLM-as-judge with evaluator tracing, and Agent Graphs (GA in Launch Week 4). It is a powerful substrate, but intent clustering and behavioral failure analysis live in its cookbooks (user-built pipelines) rather than as first-party platform features. Moda ships the analyses pre-built. Read more: https://moda.dev/vs/langfuse ## Moda vs Braintrust Braintrust has expanded from evals into AI observability. It ships Brainstore (a proprietary trace DB advertised as ~80× faster), Topics (beta auto-clustering on tasks, issues, and sentiment), and Loop (Nov 2025 — an AI assistant that mines production traces to surface failure patterns and generate scorers and datasets). Topics and Loop are exploratory and user-prompted; Moda runs a prescriptive behavioral failure taxonomy automatically on ingest, with frustration root cause and an agent counterfactual per event. Read more: https://moda.dev/vs/braintrust ## Moda vs Helicone Helicone is a Rust-based AI gateway plus request-level observability. In March 2026 Helicone was acquired by Mintlify, and the standalone product moved into maintenance mode (Experiments was deprecated in September 2025). For teams evaluating an active analytics product, Helicone is no longer the right fit; for gateway-only needs the OSS Rust gateway continues to ship. Moda is a different layer entirely — conversation analytics on top of whatever gateway you use. Read more: https://moda.dev/vs/helicone ## Moda vs LangChain LangChain is no longer just a framework. It now sells a full lifecycle suite — LangChain and LangGraph (OSS runtimes), LangGraph Platform (hosted runtime), Deep Agents, Fleet (visual agent design), and LangSmith (hosted observability with Insights Agent, Multi-turn Evals, and the LangSmith Engine for autonomous issue detection). Moda sits next to LangSmith specifically as a runtime-agnostic, conversation-semantic analytics layer with a prescriptive behavioral failure taxonomy. Read more: https://moda.dev/vs/langchain ## Moda vs CrewAI CrewAI is an OSS multi-agent framework and a managed platform — CrewAI AMP (Agent Management Platform). AMP includes a visual editor, AI Copilot, triggers, guardrails, a unified control plane, and native execution observability (LLM calls, tool calls, memory reads, cost). For deeper conversation analytics, CrewAI's docs route customers to third-party tools. Moda fits above AMP's execution telemetry with conversation-semantic analytics — intent clusters, behavioral failure modes, frustration root cause. Read more: https://moda.dev/vs/crewai ## Moda vs Letta Letta is an open, model-agnostic agent runtime organized around Memory Blocks and Context Repositories (git-backed memory). The product line now includes Letta Code (OSS coding agent, April 2026), the Letta Code SDK in TS and Python, and the Constellation managed cloud. The ADE is a developer tool for inspecting a single agent's state — not a production analytics surface. Moda sits above any Letta deployment to surface intents, behavioral failures, and frustration trajectories across the conversation population. Read more: https://moda.dev/vs/letta ## Moda vs AgentOps AgentOps ships agent-shaped observability — Time Travel Debug, Replay Analytics, multi-agent timeline visualization, cost tracking across 400+ LLMs, an OSS Python + TypeScript SDK, and enterprise compliance posture (SOC 2, HIPAA, NIST AI RMF). The unit of analysis is the session. Moda differs on the unit: Moda generates population-level intent taxonomies and surfaces behavioral failure modes and frustration root causes across thousands of conversations rather than one session at a time. Read more: https://moda.dev/vs/agentops ## Moda vs Arize Arize ships an agent-first observability platform — Arize AX (paid SaaS / Enterprise) on top of Phoenix (OSS). Recent feature work includes Sessions and Users, session-level evaluations, AI-driven cluster search for prompt-response clustering, heatmaps of underperforming slices, intent categorization that flags out-of-scope requests, and Alyx (an AI copilot across traces, evals, experiments, and prompts). It is the most directly overlapping product to Moda's wedge. Differentiation: Arize is a developer-first toolkit where you author evaluators and configure tagging; Moda ships a prescriptive behavioral failure taxonomy and frustration root cause with agent counterfactual automatically on ingest, for a product/CX/eng audience. Read more: https://moda.dev/vs/arize ## Moda vs Raindrop Raindrop is the most direct competitor — "Sentry for AI agents," $15M seed led by Lightspeed in Dec 2025. It ships default Signals (User Frustration, Hallucination, Refusal Spikes, Tool Failures, Context Loss, Infinite Loops) on top of trace and event capture, plus Topic Clustering, Trajectories, Issue Detection, custom signal authoring, and an open-source local debugger (Workshop, May 2026). The wedge against Moda is shape and frame: Raindrop is AI-native APM with custom-signal authoring as the primary workflow. Moda is the continual learning layer — production conversations turned into a learning loop with intent discovery, frustration root cause routed to a specific layer of the stack (prompt, tool, workflow, context, memory, model) with an agent counterfactual, and impact-ranked findings routed to Slack, Linear, CLI, and Ask. Read more: https://moda.dev/vs/raindrop ## Moda vs Trajectory Trajectory (Conviction-led $15M seed, May 2026; design partners Clay, Decagon, Harvey) is a continual learning data platform: an SDK that turns traces and telemetry into a standardized Trajectory primitive, then makes that data available for post-training and steering agentic models. Trajectory shares Moda's continual learning framing but sits at a different layer — it is the data plane for teams post-training their own models. Moda is the production-conversation analytics that surfaces what users want, where the agent fails, and which layer of the stack (prompt, tool, workflow, context, memory, model) needs to change. The two layers are complementary, not competitive. Read more: https://moda.dev/vs/trajectory ## Moda vs Traceloop Traceloop provides OpenLLMetry instrumentation, an OTLP span pipeline, and a hosted dashboard for traces, prompt management, and basic evaluators. Moda sits above that trace plumbing with conversation-semantic analytics: a prescriptive behavioral failure taxonomy, intent clustering, and frustration root cause with an agent counterfactual. Read more: https://moda.dev/vs/traceloop ## Moda vs HoneyHive HoneyHive is an evals-first platform for experiments, datasets, custom evaluators, prompt management, and production monitoring. Moda is harness-layer self-improvement with prescriptive behavioral failure detection and frustration root cause automatically on ingest, without requiring teams to author evaluators first. Read more: https://moda.dev/vs/honeyhive