Alternative

Moda vs Respan

For AI agents in production, Moda reads each run end to end and ranks problems by the users they reach.

Respan, formerly Keywords AI, is an open-source AI gateway with built-in observability and evals: one API key for more than 1,000 models, with fallbacks, load balancing, and cost limits, and every call traced. It was in Y Combinator's Winter 2024 batch, raised a $5 million seed led by Gradient Ventures in March 2026, and publishes its pricing with a free plan.

Moda at a glance

  • Every trace

    checked by built-in detections for frustration, tool failures, laziness, and hallucinations, with no evaluator to write and no sampling rate to set.

    How it works
  • Users affected

    ranks every problem, so the failure that reaches the most people comes first, with the traces behind it.

    How it works
  • SOC 2 and HIPAA

    SOC 2 Type I, SOC 2 Type II, and HIPAA compliance, with self-hosted deployment available.

For teams running agents in production

  • Cardboard
  • Octolane
  • Boardy
  • Pax Historia
  • Maywood
  • Final Round AI
  • Kanu
  • Brief
  • CoreLayer
  • Autumn
  • Codewisp

Replay

A state-of-the-art eval runner for production evals. No mocks to write.

Replay a prompt or model change against the traces that failed, before it ships. Moda turns those production traces into eval cases, plays each one under your current configuration and under the change, and scores both.

  • Your production traces are the test cases

    Each case comes from a real trace. A simulated user plays the scenario your user played, so the agent faces the conversation that actually went wrong, not a prompt someone wrote for a test.

  • Tool calls are simulated for you

    When the replayed agent calls a tool, Moda simulates the tool from the calls recorded in your production traces: the answer comes back in the recorded output format, and fails the way the real tool fails. You write no mocks and no fixtures, and no real tool is called.

  • Both sides judged on every case

    A judge scores every case under Production and under Proposed. The results show the pass rates side by side, and each case opens the full replayed transcript, tool calls included, so you can check a verdict against what the agent did.

In Respan, an experiment runs a prompt or model over a dataset of test inputs, built from a CSV or sampled from production spans and traces, and an evaluator scores the outputs. Simulations generate multi-turn scenarios from a prompt version, optionally grounded in production traces. For a prompt with tools, Respan returns the tool call and your code runs the function.

One replay, step by step
  1. 1Build an eval set from the production traces that failed
  2. 2A simulated user replays each conversation with your agent
  3. 3Every tool call is simulated from the tool calls recorded in your traces
  4. 4A judge scores Production and Proposed, case by case

Where Respan wins

When Respan is the better choice

You need an AI gateway

Respan's gateway gives one API key for more than 1,000 models, with fallbacks, load balancing, and cost and rate limits, and the gateway itself is open source (MIT or Apache-2.0) and free to self-host. Moda Inference, which sends model calls through Moda, is in early access for a small group of teams.

You want prompts served at runtime

Your app calls a Respan prompt by its ID, and deploying a new version changes the next call with no code change. Moda versions prompt files from your repository and is not in the request path.

You want to start today, on your own

Respan publishes its pricing and has a free plan with 100k logs and 1k scores; Team is $199 a month billed yearly or $249 billed monthly. Moda access is invite-only and pricing is custom.

You want a dataset and grader workbench

Respan evals combine datasets built from a CSV or sampled production spans, LLM-judge, code, or human graders, experiments, and annotation queues. Moda builds eval sets from production traces, replays them in the dashboard, and can post an evals check on your pull requests.

You alert in Microsoft Teams or through webhooks

Respan monitors alert by email, Slack, Microsoft Teams, or a webhook when a metric crosses a threshold. Moda alerts go to Slack, Linear, and email.

You need broad framework coverage

Respan's docs list tracing integrations for dozens of frameworks and providers, including the OpenAI Agents SDK, Vercel AI SDK, LangChain, LangGraph, LlamaIndex, and Mastra, and map GenAI, OpenLLMetry, and OpenInference attributes. Moda has Python and Node SDKs, the Vercel AI SDK and Claude Agent SDK, and OpenTelemetry.

Difference 1

Moda judges the whole agent run. Respan's behaviors score one span at a time.

What Moda does

A Moda trace is one complete agent run: every message, tool call, thinking block, and step, across turns. It can span many OpenTelemetry traces that share a moda.conversation_id.

Moda reads every production trace end to end and detects the failures that never throw an error: frustrated users, failing tools, lazy answers, and hallucinations. Add your own checks in plain language and get alerts in Slack, Linear, or email.

A hallucination is a claim the agent made that its recorded tool results contradict, such as a claimed success where the tool returned an error. There is no evaluator to write and no sampling rate to choose, and detections typically appear within minutes of ingest.

How Respan approaches it

Respan's behaviors are plain-language definitions scored on spans. Every new span is scored against each behavior that is on, and a behavior fires when its confidence is above 65%. The built-in behaviors are Refusal, Instruction miss, Incomplete answer, Structured output, and Task complete, and you add your own by describing them. Only new spans are scored, so editing a behavior does not rescore past spans. Behaviors need the Team plan or higher.

Online evals are separate: an automation runs an evaluator on live traffic that matches a condition, at a sampling rate you set. Respan suggests starting around 10% of traffic to keep cost down.

The same job in Respan and in Moda

In Respan

  1. 1Turn on behaviors, on the Team plan or higher
  2. 2Each new span is scored against each behavior
  3. 3Online evals sample live traffic at a rate you set

In Moda

  • One trace is the whole agent run, across turns
  • Claims are checked against recorded tool results
  • Built-in detections run on every trace

Difference 2

Moda ranks problems by users affected. Respan groups failed spans by error type.

What Moda does

Failures are grouped into problems ranked by how many users they affect. Each problem shows the traces behind it, the turn where it went wrong, and related problems that share the cause.

New traces are attributed to existing problems within minutes, and a problem you marked fixed reopens if the failure comes back. Problems can be filed in Linear, where status syncs both ways.

How Respan approaches it

Respan's error tracking needs no setup once traffic flows: failed spans are grouped into error groups by error class, provider, endpoint, and HTTP status, and Respan opens an incident when error rates spike. An error group can be split by customer to see who is affected.

Behaviors appear in the signals panel with their share of spans, count, and change from the previous period, and each one can be split by model, customer, provider, or prompt.

The same job in Respan and in Moda

In Respan

  1. 1Failed spans are grouped by error class, provider, and status
  2. 2An incident opens when error rates spike
  3. 3Behaviors are listed by share of spans

In Moda

  • Failures are grouped into problems as traces arrive
  • Problems are ranked by users affected
  • A fixed problem reopens if the failure returns

Side by side

Capability by capability

The same rows appear on every Moda comparison page, including the ones where Respan is ahead.

Moda vs Respan, capability by capability
CapabilityModaRespanWhy it matters
Setup effortSDKs or OpenTelemetry: moda init has your coding agent add an SDK, or point an existing OpenTelemetry exporter at Moda. The built-in detections run on traces as they arrive.SDKs, OpenTelemetry, or the gateway, which traces every call it routes. Behaviors are on by default on the Team plan and above; online evals need an evaluator, a condition, and a sampling rate.Time to the first useful answer about your agent, not just the first stored trace.
Failure detectionBuilt-in signals on every trace: user emotion and frustration, tool failures, laziness, and hallucinations (replies the trace shows are false). Custom signals in plain language.Behaviors scored per span: five built-ins (Refusal, Instruction miss, Incomplete answer, Structured output, Task complete) and your own in plain language, on Team and above. Automatic provider and API error tracking. Online evals on a sampled share of traffic.Whether failures that never throw an error are caught on the traces nobody reads, and how much setup that takes.
Grouping and ranking failuresFailures grouped into problems ranked by how many users they affect, with the traces behind each one.Failed spans grouped by error class, provider, endpoint, and status code, with incidents on spikes; behaviors listed by share of spans and split by customer, model, provider, or prompt.Decides what your team works on first.
Testing a change before it shipsReplay a prompt or model change against traces from production and compare pass rates with the current configuration. Tool calls are simulated from the tool calls recorded in those traces, so there are no mocks to write.Experiments run a prompt or model over a dataset from a CSV or sampled production spans, scored by evaluators; simulations generate multi-turn scenarios from a prompt version; the playground compares prompts across models.Shows whether a change helps on the cases that actually failed before users see it.
Offline evals and CIReplays of a change against traces from production, run and reviewed in the dashboard. Moda can also post an evals check on your pull requests: each push runs the eval sets you choose and reports pass or fail, with a link to the run.Datasets, LLM-judge, code, and human graders, experiments, and annotation queues.Matters most when your release process gates on eval scores in CI.
Alert destinationsSlack, Linear, and email.Email, Slack, Microsoft Teams, or a webhook, when a metric crosses a threshold.Alerts should land where your team already triages work.
SDKs and frameworksPython and Node SDKs; Vercel AI SDK and Claude Agent SDK; any OpenTelemetry source; HTTP API.Python and TypeScript SDKs; framework integrations including the OpenAI Agents SDK, Vercel AI SDK, LangChain, LangGraph, LlamaIndex, and Mastra; any language through the gateway.Less glue code for the languages and frameworks you already run.
OpenTelemetryOTLP/HTTP trace endpoint (JSON or protobuf). Reads OpenTelemetry GenAI, OpenLLMetry, and Vercel AI SDK attributes, and groups spans across OpenTelemetry traces by moda.conversation_id or gen_ai.conversation.id.OTLP/HTTP (JSON or protobuf) at /api/v2/traces. Maps GenAI, Traceloop (OpenLLMetry), and OpenInference conventions.One instrumentation can feed more than one tool, so trying a second one costs a config change.
Prompt managementPrompt files versioned from your repository, with release labels and per-version attribution of production calls. Not in the request path: the SDKs do not fetch prompts.Prompts served through the gateway by ID; deploying a new version changes the next call with no code change.Decides whether prompts ship with your code or are served to your app at runtime.
Open source and self-hostingSelf-hosted deployment available; contact the team to set it up. Not open source.The gateway is open source (MIT or Apache-2.0) and free to self-host; the full platform self-hosts on Enterprise.Often required when trace data cannot leave your own infrastructure.
Security and complianceSOC 2 Type I, SOC 2 Type II, and HIPAA compliance.SOC 2 Type II audited. HIPAA, with a BAA, is a $500-a-month add-on on Team and included on Enterprise.Often a procurement gate before any trial starts.
Pricing and free tierInvite-only, with custom pricing. Request access to get a workspace.Public pricing. Free plan with 100k logs and 1k scores; Team at $199 a month billed yearly or $249 monthly; Enterprise custom.Decides whether you can start today without talking to anyone.

Try it

Try Moda next to Respan

Respan's gateway can keep routing your model calls while Moda monitors them. The Moda SDK for Python or Node.js patches the OpenAI and Anthropic client libraries with one init call, so a client pointed at the gateway's URL is still captured, with its streamed responses and tool calls.

If your spans already reach Respan through an OpenTelemetry Collector, add a Moda exporter next to Respan's. Moda ingest accepts uncompressed request bodies only, so the Moda exporter sets compression: none. Set moda.conversation_id and moda.user_id on every span of a multi-turn agent, so each run stays one trace under the right user.

otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc: {}
      http: {}

processors:
  batch: {}

exporters:
  # Your existing Respan exporter, unchanged.
  otlphttp/respan:
    traces_endpoint: https://api.respan.ai/api/v2/traces
    headers:
      Authorization: Bearer ${env:RESPAN_API_KEY}
  otlphttp/moda:
    traces_endpoint: https://moda-ingest.modas.workers.dev/v1/traces
    headers:
      Authorization: Bearer ${env:MODA_API_KEY}
    # Moda ingest rejects compressed request bodies; the Collector gzips by default.
    compression: none

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/respan, otlphttp/moda]

Questions

Frequently asked questions

Respan vs Moda: what is the difference?

Respan, formerly Keywords AI, is an AI gateway with tracing, behaviors, and evals built around it. Moda is production monitoring and evals for AI agents: it reads each agent run end to end, detects frustration, tool failures, laziness, and hallucinations on every trace, ranks problems by how many users they affect, and replays a prompt or model change against the traces that failed. Respan leads on its gateway, runtime prompt serving, public pricing, and its dataset and grader workbench.

Do I need a gateway to use Moda?

No. Moda reads traces from its SDKs, OpenTelemetry, or its HTTP API, so it does not need to sit in front of your model calls. If you route calls through Respan's gateway, keep it: the Moda SDK captures calls made through the OpenAI and Anthropic clients wherever they point.

Does Moda detect prompt injection and jailbreaks?

Not as built-in detections. Moda's built-in detections cover user emotion and frustration, tool failures, laziness, and hallucinations checked against recorded tool results. You can add a check for prompt injection or anything else as a custom signal, written in plain language and run on every trace.

Can I run Moda and Respan at the same time?

Yes. Add the Moda SDK next to the gateway, or, if your spans already go through an OpenTelemetry Collector, add a Moda exporter with compression: none beside your Respan exporter. Set moda.conversation_id and moda.user_id on your spans.

What does Moda cost?

Moda access is invite-only and pricing is custom. Request access at moda.dev/signup, or email founders@modaflows.com for a quote. Respan publishes its pricing and has a free plan.

Can I self-host Moda, or is it open source?

Moda offers self-hosted deployment; contact the team at moda.dev/contact to set it up. Moda is not open source. Respan's gateway is open source, and its full platform self-hosts on the Enterprise plan.

What security compliance does Moda have?

SOC 2 Type I, SOC 2 Type II, and HIPAA. Respan says it is SOC 2 Type II audited and offers HIPAA with a BAA as an add-on on its Team plan.

Try Moda

Infrastructure for catching AI agent mistakes in production and turning them into evals.