Production field guide
Detecting user frustration and sentiment in AI agent traces
Most unhappy users never click thumbs-down. They repeat themselves, ask for a person, or quietly leave. Here is how to find those reactions in your traces, count them against a denominator you can defend, and check whether the people behind them come back.
Constructed example · no customer data
One trace. Three negative rates.
Change what the rate counts. The transcript stays the same.
Task 1: change the shipping address
User
Can you change the shipping address on order A-88213 to 22 Park Lane?
Agent
Done. Order A-88213 now ships to 22 Park Lane.
User
Great, thanks.
gratitude
Task 2: refund a duplicate charge
User
I was charged twice for that order. Can you refund one of the charges?
Agent
I can help with that. Which order is this about?
User
The same one, A-88213.
A restated request with no reaction to having to restate it. Not frustration on its own.
Agent
I could not find a duplicate charge on that order.
User
I already told you, there are two $49 charges on my card. Can I talk to a person?
exasperationescalation request
Agent
I understand. Is there anything else I can help with?
User
Forget it.
resignation
Negative rate
33%
2 of 6 user turns had a negative detection
Each user message counts once. It shows which replies set users off, and long traces full of neutral turns pull it down.
Why thumbs-down and CSAT miss most frustration
Explicit feedback is rare. In a 2025 study of a developer assistant, explicit ratings covered about 200 turns, 0.6% of interactions, spread across about 2% of conversations. Reading sentiment from what developers actually typed found a signal in about 8% of interactions, more than 13 times the explicit rate (Nam, Salawa and Chandra, 2025).
It is also lopsided. In Bing Copilot conversations, Microsoft researchers found that users “give more positive feedback through thumb feedback and more negative feedback through their utterances” (Lin et al., 2024). So a thumbs-down rate undercounts the thing you most want to see.
Surveys have the same gaps, later. A CSAT or NPS survey arrives after the work is over, reaches the people who choose to answer, and returns one number per respondent. None of that tells you which turn went wrong.
Keep the buttons and the surveys. They are the ground truth you will calibrate against. But the unhappy users who never click anything are still in your traces, in their own words.
What frustration looks like in a transcript
Users rarely write “I am frustrated.” A team studying a deployed task-oriented assistant found that frustration in real traffic “is frequently conveyed through context and repetition rather than explicit wording” (Hernandez Caralt et al., 2024). These are the patterns worth reading for.
Repetition and rephrasing
A user restates a request when the first answer missed. Search engines learned this early: a similar query issued within about five minutes implies dissatisfaction (Awadallah et al., 2013). Microsoft’s satisfaction rubric for Bing Copilot lists the same pattern: “The user repeats their query or request multiple times.”
Repetition alone is weak evidence, though. People also restate to clarify, or because the agent asked them to. The signal gets strong when the user reacts to having to repeat: “I already told you,” “for the third time,” “why can’t you just.”
Escalation requests
“Can I talk to a person?” “Transfer me.” “Cancel my subscription.” These are explicit, easy to detect, and often the last thing a user says before leaving. Record what prompted the request. A user asking for a human because policy requires one is not the same as a user giving up on your agent.
Abandonment and resignation
Some users say it: “never mind,” “forget it,” “I’ll just do it myself.” Others just stop. The trace ends on an agent message that did not resolve the task, and the user does not come back to it.
Silence has no words to classify, so detect it from the trace’s shape instead: the last message is the agent’s, the task has no resolution, and no follow-up arrives. Be careful here. A user who got exactly what they needed also stops talking. Pair abandonment with evidence about the outcome before you call it a failure.
Sarcasm and polite frustration
“Wow, really helpful.” “Great, another error.” A word list scores both as positive. Sarcasm is hard for models too: across six sarcasm benchmarks, current LLMs underperformed supervised baselines built on pre-trained language models (Zhang et al., 2024).
Polite users are harder still. “Thanks anyway” after a failed refund is not gratitude.
Confusion and distrust
Not every negative reaction is anger. “What do you mean?” is confusion. “You’re making that up” and “I’ll check that myself” are distrust. They point at different fixes: confusion usually means the agent explained badly, and distrust often means it said something the user knew was wrong. Keep them as separate families instead of folding everything into one sentiment score.
What frustration is not
Terse commands, plain edit requests (“make it shorter”) and a user’s mood about something else are not frustration with your agent. A calm “no, not that one” rejects the output without any heat; it is worth tracking, but separately from frustration. A user venting about their bank is upset, and your agent may be handling it well. Track who or what the emotion is aimed at, or your rate will measure your users’ week instead of your agent.
Detection approaches and their tradeoffs
There are four common ways to find these reactions. Most production setups combine two or more.
Keyword and regex rules
Match profanity, “talk to a human” and a list of known phrases. Rules are cheap, fast and easy to explain. They are also nearly blind. In the deployed system studied by Hernandez Caralt et al., keyword matching scored a precision of 1.00 and a recall of 0.01 on frustrated conversations: everything it flagged was real, and it found almost nothing.
Use rules as a guardrail for escalation phrases. Do not use them as your frustration rate.
Off-the-shelf sentiment classifiers
A classifier labels each message positive, neutral or negative. It reads tone well enough. It does not know what the tone is aimed at, it misses sarcasm, and it cannot tell that “The same one, A-88213” is a second ask. The same study found that open-source sentiment and dialog-breakdown methods showed limitations on real-world frustration.
Behavioral signals from the trace
Count rephrased requests, repeated tool calls with the same result, escalation, short gaps between user turns, traces that end early and reopened tasks. These signals need no model calls and work in any language. Each one is ambiguous alone. They are most useful as context for the words, and for ranking which traces to read first.
Language-model judges that read the conversation
Give a model the user’s turn and the conversation before it, and ask for labels from a fixed taxonomy, each backed by a verbatim quote. This catches context-dependent frustration that the other methods miss: in the same study, the LLM-based approach achieved a 16% relative improvement in F1 on the team’s internal benchmark.
It has costs:
- Tokens. Every user turn is read with its context.
- Latency. Judging is usually asynchronous, after a trace goes quiet, so results lag the conversation.
- Instability. Scores can move when the prompt changes in ways that should not matter. In our own test of twelve decision systems on 1,000 text samples, changing only the order of the emotion choices often changed the emotion scores.
Reduce the risk: fix the taxonomy, require a quote copied from the user’s message, record a confidence, pin the prompt and model version, and compare against human labels whenever either changes.
Calibrate before you alert
Label a sample of traces by hand before you trust any detector. Include ordinary successes, positive reactions and ambiguous cases. Then measure two things for each signal:
- Precision: of the traces flagged, how many did a reviewer confirm? This sets how much review time an alert costs.
- Recall: of the frustrated traces in a random sample, how many did the detector catch? Reviewing only flagged traces cannot measure what it missed.
Check each family separately. Sarcasm and resignation fail in different ways, and an average hides both.
How to measure frustration per turn, task and trace
A negative rate is a numerator over a denominator. The denominator is a choice, and it changes the number a lot. The constructed trace at the top of this page has one bad task in it. Its negative rate is 33% per user turn, 50% per task and 100% per trace.
Per user turn
Each user message counts once. This is the best unit for asking which agent replies set people off, and for comparing two prompt versions on the same traffic. Long traces with many neutral turns pull it down, so a chattier product can look calmer without being better.
Per task
A task counts once if any of its turns carried a negative reaction. This is closest to what most teams mean by “how often does a request go badly.” It needs each trace split into the tasks the user asked for. Research on intelligent assistants supports measuring here: “overall task-level satisfaction cannot be reduced to query-level satisfaction alone” (Kiseleva et al., 2016).
Per trace
A trace counts once if anything in it was flagged. It is the simplest unit to compute. It also rises with trace length, and one bad moment in a long, mostly successful run marks the whole trace negative.
Affected users
Count distinct users with at least one negative reaction in the window. Volume rates can be dominated by one heavy user with forty bad traces. Affected users tells you how many people had a bad time, which is usually what you should rank problems by. It needs a stable user ID on every trace.
Keep the denominator attached
Report the rate with its unit, window and coverage, as in “per analyzed user turn, last 7 days.” If you analyze a sample of traces, say so. A shift in traffic mix can move an aggregate rate without any change in the agent, so compare by use case and agent version before blaming a model or prompt. Count positive reactions too. A release that halves frustration and also halves thanks may just be getting fewer engaged users.
Detecting NPS-style signals in agent traces
Net Promoter Score comes from one survey question: “How likely are you to recommend us to a friend or colleague?” on a 0 to 10 scale. Respondents who answer 9 or 10 are promoters, 7 or 8 are passives, and 0 to 6 are detractors. NPS is the percentage of promoters minus the percentage of detractors (Bain & Company).
A trace cannot contain an NPS. NPS is a person’s stated answer to a question about recommending you, and most of your users will never be asked it. What a trace does contain is expressed reactions: thanks, approval, enthusiasm, and every negative family above. That supports three NPS-style uses.
- A net expressed-sentiment trend. Take the share of tasks with a positive reaction and subtract the share with a negative one. The result runs from -100 to 100, which makes it look like NPS. It measures what users said in the moment, though, so give it its own name on the dashboard and never call it NPS.
- Calibrating against the survey you already run. Join each NPS response to the respondent’s traces from the weeks before they answered. If detractors show no more frustration than promoters, your detector or your taxonomy is missing whatever drives the score.
- Choosing whom to ask. Users with sustained frustration and no resolution are the people to interview or survey next, while they still remember why.
Positive reactions need the same care as negative ones. A reflexive “thanks” is often politeness. Approval of a specific result (“perfect, post it”) is stronger evidence than a sign-off.
How to track AI agent retention
Retention answers one question: do users come back? For an AI agent, define it from traces.
- Active: a user sent the agent at least one message in the period. Leave out traffic with no human user ID, such as scheduled jobs and API callers.
- Cohort: group users by the week they were first seen.
- Week N retention: the share of a cohort active in the Nth week after their first week. Mixpanel distinguishes “On” retention, active in exactly that period, from “On or After,” active then or any time later (Mixpanel docs). Say which one you report.
Both depend on a stable user ID on every trace. Anonymous traffic can be neither retained nor counted as an affected user.
Connect frustration to retention
- Split each signup cohort by whether the user had a negative reaction in their first week.
- Compare week-4 retention for the two groups.
- Compare within a use case. Frustrated users may simply be the ones attempting harder tasks, and harder tasks may have different retention for reasons unrelated to frustration.
- Look at which reactions sit at the end of traces. Resignation or an escalation request with no resolution is a stronger warning than frustration that the agent recovered from.
Treat the result as correlation. We could not find a published study that measures how much in-conversation frustration predicts churn for AI agents, and we have not published one either. Run the comparison on your own users before you put a churn number in a slide. Retention takes weeks to show up in a cohort. Frustration shows up in the trace, early enough to act on.
How Moda detects user emotion in traces
Moda is infrastructure for catching AI agent mistakes in production and turning them into evals.
Moda’s User Emotion signal reads each user turn with the earlier conversation as context and labels emotion expressed in the user’s own words. It uses a fixed taxonomy of seven families and 17 signals:
- Frustration: exasperation, anger, profanity, insult, sarcasm, escalation request
- Sadness: disappointment, resignation, self-blame, personal distress
- Confusion: confusion
- Anxiety: urgency
- Trust: distrust
- Redirection: redirect, a calm rejection of something the agent produced (“no, not that one”) with no annoyance in it
- Positive: gratitude, satisfaction, delight
The first five families are negative. Redirection is neither negative nor positive, so it never counts toward the negative rate.
Each detection records what elicited the emotion (the agent, something external, the user, or ambiguous), its trajectory across the trace (none, building, peak, resolved or sustained), a 0 to 1 confidence, a risk score, a primary cause and verbatim user quotes. Each quote links into the trace at the quoted turn.
The User Emotion tab on the Signals page reports two figures for the selected date range:
- Negative rate: the share of units with at least one negative-family detection. The unit is a user turn by default, and you can switch it to a task or a trace.
- Affected users: users with at least one negative reaction in the window.
Below them, a strip counts detections per family, including Redirection and the positive family. Selecting a family filters the list of traces.
Emotion analysis runs once a trace has been quiet for about 30 minutes, and at least hourly on a trace that keeps going. It is not real-time. If your workspace is set to analyze a sample of its traces, emotion analysis covers only that sample, so the rates describe the sample.
From there:
- Alerts. Alert on the built-in Negative user emotion signal, or on a single family such as Frustration, in Slack, Linear or email.
- Problems. Negative signals are grouped into problems ranked by how many users they affect, with the traces behind each one.
- Retention. The Users page shows retention by signup week: users grouped by the week they were first seen, and the share of each cohort with a trace in every later week. Traffic without a user ID is left out.
- Programmatic access. Read detections with
GET /emotionsandGET /frustrationsin the Data API, ormoda detections list --family=frustrationin the CLI.
Your team makes the change. Replay a prompt, model or tool change against the exact traces that failed and compare pass rate, latency and tokens, then keep watching the negative rate after release. The Signals docs and Analysis docs describe each field.
Frequently asked questions
Can I detect user frustration without an LLM?
Partly. Keyword rules and behavioral signals such as repeated requests and escalation phrases are cheap and precise, but they miss most frustration because users express it through context. In one deployed system, keyword matching found about 1% of frustrated conversations. Use them as a first pass and add a model that reads the conversation for recall.
Is sentiment analysis the same as frustration detection?
No. Sentiment analysis labels the tone of a message. Frustration detection asks whether the user is reacting badly to the agent, which depends on what came before and on who the emotion is aimed at. A neutral restated request can be the strongest sign of frustration in a trace.
Should a thumbs-down count as frustration?
Count it as a separate, high-precision signal. Thumbs ratings are rare and skew positive, so a low thumbs-down rate does not mean users are happy. Use the thumbs-downs you do get to calibrate detectors that read the transcript.
Can you measure NPS from agent traces?
Not directly. NPS is a survey answer about likelihood to recommend. Traces can give you an NPS-style trend from expressed positive and negative reactions, and you can join survey responses to traces to check whether detractors show more frustration.
Does a frustrated user mean the agent failed?
Not always. Users get frustrated at correct refusals, at their own mistakes, and at things outside the agent. Record what elicited the emotion and check the outcome before treating a detection as a failure.
How quickly does Moda flag a frustrated user?
Moda analyzes user emotion once a trace has been quiet for about 30 minutes, and at least hourly while a trace keeps going. Results then appear on the Signals page, in alerts and in the Data API.
A checklist for your first frustration review
- Attach a stable user ID to every trace, so affected users and retention can be counted.
- Pick a taxonomy with separate families for frustration, confusion, distrust and resignation, plus positive reactions.
- Require every detection to quote the user’s own words.
- Label a random sample by hand and measure precision and recall for each family.
- Choose the rate’s unit on purpose and report it with its window and coverage.
- Rank what you find by affected users, not detection volume.
- Join survey and thumbs feedback to traces to check your detector against what users told you directly.
- Compare signup-cohort retention for users with and without early frustration, within the same use case.
Start with the traces where a user asked the same thing twice. Read what the agent said in between.