Evals for GTM agents: how to know your fleet is working
On this page
You know your fleet is working when it passes evals you designed, and on no other evidence. AI agent evaluation for GTM comes down to five instruments: backtests against your own CRM history, sampled reads of decision traces, guard metrics behind every volume number, regression suites for model swaps, and calibration reviews for predictions. Observability shows what agents did. Evals decide whether it was good, and whether autonomy expands or contracts next quarter.
The industry runs the first half of that sentence and skips the second. LangChain's State of Agent Engineering survey (June 2026) found 57 percent of teams have agents in production, rising to 67 percent at enterprises above 10,000 employees, and quality is the top blocker they name. Yet while 89 percent run observability, only 52.4 percent run offline evals. Most fleets are watched and untested. RevSure's own 306-leader study shows the same posture in GTM specifically: 76 percent are deploying or implementing agentic AI, and 41 percent have implemented. In the same study, 47 percent cite lead quality and data reliability as their top barriers, which are exactly the worries evals exist to retire.
Everyone watches, few test
Sources: LangChain, State of Agent Engineering, Jun 2026 · RevSure, The 2026 State of Agentic AI in B2B GTM
Trust arrives on a schedule
Revenue leaders already run an informal eval program in their heads. A marketing analytics advisor described how predictions earn standing with an operating team:
you go on a first date, it sounds great, but it's only after you've known someone a while that you start to have some confidence.
a marketing analytics advisor
That is the trust gradient, and it is correct. The problem is that most teams leave it informal, so trust accrues to whichever agent had a lucky month and evaporates on the first visible miss, with nobody able to say what the miss rate actually was. The playbook below formalizes the first-date instinct into five practices, so that autonomy is granted on evidence and withdrawn on evidence, per agent, per action class.
Formalizing it also changes the conversation upward. A fleet whose autonomy map cites backtest scores and guard cohorts can defend itself in a QBR. A fleet running on impressions is one bad anecdote away from a freeze. The eval record is what converts an agent program from an experiment somebody sponsors into infrastructure somebody budgets.
Practice 1: backtest against your own CRM history
Before an agent touches a live record, replay it against your past. Your CRM history is a golden dataset you already paid for: several quarters of known outcomes, with the answers in the back of the book. The eval question is concrete. Would the agent have caught what happened?
The published RevSure example is the standard to hold: a team caught a $3M revenue deficit 90 days early and saved hundreds of thousands of dollars by reallocating in time. Run the equivalent question against a candidate agent: given the data as it stood at day 1, would it have flagged the deficit your team discovered at day 90? Would the Deal Risk agent have flagged the deal that slipped last quarter, while it was still recoverable? Would a Campaign Reallocation agent have proposed the budget move your team eventually made by hand, and how many weeks earlier? Score each answer on two axes: did the agent catch it, and how early. The second axis is where agents earn their keep, because your team eventually caught most of these by definition. Earlier is the product.
Backtests also expose the timing failure that plagues predictive systems. One customer success software company watched a pipeline model land on the correct number, $26.69M, only on the last day of the fiscal quarter. A backtest scored on final accuracy would grade that model an A. A backtest scored on when the model got within range grades it honestly, and timing is the score that matters, because a forecast that converges after decisions are locked converges for the historians.
The practical barrier is usually data assembly rather than method, and it is lower than teams expect. One published day-one story: 7 data sources connected in under 30 minutes. The history is sitting there.
Two design notes keep backtests honest. Replay with the data as it stood on the decision date, never with today's cleaned-up record, or you hand the agent hindsight and grade it as skill. And score against moments rather than averages: five decisive misses your team still remembers are worth more as test cases than a thousand routine weeks, because the misses are what you are buying insurance against.
Less effective: judging a new agent on a two-week live pilot, where volume is thin and nobody agrees afterward on what would have happened anyway.
Recommended: replaying four quarters of your own CRM history and scoring the agent on whether it caught the misses your team remembers, before it earns any write access at all.
Practice 2: read the traces
Error analysis is the core habit of the eval canon in AI engineering: sample outputs, read them closely, tag the failure modes, fix the biggest one, repeat. The GTM version is decision-trace review. Every week, pull a fixed sample of agent decisions, read the recorded reasoning end to end, and log what you find in a running taxonomy: wrong data read, right data misread, right conclusion for a wrong reason, right proposal at a wrong time.
The wrong-reason cases are the ones only trace reading catches. An agent can reallocate budget toward the channel a human would have chosen, for reasons that will produce the opposite call next month. Outcome metrics stay green while the reasoning drifts. This is why review has to happen at the action level, on a schedule, rather than as a reaction to visible failures.
Salesforce's newsroom documented the discipline at Asymbl, which runs weekly line-by-line performance reviews across roughly 200 digital workers working beside 170 humans, on the blunt grounds that "thumbs-up and thumbs-down buttons won't get the job done." The cadence is the point. A trace review that happens when someone gets around to it is an alibi, and one that happens every Thursday is an eval.
What the log should capture per sampled decision: the action class, the verdict (correct, correct for wrong reasons, wrong, wrong but caught), the failure tag, and whether the approval queue would have stopped it. Where this goes wrong: the read decays into skimming once agents feel reliable, and the sample quietly shrinks to the decisions that already look interesting. Fix the sample size in advance and draw it randomly. Boring weeks are the point. They are what reliability looks like when it is real, and the log is what proves the boredom was earned.
Practice 3: put a guard metric behind every volume number
Agents move volume metrics first, because volume is the easiest thing in a funnel to manufacture. The eval discipline is to pair every volume number an agent can move with a quality cohort it cannot flatter: meetings with meeting-to-opportunity conversion, MQLs with stage progression. A marketing analytics leader at a cloud security company gave the reason in five words: "Click and impressions are... a false positive."
Guard-metric monitoring is the eval that runs continuously between reviews. The volume metric tells you the agent is active. The guard cohort tells you whether the activity is worth anything, and it is the number that should gate autonomy. An agent whose meetings double while meeting-to-opportunity conversion holds has earned wider limits. An agent whose meetings double while conversion halves has automated a cost center.
A published deployment shows what passing looks like. An enterprise security company ran 120,000 personalized emails across 40,000 leads through an agent chain with zero additional headcount, and the guard cohorts moved with the volume: lead-to-MQL conversion rose 25 percent and median time to MQL fell 15 percent, while open rate climbed 40 percent. Volume with guards holding is the passing grade. Volume alone is unmeasured cost.
The end state for teams that skip this practice is already documented in the field. A marketing leader at a cybersecurity company moved away from lead scoring entirely "because there wasn't confidence in how the scores were adding up." Nothing was wrong with the ambition. The scores shipped without a guard cohort or an explanation, trust broke once, and the program died. Guards are how scoring programs survive their first bad month.
Practice 4: regression-test every model swap
RevSure customers choose the LLM per agent, and the published price list explains why swaps are a permanent fact of fleet operations: Gemini 3.1 Flash Lite runs $2.50 per million tokens, Gemini 3.0 Flash runs $17.50, and Gemini 3.1 Pro runs $70. That is a 28x spread for the same seat in the same workflow. A single personalized email consumes about 31,000 tokens, and agent runs meter at $0.25 per run, so finance will eventually ask, correctly, whether the expensive model is earning its premium on every agent that uses it. The per-workflow arithmetic sharpens the question: at published credit rates a personalized email runs about $0.40 all-in and a deep account research brief about $0.50, so model choice moves real margin once the fleet is at volume.
The price spread that makes model swaps inevitable
Source: RevSure published pricing, 2026, per 1M tokens; customers choose the LLM per agent
The question is healthy. Answering it without a regression suite is how a cost optimization becomes a production incident. A model swap changes the reasoning engine inside a running agent, and behavior that held for two quarters can shift in ways no dashboard flags until a guard metric sags weeks later.
The suite itself is modest: a frozen set of real, redacted cases with agreed-correct behavior, the golden dataset from Practice 1 doing double duty. Run every candidate model against it, diff the decisions, and gate the swap on the results.
Less effective: upgrading the model behind a live agent because the new one benchmarks better or prices lower, then watching production and waiting for complaints.
Recommended: running the frozen case set against the new model first, then shipping the swap only when it matches or beats the incumbent on your cases. Anything else rolls back.
Practice 5: calibrate for earliness
Predictive agents need a different scoring rule than the one most teams inherit from finance, where accuracy at close is what counts. The field standard came out of a working session on forecast calibration: "if you're hitting 85 to 95% accuracy in the early days of the quarter, that is exactly what you want." An 85 percent answer at day 15 is actionable. A 99 percent answer at day 85 is a press release. The $26.69M model from Practice 1 is the cautionary case: perfectly right, perfectly late.
Calibration reviews should also score volume and value separately, because they fail separately. A revenue operations leader at a contract lifecycle management company, asked where his projections hold up: "Historically we've been more accurate on a volume perspective than the value perspective... the volume perspective is slightly more accurate usually." A fleet whose deal-count predictions are trustworthy while its deal-size predictions wobble should route decisions accordingly, and only a calibration review broken out by dimension reveals the split. Route accordingly means exactly that: let volume-calibrated predictions drive early-quarter coverage conversations, and hold value-weighted calls for reviews where a human can interrogate the deal list behind them.
The quarterly ritual, then: for each predictive agent, plot predicted against actual by week of quarter, find the day predictions settled within range of the eventual truth, and manage that date down. Convergence timing is the KPI. Endpoint accuracy is trivia. Publish the convergence date next to the forecast itself, because an executive who knows the number firms up around day 20 reads day-10 output as directional rather than broken.
When to distrust a score
The propensity story is the one to keep taped above the eval program. A model scored 20,000 leads and put 99 percent of them under 5 percent likelihood. The marketing operations leader at a cybersecurity company looked at the distribution and refused to write the scores into the CRM:
I couldn't trust those numbers... it's going to really make everybody feel, should we trust this or not?
a marketing operations leader at a cybersecurity company
The score that did not ship
What the model produced
What the team did
Source: RevSure working sessions with an enterprise cybersecurity team, descriptor level
Read what he actually did, because it is the whole discipline in miniature. He compared the distribution against what any working funnel looks like, then weighed the blast radius: scores written into a CRM shape rep behavior for quarters. The release stopped there. The refusal was the eval. And his framing names the real stake: one implausible score sheet in a system of record puts every future number the platform produces on trial.
A marketing analytics leader at an enterprise AI search company explained why the gate has to be a person with context, at least until the checks are encoded:
the reason that this works for somebody like me is because I have context about the data. Somebody else can just take this at face value.
a marketing analytics leader at an enterprise AI search company
Face value is the failure mode. The eval program's job is to institutionalize that leader's skepticism so it still runs the week she is on vacation: distribution checks on every score release, plus a hard rule that scores reaching a system of record pass a review first. Evals exist so the should-we-trust-this moment happens in a staging environment, never in front of a sales team.
The trust math is asymmetric, which is why the gate earns its cost. A score sheet that ships and embarrasses itself once costs quarters of adoption, the way the abandoned scoring program above cost a company the whole practice. A score sheet held back for review costs a week. Every instrument in this playbook is cheap against the price of the first public miss.
What this looks like when the trace is native
Every practice above can be run with exports and a spreadsheet, and the first teams ran them exactly that way. The RevSure design position is that evals belong in the substrate. Each agent decision on the platform records a Decision Trace, so the weekly read is a sampling exercise instead of a reconstruction. The GTM Harness loop, Propose, Approve, Commit, Roll back, turns review into a workflow with an undo, which is what makes a failed regression test recoverable. Guard cohorts compute on the Full Funnel Data Graph, the same substrate the volume numbers come from, so a guard and its target can disagree for real reasons instead of definitional ones. And because the platform lets customers choose the model per agent, the regression suite in Practice 4 is treated as a first-class operating asset rather than an integration afterthought. Calibration reviews run the same way against the Predictive AI Engine's outputs: predictions plotted against actuals by week of quarter, convergence dates tracked over time, and the volume and value dimensions scored separately, because the field record says they fail separately.
One honest limit from our own field record. Eval discipline consumes real attention, and it can curdle into configuration theater. An operations leader at a contract lifecycle management company said it about their own rollout: "we got too focused on configuring the system correctly, but we are not really able to use the system in our day-to-day right now." An eval program with five instruments and no operating rhythm is that failure with better vocabulary. Start with one backtest and one weekly trace read, prove the loop, then widen.
What to do next quarter
Six moves, in order of dependency.
- Assemble the golden quarters. Export four quarters of CRM history and mark the moments that mattered: the slipped deals, the late-discovered deficit, the reallocation someone eventually made by hand, the forecast that converged after it stopped being useful.
- Backtest every agent that wants write access, and score timing, asking when it would have caught each miss, not only whether.
- Put the weekly trace read on the calendar: a fixed sample, one named reader, a running failure taxonomy, and a log the next QBR can cite. Thirty minutes, every week, no exceptions.
- Pair every volume KPI an agent can move with a guard cohort it cannot flatter, and gate autonomy on the guard.
- Build the regression case set now, before the first model-swap request arrives from finance. It will arrive.
- Add a distribution check in front of every score write-back. Any release that fails it stops at the gate, the way the best operators already stop them by hand.
Where this comes from
Built from our work inside enterprise GTM teams: indexed, verbatim working sessions with marketing, RevOps, and revenue leaders, quoted here at descriptor level with permission discipline. Third-party figures are cited to their publishers with dates. One limit worth stating: backtests grade agents against decisions your team already made or eventually caught, so they reward imitation of the past and cannot score the move nobody tried. Treat history as the floor for trust, and keep a budget of supervised live trials for the decisions that have no precedent in your CRM.
Frequently asked questions
What is AI agent evaluation?
AI agent evaluation is the discipline of testing whether an agent's decisions are good, separate from watching what it did. It covers offline evals against historical data, sampled review of decision traces, guard metrics behind volume numbers, regression tests when the underlying model changes, and calibration checks on predictions. LangChain's 2026 survey found 89 percent of teams run observability while only 52.4 percent run offline evals.
How do you evaluate a GTM agent before trusting it?
Backtest it against your own CRM history. Your past quarters form a golden dataset with known outcomes, like the deal that slipped or the deficit that surfaced too late to fix. Replay the agent against that record and score whether it would have caught what actually happened, and how early. Autonomy then expands as observed accuracy accumulates, quarter over quarter.
What is a decision trace?
A decision trace is the recorded reasoning behind one agent action: what the agent read, what it inferred, what it proposed, and the evidence supporting the call. RevSure records a Decision Trace for every agent decision, which makes weekly sampled review practical. Reading traces is the GTM equivalent of error analysis, the core habit of AI engineering's eval canon.
Why do lead scoring models lose trust?
Scores lose trust when their distribution contradicts lived reality and nobody can explain why. One propensity model scored 99 percent of 20,000 leads under 5 percent likelihood, and the operations leader refused to write the scores into the CRM. Teams in that position often abandon lead scoring entirely. Calibration review exists to catch that failure before anything ships to a system of record.