A real agent leaves evidence. A demo leaves an impression. The difference is checkable in one sales cycle with five requests: production run counts over time, a decision trace for a real action, approval and override rates, one rollback demonstrated end to end, and the name of an agent the vendor killed. Vendors selling real agentic AI produce artifacts. Vendors selling demos produce adjectives, and this field guide shows you how to hear the difference.

The category has earned the scrutiny. Gartner coined the term agent washing in June 2025 and estimated that of the thousands of vendors claiming agentic products, only about 130 are building real agents. The same forecast expects over 40 percent of agentic AI projects to be canceled by the end of 2027, citing unclear business value and weak risk controls among the reasons. A July 2026 Forbes analysis by Robert Szczerba named the underlying mechanism the capability-deployment verification gap: agents that work in demos and fail on live proprietary data, with Forrester finding 75 percent of enterprises have adopted agentic AI while few reached production. The AI SDR segment already supplied a cautionary tale: a widely reported AI SDR vendor controversy involved claimed customers the company did not have, with ex-employees describing roughly 70 to 80 percent churn of early customers. Cancellation at that scale is a procurement failure before it is a technology failure. The projects were approved on demonstrations that could not be falsified.

Figure

The market this audit exists for

~130of thousands of vendors claiming agentic products build real agentsGartner, 2025

>40%of agentic AI projects forecast to be canceled by end of 2027Gartner, 2025

75%of enterprises adopted agentic AI; few reached productionForrester, 2026

47%of revenue leaders cite lead quality and data reliability as top barriersRevSure, 2026

Sources: Gartner, Jun 2025 · Forrester via Forbes, Jul 2026 · RevSure, The 2026 State of Agentic AI in B2B GTM

None of this means buyers should sit out. It means buyers need an instrument, because the posture most walk in with is willing and unarmed. A CMO at a field service management software company described the position exactly:

I'd much rather buy than build. I'm not a blocker here. I just want to understand what we would be buying if we did.

a CMO at a field service management software company

Understanding what you would be buying is a verification problem, and verification runs on artifacts. The instrument also has to be cheap to run, because the buying process is already expensive: Gartner's buying research puts 6 to 10 decision-makers in a complex B2B purchase, with about 17 percent of buyer time spent meeting suppliers at all. Five requests that fit in one email respect that math. Here is the audit.

The five tests

Run the tests in order, because they cheapen each other: a vendor that passes live runs has the data to answer overrides, and a vendor with real traces can usually produce a rollback on request. The order also front-loads the least confrontational asks, so the audit reads as diligence rather than hostility. Total cost to the buyer is one email and one working session. Total cost to a vendor with a real fleet is an afternoon.

Test	What to ask for	What the artifact looks like	What a red flag sounds like
1. Live runs	Production run counts over time for the agent class you are buying	A usage view: runs per week across months, with the dips left in	"Adoption figures are confidential"
2. Traces	The decision trace for one real, recent action, redacted	What the agent read, inferred, proposed, and did, with evidence attached	"Let me walk you through the architecture instead"
3. Overrides	Approval and override rates for a live deployment	A review queue where humans approved, edited, and rejected proposals	"Our customers approve almost everything"
4. Rollback	One reversed action, shown end to end	The commit, the reversal, the restored state, and the timestamps that order them	"You would just file a support ticket"
5. A killed agent	An agent or behavior the vendor retired, and why	A named failure mode and what replaced it	A roadmap answer, or a claim of none

Test 1: show me live runs

Ask for production run counts over time, for the specific agent class in your evaluation, from a real deployment. The artifact is a time series with texture: weekly runs across months, with seasonal dips and the flat stretch where a customer paused during a migration. Real usage data is lumpy because real businesses are lumpy.

The red flag sounds like confidentiality where aggregation would do, or a sandbox demonstration offered in place of history. A vendor can always redact who; a vendor who cannot show how much and since when is showing you a product that has run at scale only in slides.

Why the test works: run history is the one artifact that cannot be produced retroactively. A convincing demo can be built in a sprint. Nobody can backfill a year of production telemetry, which is why this is the first ask and why the answer's speed matters as much as its content. Real fleets answer from a dashboard. Demos answer from a meeting that needs scheduling.

Test 2: show me the traces

Ask for the decision trace behind one real action from the last few weeks, redacted as needed. The artifact reads like reasoning: the records the agent consulted, what it inferred, what it proposed, what evidence supported the call, and what happened next. If the vendor's system cannot reproduce that chain for an arbitrary recent action, the system is deciding without a memory of why.

This test carries extra weight because the industry's own data says inspection is the exception. LangChain's June 2026 survey found 89 percent of teams run observability while only 52.4 percent run offline evals: plenty of watching, less testing. A vendor with real traces will show one within days. The red flag is a general description of how reasoning works, delivered instead of an instance of it.

Figure

Vendors watch agents more than they test them

Source: LangChain, State of Agent Engineering, Jun 2026

The trace also previews life after the purchase. Whatever the vendor can show you about one decision today is what your RevOps team will have when a CFO asks why the agent moved budget in March. If the answer is an architecture diagram now, it will be an architecture diagram then.

Test 3: show me the overrides

Ask what share of the agent's proposals humans approve, edit, and reject in live deployments. The artifact is a review queue with a history: proposals that shipped and proposals that died in review, with the amendments in between. Override rates are the vital sign of a governed fleet, and their absence is diagnostic. A vendor claiming zero overrides is describing either perfection or the absence of review, and perfection has never yet been the answer.

The red flag sounds like "our customers approve almost everything," offered as reassurance. In a young deployment that number should worry you; early trust is rented, and review is how it gets bought. The honest version of this answer includes cases where the human was right to say no.

Why the test works: override rates are the residue of genuine human review, and they mature on a curve. Heavy early review that relaxes as observed accuracy accumulates is the signature of a fleet earning autonomy. Uniform approval from day one is the signature of a checkbox.

Test 4: show me a rollback

Ask to watch one committed action get reversed, end to end. The artifact is the full arc: the change as committed, the reversal executed, the prior state restored, and the timestamps that order them, in the actual systems of record. Reversibility is the precondition for giving an agent write access to a CRM, a budget, an audience, or a customer-facing message, because the alternative is that every agent error becomes a data-cleanup project with a blast radius.

The red flag is rollback described as a support process rather than a product function. "You would just file a ticket" means the undo lives in another company's queue, and your quarter waits on it.

Why the test works: rollback is expensive to build and invisible in a demo, so it tends to exist only where production incidents forced it into the product. Its presence is indirect proof of the operating history you cannot otherwise inspect.

Test 5: show me an agent you killed

Ask which agent or behavior the vendor retired, and why. Every real fleet has one, because production is where designs meet data they did not expect. The artifact is a specific story with a named failure mode and what replaced it. A vendor that cannot name a failure mode has not deployed, or has deployed and is not telling you, and the second possibility is worse.

The red flag is an answer about the roadmap, or the suggestion that failures have not occurred. In a category Gartner has publicly seeded with distrust, a confessed failure is the most credible sentence a vendor can say.

The question also lands because it maps to the buyer's real anxiety. In RevSure's 306-leader study, 47 percent of revenue leaders name lead quality and data reliability as their top barriers to agentic AI. A vendor with a kill story has met both barriers in production and adjusted. That is what meeting them sounds like.

The mechanism: curated context is the whole trick

The five tests all probe one underlying question, so it is worth naming the mechanism they detect. Demos overstate systematically, and mostly without anyone lying. A demo runs on context the vendor curated: records cleaned, identities pre-resolved, one system of record, no contradictions. Production runs on your context: RevSure's deployment research puts the average company at 22 GTM tools, and the company's vision memo describes one enterprise deployment spanning 64 systems, with roughly 7 handoffs across the GTM teams that use them. The agent did not get worse between the demo and the deployment. The context did.

Figure

What the demo removed

The demo environment

ContextCurated

IdentitiesPre-resolved

Systems that disagree0

Your environment

GTM tools at an average company22

Systems in one enterprise deployment64

Handoffs across GTM teams~7

Sources: Forbes, Jul 2026 · RevSure GTM Engineer research; 64-system and 7-handoff figures are RevSure's stated thesis

The external research says this gap is the failure mode of the whole generation of tools. MIT's GenAI Divide study found roughly 95 percent of enterprise GenAI pilots show no measurable P&L impact and traced the cause to a learning gap: tools that do not retain feedback or adapt to the context they operate in. Chroma's Context Rot research found performance degrading non-uniformly across 18 models as input grows, which is what happens when live, messy context replaces the curated kind. The engineering literature converges on the same diagnosis from the other direction. Anthropic's guidance on context engineering for agents describes a finite attention budget and the discipline of finding the smallest set of high-signal tokens for each step, and Andrej Karpathy compressed the field's shift in 2025: context engineering is "the delicate art and science of filling the context window with just the right information for the next step." A demo is context engineering performed by hand, once, by the vendor. Production is context engineering performed continuously, by a system, on yours.

As Deepinder Singh Dhingra, RevSure's founder, compresses the diagnosis: "AI for GTM doesn't fail at the model. It fails at the context." Buyers already believe it, whether or not they have named it. In the 306-leader study, 96 percent of revenue leaders say agents with full-funnel context would significantly improve execution. The audit tests whether a vendor has actually built what those leaders are describing, on somebody's real systems, and kept it running.

This is why the audit asks for artifacts of digestion rather than proofs of intelligence. Run counts, traces, override rates, rollbacks, and retired agents are all evidence that somebody's real context has passed through the system and the system survived. A demo can fake fluency. It cannot fake a year of production residue.

RevSure's answers to its own audit

An audit's author should be its first subject, so here are our five answers, including the uncomfortable one. The format is the same one we recommend pointing back at us: artifact or adjective, with every artifact available inside a normal sales cycle.

Live runs. Three agents are live and named on our site: the Campaign Reallocation agent, the Deal Risk agent, and the Deanonymization agent. Published deployment volume includes an AI software company whose agent workflows have logged 1.12M+ runs, and an enterprise security company that sent 120,000 personalized emails across 40,000 leads, written back into Marketo, with zero additional headcount.

Traces. Every agent decision on the platform records a Decision Trace: what was read, what was inferred, what was proposed, and the evidence behind it. We show redacted traces from real deployments in sales cycles, on request, for the agent class you are evaluating.

Overrides. The GTM Harness runs every consequential action through one loop: Propose, Approve, Commit, Roll back. Proposals and their outcomes are recorded per agent, so approval and override rates are a query against the harness rather than a claim, and we will run that query with you.

Rollback. Roll back is a stage of the loop, present in the product surface rather than a support process. We demonstrate a reversal end to end, timestamps included.

An agent we killed. The clearest public case in our record is a score that never shipped. An early propensity model scored 99 percent of 20,000 leads under 5 percent likelihood, and a marketing operations leader at a cybersecurity company refused to write the scores into the CRM: "I couldn't trust those numbers." He was right. The scores stayed out of the system of record, and that episode is the one we cite when buyers ask what a kill looks like from the inside.

And the honest edge, because a self-audit without one is marketing: our onboarding standard is live in under 4 weeks, and our own customer corpus includes deployments that ran much longer than that.

we are almost more than halfway through our contract for the year and we're still in setup mode.

a marketing leader at a healthcare benefits company

Both sentences are true of us. Ask every vendor, including us, for the median and for the worst case, and be suspicious of any corpus that contains no friction at all.

What certification proves, and what it does not

RevSure holds ISO/IEC 42001:2023 certification, the first international standard for AI management systems, and it belongs in this guide precisely because buyers routinely misread what it means. Certification proves that an accredited auditor examined the management system around the AI: how risks are assessed and how oversight is controlled across the lifecycle. That is a real signal about operating discipline, and vendors without it are asking you to take governance on faith.

What certification does not prove: that any agent performs on your data, that outcomes will follow, or that the demo resembles the deployment. A management standard audits the process around the system, and the five tests audit the system's contact with reality. Use certification the way underwriters use it: as hygiene that narrows the field. Then run the audit anyway. A vendor who treats the certificate as a substitute for the artifacts has answered a question you did not ask. The same logic binds the buyer in reverse: demanding certification while skipping artifacts selects for vendors who are good at audits of process. You want vendors who are good at audits of production, and ideally both.

What to do next quarter

Put the five requests into your evaluation template verbatim, and send them before the first demo, so the vendor knows the artifacts are the agenda.
Score every answer as artifact or adjective. The scoring is binary on purpose; a category this noisy deserves an instrument that resists charm.
Make the requests specific to the agent class you are buying: run counts for that class, a trace from the last few weeks, override rates from a comparable deployment, and a rollback shown live.
Ask the vendor and a reference customer the same setup-time question separately, and compare the medians and the worst cases you hear.
Treat certification as a filter, never a verdict, and read what standard was actually certified.
Turn the audit inward once a year: the same five questions, asked of your own fleet, are an AI agent evaluation program in miniature, and the answers age fast.

One more note on posture. Deepinder Singh Dhingra, RevSure's founder, on what enterprise buyers are actually purchasing: "We think buyers want speed. But what they're really paying for is certainty." The audit is how certainty gets manufactured on the buy side. Vendors who resent producing artifacts are asking you to fund their uncertainty with your quarter.

Where this comes from

Built from our work inside enterprise GTM teams: indexed, verbatim working sessions with marketing, RevOps, and revenue leaders, quoted here at descriptor level with permission discipline. Third-party findings are cited to their publishers with dates. One limit worth naming: the five tests favor vendors that instrument their agents heavily, and RevSure is one of them. We wrote the audit we can pass. That is exactly why it should be run on every vendor in your evaluation, this company included, and why the artifacts, never the authorship, should decide.

Frequently asked questions

What is agent washing?

Agent washing is rebranding conventional software, chatbots, or scripted automation as autonomous AI agents. Gartner coined the term in June 2025 and estimated that of the thousands of vendors claiming agentic products, only about 130 were building real agents. The same forecast expects over 40 percent of agentic AI projects to be canceled by the end of 2027.

How do you tell a real AI agent from a demo?

Ask for artifacts instead of adjectives. Five requests do the work: production run counts over time, a decision trace for a real action, approval and override rates, a rollback demonstrated end to end, and an agent the vendor retired. A real fleet produces all five quickly. A demo produces a slide or a scheduling delay.

Why do agentic AI demos work when deployments fail?

Demos run on context the vendor curated: clean records and resolved identities, with no conflicting systems. Production runs on live proprietary data, where the average company operates 22 GTM tools that disagree. Forbes calls this the capability-deployment verification gap, and MIT's GenAI Divide research traces failed pilots to tools that cannot retain context or fit real workflows.

What does ISO/IEC 42001 certification prove about an AI vendor?

ISO/IEC 42001:2023 is the first international standard for AI management systems. Certification proves an accredited auditor examined how the vendor governs its AI and controls risk across the lifecycle. It does not prove any agent performs on a buyer's data or delivers outcomes. Treat it as hygiene that narrows the field, then run the five verification tests anyway.

The market this audit exists for

The five tests

Test 1: show me live runs

Test 2: show me the traces

Vendors watch agents more than they test them

Test 3: show me the overrides

Test 4: show me a rollback

Test 5: show me an agent you killed

The mechanism: curated context is the whole trick

What the demo removed

The demo environment

Your environment

RevSure's answers to its own audit

What certification proves, and what it does not

What to do next quarter

Where this comes from

Related reading

Frequently asked questions

What is agent washing?

How do you tell a real AI agent from a demo?

Why do agentic AI demos work when deployments fail?

What does ISO/IEC 42001 certification prove about an AI vendor?

The agent washing audit: how to tell a real agent from a demo