← ALL WRITING
VENTURE

Beyond the demo: how investors judge an AI product on evidence

A demo shows what a product can do once, on a good day, in the founder's hands. Investment decisions need to know what it does every day, in a customer's.

WRITTEN
READINGEleven minutes
FORVCs, angels, seed and Series A investors, corporate venture teams

To evaluate a startup demo properly, treat it as a claim and test it. Bring your own inputs, push the product off its scripted path, time it, ask what happens when it fails, and then ask for the logs, eval results, usage data and cost per task that show what the product does when nobody is presenting. A demo can prove a capability exists. It cannot prove the capability is reliable, automated, affordable or used.

That distinction matters more in AI than it did in ordinary software. A conventional SaaS demo mostly shows you an interface, and interfaces either work or they don't. An AI product is probabilistic. The same prompt can produce a good answer on Tuesday and a confident wrong one on Wednesday, and the founder knows which inputs give the good answer. A demo is a sales asset. It was built to win the meeting you are sitting in.

A demo is a sales asset. It was built to win the meeting you are sitting in.

The money involved has made this a bigger problem. The NVCA's 2026 Yearbook, built on PitchBook data, put AI at 65.4% of US venture deal value in 2025, up from 50.9% in 2024, and PitchBook's global figures had AI taking more than half of the $512.6bn invested worldwide. When that much capital chases one category, demos get better at looking like products.

What a demo can and can't prove

I have sat on both sides of this table: as a founder showing a product to people deciding whether to fund or buy it, and as an investor watching founders do the same. A good demo answers a narrow set of questions well, and investors routinely ask it to answer the rest.

A live demo can show you A live demo cannot show you
That the capability exists at least once How often it works across real inputs
The founder understands the user's problem Whether software or people are doing the work
The interface and the intended workflow Latency and cost at production volume
How the founder handles a hard question Whether customers use it weekly or tried it once
Roughly how mature the product is What happens when the model vendor changes something

Everything in the right-hand column decides whether the company is a good investment. Almost none of it can be settled in a 30-minute meeting. The job of the meeting is to find out which of those questions the founder can answer with evidence, and how they behave when you ask.

How startup demos get staged

Staging is a spectrum. Most of it is ordinary salesmanship, and some of it has ended in enforcement action. You should be able to recognise each kind.

Scripted paths

The demo follows a route the team has rehearsed dozens of times. The inputs are known, the edge cases are avoided and the order of clicks is fixed. There is nothing dishonest about rehearsing a demo. The problem is the inference investors draw from it: that the product handles the unrehearsed path equally well.

The most famous example is from outside AI start-ups. In January 2023 Reuters reported sworn testimony from Tesla's director of Autopilot software that the company's 2016 self-driving promotional video ran on a predetermined, 3D-mapped route, with drivers intervening during test runs, and showed capabilities the system did not then have. The engineer said the aim was to show what was possible to build, rather than what customers could buy.

Cherry-picked inputs and edited output

The second kind uses real model output, chosen carefully and presented in a way that changes what the viewer thinks happened. When Google launched Gemini in December 2023, its "Hands-on with Gemini" video appeared to show a model responding live to spoken questions and moving video. TechCrunch reported that it was assembled from still frames and written text prompts, with latency cut and outputs shortened, and Google's own developer post described the image-and-text prompting behind it. Google said the prompts and outputs were real. They were. The interaction the viewer saw was not.

Start-ups do a smaller version of this every week: a pre-recorded walkthrough, a "live" run against a cached response, a document the model has effectively seen before.

Humans in the loop, undisclosed

The third kind is people doing work the audience believes the software is doing. Using people in the loop is normal and often sensible. Hiding them is the problem.

In January 2025 the SEC settled charges against Presto Automation, a Nasdaq-listed restaurant technology company, finding it had made materially false and misleading statements about its AI drive-thru ordering product. The SEC found Presto claimed the product eliminated human order-taking when the vast majority of orders through one version needed human intervention, and that it had not disclosed that the speech recognition in its deployed units was, for a period, owned and operated by a third party.

Wizard of Oz

At the far end, the product is mostly a person behind a curtain. In April 2025 the SEC charged Albert Saniger, founder of the shopping app Nate, alleging he raised over $42m by telling investors the app completed online purchases with AI. The parallel criminal case in the Southern District of New York alleged that purchases were in fact completed manually by contract workers and that the app's real automation rate was effectively zero. These are allegations, not findings, and Saniger is entitled to contest them.

Nikola is the non-AI version investors still quote. The SEC's 2021 complaint against its founder said a video of the Nikola One apparently driving at speed was filmed after the truck was towed up an incline and allowed to roll down it. Nikola later agreed to pay $125m to settle SEC charges.

Regulators now treat AI claims as claims

The SEC's first AI washing cases, in March 2024, fined two investment advisers, Delphia and Global Predictions, $225,000 and $175,000 for misleading statements about their use of AI. The FTC's September 2024 Operation AI Comply included DoNotPay, the "robot lawyer", which the FTC said had not tested whether its output matched a human lawyer; it agreed to pay $193,000. In August 2025 the FTC finalised an order against Workado, which had advertised its AI content detector as 98% accurate when, according to the FTC's complaint, independent testing on general content found 53%.

Notice what the Workado case is about. Not a fake product. A real product with an untested accuracy number. That is the most common problem in a seed-stage AI deck, and it is the one investors check least. I've written about the wider pattern in AI washing in due diligence.

How to evaluate a startup demo live

Ask for the demo to be live, on the production system, with you choosing some of the inputs. Tell the founder in advance. A founder with a real product will agree. One who needs a week to "prepare an environment" has told you something before you have seen anything.

Then work through these, in roughly this order.

  1. Use your own inputs. Bring five to ten real examples from the problem domain: an ugly scanned invoice, a contract with a missing schedule, a customer email written in anger with three questions in it. If you don't have domain material, ask a friendly operator in that sector for some. The founder's examples tell you what the product can do. Yours tell you what it does.
  2. Push it to the edges. Give it something just outside its stated scope, something ambiguous, something in a second language, a file twice the usual size. You are looking for how it fails. Good products fail visibly: they flag low confidence, ask a question or route to a person. Weak ones produce fluent, confident nonsense.
  3. Time it. Note how long each response takes. A three-second answer in a demo can be a 40-second answer under load, and in some workflows 40 seconds kills adoption. Ask what the p95 latency is in production. Most founders who know their product will have a number.
  4. Run the same input twice. AI output varies between runs. If the second answer is materially different from the first, ask how the product handles that and how customers are told.
  5. Ask what happens on failure. Who finds out when the model gets it wrong, and how? Is there a review queue? Who works it? How many items went through it last month?
  6. Ask to see it outside the demo account. A real customer workspace (anonymised, with permission) or the admin view of production usage. Demo tenants are curated. Production is not.

Watch the founder as much as the product. In my experience the most useful signal in the room is how someone reacts when the product gets something wrong in front of you. "That's a known weakness, here is how often it happens and what we're doing about it" is a good answer. Talking over the failure, or quietly switching back to the prepared example, is not. Our AI startup demo scorecard turns these tests into twelve criteria you can score in the room.

Questions to ask an AI founder

These belong in the first meeting, before anyone is attached to the deal. Each one has a version of a good answer.

  • What exactly does the model do, and what does the rest of the product do? You want a clear split, not "the AI handles it".
  • Which models do you use, from which providers, and why? A good founder knows what they'd switch to and roughly what it would cost.
  • What share of tasks does a person touch today, and what was it six months ago? A measured, falling number is fine. "None" is rarely true at seed.
  • How do you know it's getting better? Look for an eval set built from real customer cases, scored on every release.
  • What does one completed task cost you, all in? Model fees, retrieval, hosting and human review. Founders who haven't worked this out haven't priced the product.
  • What would break if your model provider changed its pricing or retired the model you use? There should be an answer, even if it's an uncomfortable one.
  • Which customer uses it most, and can I speak to them without you on the call?

What to request after the meeting

The demo opens the question. The follow-up pack answers it. Ask for these within a week, while the founder still wants the round and is responsive.

Request What it tells you What good looks like
Sandbox or trial login Whether it works without the founder driving You can complete a real task alone in under an hour
Sample of production logs (anonymised) Real inputs, real outputs, real error rates Messy inputs, visible failures, a team that has looked at them
Eval set and recent scores Whether quality is measured or assumed Hundreds of real cases, scored per release, trend shown honestly
Weekly active usage by customer Whether paying customers use it Usage concentrated in workflows, not one-off logins after a pilot
Cost per task, broken down Whether margins survive scale Model, infrastructure and human review all counted
Human intervention rate over time How much is automated today A real number, falling, with a plan
Model and vendor list with terms Dependency and switching risk More than one viable provider, terms read and understood
Two references you choose The view from outside the pitch Customers who describe a changed process, not a nice tool

None of this is unusual to ask for under NDA. Most of it should exist inside any AI company that is past its first paying customer. The technology due diligence checklist covers the wider engineering, security and IP questions, which matter at Series A and above.

Usage data deserves particular attention. Pilots convert to logos on a slide easily. Logos convert to weekly usage much less often. Ask for usage at the level of individual customers and look for the ones that went quiet after month two.

Red flags in an AI startup demo

Any one of these is a reason to dig. Two or three together usually mean the product is less automated, less tested or less used than the deck says.

  • The demo cannot be run live, or only on the founder's laptop against a "demo environment".
  • You are not allowed to supply inputs, or your inputs are "not representative".
  • Accuracy is quoted as a single headline number with no test set, sample size or date behind it.
  • Nobody can tell you the human intervention rate, or the answer is zero.
  • Cost per task is unknown, or excludes model fees because "they're coming down".
  • The technical co-founder is unavailable for diligence, or every technical question goes back to the CEO.
  • Customers in the deck are pilots, free trials or design partners, and that is not made clear.
  • Latency in the demo is suspiciously identical between runs.
  • The product's claims move when you ask for evidence: 95% becomes "up to 95%", becomes "in our best cohort".

Treat the last one seriously. Optimism is normal in founders. Claims that retreat under questioning, without the founder noticing, are how small companies end up in regulators' press releases.

When to bring in technical due diligence

Most seed and Series A investors can run the live test and the follow-up questions themselves. Where it gets harder is reading the answers. A sample of logs, an eval spreadsheet and an architecture diagram are only useful if someone can tell whether they describe a product or a prototype held together by a support team.

That is the point at which a short, independent check pays for itself. Our technical due diligence for VCs is a pre-term-sheet check: two to three days, £4k to £8k fixed, in which we test the product with our own inputs, look behind the demo at what is automated and what is manual, review the model set-up, costs and dependencies, and talk to the engineers directly. You get a short written view on whether the product does what the deck says, what it costs to run, and the three or four things to put in the term sheet or the 100-day plan. For later rounds or control deals, the questions in AI due diligence for private equity apply at greater depth.

If you have a demo coming up and want a second pair of eyes before you commit, our VC technical due diligence page sets out how the check works and what it costs.

FREE RESOURCE · PDF + XLSX
AI startup demo scorecard
DOWNLOAD FREE →

Questions

What is a Wizard of Oz demo?

A Wizard of Oz demo is one where a person secretly does work the audience believes the software is doing. The term comes from user research, where it is a legitimate way to test an idea before building it. In a fundraise it becomes a problem when investors are not told. The SEC alleged in 2025 that the founder of shopping app Nate presented automated purchasing to investors while contract workers completed orders by hand.

How long should technical due diligence take at seed or Series A?

Two to three days is usually proportionate before a term sheet at seed or Series A. That is enough to test the product with your own inputs, see the architecture and model set-up, check usage data against the deck and talk to the engineer who built it. Control deals and later rounds justify one to three weeks, because there is more code, more history and more money at stake.

Should I sign an NDA to see a startup's logs or evals?

Usually yes, and most investors already do for data rooms. Logs, eval results and cost data are commercially sensitive, and a founder is right to protect them. What you should not accept is an NDA used as a reason to show nothing. A founder who will share production metrics under a mutual NDA is behaving normally. One who will share nothing under any terms is telling you something.

Is it a red flag if an AI startup uses humans in the loop?

Not on its own. Plenty of good AI products route hard cases to people, and early-stage companies often do things manually while they learn. It becomes a red flag when the human share is hidden, unmeasured or falling only in the deck. Ask what percentage of tasks a person touches today, how that has changed over six months and what it costs per task.

What should a VC ask for after an AI startup demo?

Ask for a sandbox or trial account you can use without the founder present, a sample of production logs, the eval set and recent scores, monthly active usage by customer, cost per task including model fees and human review, the model and vendor list, and two customer references you choose. Ask for them within a week of the meeting, while the founder still wants the round.

Brannan Coady, founder of Anyway Ventures
Brannan Coady
Founder of Anyway Ventures. Exited founder, former CEO of YourParkingSpace, investor and board member; now building AI inside PE-backed and mid-market businesses. More →

Keep reading

ALL WRITING →
DILIGENCE · 9 MIN
AI washing: how to test a target's AI claims in due diligence
When the equity story leans on AI, the diligence question is simple. Take the AI away: what's left, and what does it cost to keep it running?
DILIGENCE · 10 MIN
Technology due diligence checklist for private equity deals
Every tech DD checklist covers the same ground. The useful ones say which answers should change the price, the structure or the first 100 days.
DILIGENCE · 10 MIN
AI due diligence for private equity: what to test before you sign
Most AI diligence reports check whether the target's AI is real and stop there. That is one job of three, and it is rarely the one that moves the price.

Talk about this piece.

The first conversation is free and takes under an hour. If there's an engagement worth having you'll leave knowing its shape and its price. If there isn't, I'll say so.

Book 30 minutes brannan@anywayventures.com