What an AI development company should show you before you sign

Every software company sells AI now. The demos are uniformly impressive, because a demo is the one environment where the model always behaves, the data is clean and nobody has tried anything unexpected.

The gap between a demo that works and a system you can put in front of customers is where almost all AI projects die. These are the things worth asking to see before you sign, roughly in order of how much they predict the outcome.

1. How they know the thing got better

This is the single strongest signal, and most suppliers cannot answer it.

Ask: when you change a prompt, swap a model, or adjust the retrieval step, how do you know the system improved rather than got worse in a way nobody noticed for three weeks?

The answer you want involves a set of test cases with known-good outputs, run automatically on every change, with a score that goes up or down. It does not have to be sophisticated. It has to exist. Teams without one are tuning by vibes, and tuning by vibes means every fix has a decent chance of breaking something else invisibly.

A supplier who has done this before will start talking about their eval set without being prompted twice. One who has not will talk about how experienced their engineers are.

2. What it does when it does not know

Ask to see the system given a question it cannot answer from its data. Not a hostile question — an ordinary one that happens to be outside what it knows.

The correct behaviour is to say it does not know. The common behaviour is to produce a confident, fluent, entirely invented answer, because that is what language models do by default and preventing it takes deliberate work.

If they will not run that test live in front of you, that is the answer.

3. Where the answer came from

For anything that retrieves from your documents or data, ask to see the citations. Then ask them to open one and check that it actually says what the answer claims.

Systems that cite sources but cite them loosely are worse than systems that cite nothing, because the citation buys trust the answer has not earned. This is the failure that gets organisations in trouble, and it is invisible unless someone checks.

4. The cost per request, and what happens when it multiplies

Ask what a single interaction costs today, and what the bill looks like at ten times the volume.

A team that has run something in production will have this number and will have opinions about it — caching, smaller models for the easy cases, batching. A team that has not will estimate on the spot. The difference matters, because AI systems have an unusual property: the unit economics can be fine at pilot scale and ruinous at real scale, and you find out after you have committed.

5. What happens when the provider changes something

Model versions get deprecated. Behaviour shifts under you without a code change on your side. Ask what their plan is for the day the model they built on is retired, and whether anything they have built is portable.

You are looking for evidence they have thought about it, not a guarantee. Anyone who guarantees this has not been doing it long.

6. Who sees the data

Ask specifically: does our data go to a third-party model provider, is it retained, is it used for training, and where does it physically sit? Get the answer in writing rather than in a meeting.

For regulated work — health, financial, anything with personal data under UK GDPR — this is not a checkbox at the end. It changes the architecture, and retrofitting it costs far more than building with it.

7. A project where the answer was not AI

Ask for one. Every team that has done this work seriously has told a client that the problem in front of them was a rules engine, a search index, or a fixed reporting bug, and that a model would add cost and uncertainty for no gain.

A supplier who has never once concluded that is either very lucky in their client base or is selling the thing they sell regardless of the problem.

The demo questions that actually work

When they show you the demo, ask for three things:

1. Type an input yourself, rather than watching the prepared one.
2. Ask it something adjacent to but outside its knowledge, and watch whether it declines or invents.
3. Ask what the worst failure they have seen in this system was, and what changed afterwards.

The third question is the one. Anyone who has run an AI system in production has a story. The specificity of that story tells you almost everything about how much real operating experience is in the room.

What none of this is about

It is not about which model they use. Model choice changes every few months and is one of the more reversible decisions in the system.

It is not about how many AI projects they have delivered, which is unverifiable and rarely defined.

It is about whether they can measure quality, handle being wrong, control cost and tell you when the answer is not AI at all. Those four hold regardless of what the models do next.

If you want a straight read on whether the AI part of your project is the right shape — including the possibility that it should not be AI — tell us what you are trying to build.