Is your AI product ready to build, harden, or scale?
A working demo proves an AI feature is possible. Use these 12 questions to decide whether to validate the problem, define the build, harden the product, or prepare it to scale.
- 6Readiness dimensions
- 12Diagnostic questions
- 5Decision levels
Quick verdict
A demo answers one question. Readiness answers six.
A prototype can show that a model responds, an agent completes a task, or a workflow can be automated. Product readiness asks what happens around that moment.
The gap between a demo and a dependable product is usually not another prompt. It is the product and engineering system around the model.
How Robust Devs approaches production AISix readiness questions
- Is the problem repeated and valuable enough to solve?
- Can users discover, understand, and trust the workflow?
- Does the outcome support a credible business case?
- Can the team measure quality against representative inputs?
- Are failures, latency, cost, and human review controlled?
- Are data boundaries, infrastructure, and ownership explicit?
Readiness map
Six dimensions of AI product readiness
Use the framework as decision guidance, not a scientific score. A weak answer in one critical area can matter more than strong answers everywhere else.
| Dimension | What you need to know | Risk when it is unclear |
|---|---|---|
| Problem evidence | Who has the problem, how often it occurs, and why it matters now | A technically impressive feature nobody needs |
| Workflow and adoption | Where the product fits, how users reach it, and what earns trust | Low discovery, weak repeat use, or a workflow that creates more work |
| Commercial logic | How the outcome affects revenue, retention, expansion, or operating cost | Useful software that cannot support a viable product |
| Model quality and evaluation | What a good result means and how quality is measured | Inconsistent output, hidden regressions, and arguments based on anecdotes |
| Production operations | How the system handles latency, cost, retries, failures, and review | A demo that becomes unpredictable under real use |
| Security, compliance, and ownership | What data crosses which boundary and who controls the system | Exposure, unclear responsibilities, and difficult handoff |
Most AI products fail on the workflow, not the model
Dimension 01
Problem evidence
Decision signal
If the buyer or problem remains unclear, do not accelerate the build. Run the smallest validation that could change the decision.
Can you name one buyer, one repeated problem, and one current workaround?
If the answer needs several audiences or a long explanation, the product may still be in exploration. Define who experiences the problem, when it happens, what they do today, and what the current failure costs in time, money, risk, or missed opportunity.
What evidence would justify engineering investment?
Useful evidence can include customer interviews, repeated support requests, observed workflow breakdowns, paid pilots, waitlist behavior, or a clear pattern in usage data. The standard should match the size and risk of the build. A two-week experiment needs less evidence than a multi-month platform commitment.
Sources: The most expensive feature is the one nobody asked for, Skipping discovery and diving straight into code
Dimension 02
Workflow and adoption
Decision signal
If the feature works but users do not reach or repeat it, investigate the workflow before adding capabilities.
Does the AI improve a workflow users already perform?
“Add a chatbot” is not a workflow definition. Identify the trigger, inputs, decision, action, and handoff. The strongest AI features reduce a painful step, improve the quality of a decision, or complete work that users already need to do.
Can users discover the value without being trained around poor UX?
Shipping is not the same as solving. Test whether users can find the feature, understand what it is for, provide the right input, interpret the result, and recover when the output is uncertain. If adoption depends on a long product tour, the workflow may still be fighting the user.
Sources: Shipping is not the same as solving, Adding AI as a side feature usually fails
Dimension 03
Commercial logic
Decision signal
If the feature has utility but no commercial role, test willingness to pay or its effect on retention before expanding scope.
What valuable outcome does the customer buy?
Customers rarely buy access to a model. They buy fewer manual steps, faster decisions, lower operating cost, reduced risk, or a capability they could not perform reliably before. Lead with that result and make the AI implementation secondary.
Does the outcome support the product and pricing model?
A feature can be useful without being commercially important. Decide whether it improves acquisition, activation, retention, expansion, delivery margin, or another meaningful business measure. If the team cannot explain how the feature creates or protects value, it is too early to specify a large build.
Sources: The best AI products solve problems that happen to need AI, Pricing should shape the product before launch
Dimension 04
Model quality and evaluation
Decision signal
If quality is judged through isolated demos, harden the evaluation process before widening access or automating higher-risk work.
What does a good result mean for this exact workflow?
“Looks good” is not an evaluation method. Define observable criteria for the task: correctness, completeness, citation quality, policy compliance, tone, extraction accuracy, tool-use success, or another workflow-specific result. Include unacceptable failures, not only ideal outputs.
Do your tests represent real inputs and edge cases?
Build an evaluation set from representative user data, difficult examples, known failure modes, and cases where the model should abstain or route to a person. Re-run it when prompts, retrieval, tools, providers, or surrounding code change.
Evaluation loop
- 01
Representative inputs
- 02
Model or agent
- 03
Scored output
- 04
Failure review
- 05
Release decision
Sources: Robust Devs AI Development, Robust Devs AI Evaluation & Observability
Every prompt, retrieval, or provider change is a reason to re-run the set
Dimension 05
Production operations
Decision signal
If the system works only when every dependency behaves, it is a prototype. Define failure handling and operational visibility before calling it production-ready.
What happens when the model, tool, or dependency fails?
Production AI needs explicit failure paths. Decide when the system retries, changes provider, falls back to a simpler workflow, asks for clarification, queues work, or hands control to a person. Partial success must be visible; otherwise users cannot tell whether the task completed safely.
Can the team see latency, cost, quality, and usage together?
A product can be accurate but too slow, affordable but unreliable, or popular but economically unsustainable. Observe the measures that shape the user experience and business case together, then set thresholds that trigger review.
Sources: Robust Devs AI Evaluation & Observability, Choose the model for the job, not by default
Dimension 06
Security, compliance, and ownership
Decision signal
If data handling or operational ownership is ambiguous, resolve the boundary before expanding use.
Which data crosses which boundary?
Map the data entering the product, where it is stored, which model or tool providers receive it, how long it is retained, and who can access it. Regulated or sensitive workflows may also need explicit review, audit evidence, consent, deletion, and human-oversight paths.
Who owns the product after launch?
Make repositories, cloud accounts, data stores, credentials, deployment pipelines, documentation, runbooks, and vendor relationships explicit. A production product should not depend on knowledge or access that disappears with the delivery team.
Decision levels
Find your next responsible move
The levels guide the next decision. They are not a validated industry benchmark, and they are not a reason to force every product into a larger build.
Explore
You are here when: The buyer, problem, or reason to use AI is still unclear.
Do next: Interview users, observe the current workflow, test the problem statement, and define what evidence would justify a build.
Validate
You are here when: The workflow is understandable, but demand, adoption, or commercial evidence remains thin.
Do next: Run a narrow prototype or pilot with explicit success and stop criteria. Avoid platform-scale architecture.
Build
You are here when: The buyer, workflow, outcome, scope, and quality criteria are credible enough to support implementation.
Do next: Define architecture, risks, delivery stages, ownership, and a fixed production scope. A Discovery Sprint is the right entry point when those decisions are not documented.
Harden
You are here when: The product works, but evaluation, failure handling, security, observability, cost controls, or documentation are incomplete.
Do next: Review the current system and fix the highest-consequence production gaps first. Use a Tech Audit when an existing codebase needs an independent technical view.
Scale
You are here when: Quality, operations, ownership, and commercial performance are measured and stable enough for wider use.
Do next: Test capacity, recovery, support, data quality, provider dependencies, and cost under the next expected level of demand.
Most teams land between two levels. The gap is usually the thing worth resolving first.
Talk it through with a founderPre-build checklist
Before you build more, can your team answer these?
- 01Who has the problem, and what do they do today?
- 02What evidence justifies engineering investment?
- 03Where does the AI fit in the user's real workflow?
- 04Can users discover, understand, and trust it?
- 05What valuable outcome does the customer buy?
- 06How does that outcome support the product model?
- 07What does a good result mean for this task?
- 08Do tests represent real inputs and failure cases?
- 09What happens when a dependency fails?
- 10Can the team see quality, latency, cost, and usage?
- 11Which data crosses which boundary?
- 12Who owns the system after launch?
If several answers are missing, the next step is not automatically a larger build. It is the smallest engagement that resolves the most important uncertainty.
Turn the unknowns into a build decision
Bring the answers you have and the ones you cannot yet give. A founder will help work out which uncertainty matters most, and whether a Discovery Sprint to define the build or a Tech Audit of what already exists is the better next step.