What Should You Ask an AI Vendor? Eight Questions
Last Updated: September 2026
The questions to ask an AI vendor are the ones that assume something will go wrong. A demo answers the easy question: can the tool do the job? The harder ones sit on either side of that. How is our data used, kept, and wiped? Who calls us when the thing breaks, and how fast? And when the deal ends, how does our work come out?
AI Smart Ventures works with growing businesses and founder-led teams picking their first real AI tools, and the same gap shows up again and again. Buyers turn up with a feature checklist and no exit plan. The vendors worth signing welcome the awkward questions, because they have written the answers down somewhere.
Getting this wrong is rarely loud on day one. It shows up a year later, when a tool your team leans on changes its terms, loses a batch of records, or drops the one feature your process was built on. By then your options are whatever you thought to write down at the start. Practical AI adoption is mostly that: fewer shocks, arranged in advance.
Key Takeaways
- Ask about the exit before you ask about features: a tool you cannot leave is a choice you make once, so put export scope and hand-over help in writing before the trial ends.
- Put a clock on incident notice: promptly is not a promise, so ask for several hours and the name of the role that makes the call at night.
- Split storage, keeping, and training use: these are three claims, not one, and a vendor can hold to one of them while quietly breaking the other two.
- Ask for proof, not comfort: the scope of an audit, a control map, or a recent report tells you far more than any answer given on a sales call.
Read together, those four points make one test. Can this vendor talk about their own failure without flinching? Teams who have planned for it answer by reaching for a file, because the file exists. That tell is free to check, and it works on the first call.
Why ask about exit and incident first?
Because those are the two moments when a contract stops being paperwork and turns into the only thing guarding you. A feature list tells you what the software does on a good day. Exit and incident questions tell you what happens on a bad one, and that part is rarely reopened. Any vendor can demo a tidy workflow. Far fewer can say in writing how fast they call when the model starts talking nonsense.
The odds argue for asking early. Australia’s privacy regulator, the OAIC, logged 1,205 data breach reports in 2025, an 8% rise on the 1,112 filed the year before, and 716 of those came from crime or malice. Add a tool that reads your client records, and it stops being a maybe. Listen to how each answer lands. A vendor with a real playbook reaches for a file. One without reaches for adjectives.
What should you ask about your data?
Ask three things, one at a time: where the data sits, how long it is kept, and whether it trains a model. Buyers fold all three into one question about safety, then take one calm answer that covers none. A vendor can hold your records in the right country, keep them for years longer than you assumed, and still feed a slice into training. Each claim needs its own contract line.
Suppliers are where this comes apart. Most AI products are assembled, not built, so your text may pass through a model firm, a host, and a logging firm before it comes back. Ask for that list by name, then ask which of them hold data at rest. ISO/IEC 42001, out in December 2023 as the first AI management system standard, exists partly because such chains are hard to pin down in a chat.
- Question 1, training use: do you train any model on our inputs, outputs, or support tickets, and does that bind your suppliers too? It matters because defaults shift quietly. A good answer is a flat no, written into the contract.
- Question 2, where it sits and how long: where does our data live, and how long do you hold it after we press delete? Those are two claims, not one. A good answer names the regions, gives several days, and confirms the wipe.
Who tells you when the AI goes wrong?
Someone has to, and your contract should name them. Regulators have already set the shape of this for personal data. The European Data Protection Board tells firms to alert their data protection authority within 72 hours of learning of a breach, and says a processor handling data for you must tell you without undue delay. Your AI vendor is often that processor. If their clock is vague, yours cannot be tight.
Security is not the only failure worth a phone call. In December 2025, NIST put out an early draft of NIST IR 8596, the Cyber AI Profile, which sorts the work into three areas: securing the parts of an AI system, AI-enabled cyber defence, and thwarting AI-enabled attacks. Comments closed on 30 January 2026. It names the failure types worth asking a vendor to catch, not just the ones that reach the news.
- Question 3, the notice clock: how many hours pass between you knowing about a fault and us knowing? Your own clock starts the moment you are told. A good answer is a fixed number in the contract, plus a named role who calls at night.
- Question 4: What can we see? Can we read error rates, logs, and a status page without opening a ticket? Self-serve sight cuts short every later argument. A good answer is a dashboard you can reach today, plus a window for the logs.
How do you get your data out and leave?
Ask on the first call, not the last. Europe wrote a benchmark into law: the EU Data Act has applied since 12 September 2025, and the European Commission says providers of data processing services must let buyers switch quickly and smoothly, with no loss of data or app function. Open interfaces are required, and exports must come in a common, machine-readable format. Borrow that wording even where the rule does not reach.
Getting your data out is not one thing, which is how a vendor says yes and still leaves you stuck. There is the raw content, the shape around it, the settings your team tuned, and the record of what the system decided. A text dump of the first is not an exit. AI Smart Ventures observes that buyers who ask for a sample export during the trial hear a different answer from those who ask whether export exists.
- Question 5, export scope: what exactly comes out, in what format, and can we see a real sample this week? Content without its shape is not an exit. A good answer is a working download holding content, shape, settings, and your decision log.
- Question 6, the last day: what help do we get moving off, how long does our data stay reachable, and when is it wiped? Exits happen under time pressure. A good answer is a written hand-over window, then a confirmed wipe.
What proof should a vendor be able to show?
Files, not comfort. Ask for the scope of any certificate, a map from their controls to a named framework, and the latest audit report under NDA. Shared frameworks now exist for this. In February 2026, the US Treasury put out an AI Lexicon and a Financial Services AI Risk Management Framework that adapts the NIST AI Risk Management Framework to one sector, so both sides argue in the same words.
Scope is the word doing the real work. A badge that covers a firm’s own ticketing system says nothing about the AI product you are buying, and the scope note admits it if you read it. Ask which systems sit inside it, when it was last checked, and what has changed since. Tool-first AI agencies often point at their suppliers’ badges rather than their own, which is fair when they say so plainly.
- Question 7, model change notice: how much warning do we get before a model version is retired or swapped under us? Your prompts were tuned to one version. A good answer offers a change log, a notice window, and a tested path forward.
- Question 8: Who is at fault: what does the contract say when wrong output reaches our clients? Someone carries that risk, and silence means it is you. A good answer splits vendor fault from buyer fault, and names the fix for each.
Turning eight questions into a scored shortlist is the step most teams skip. AI advisory puts a vendor-neutral scorecard behind the talk, so you weigh written answers, not sales decks.
Frequently Asked Questions
What questions should you ask an AI vendor?
Ask eight. Whether they train on your data, where it lives and how long they keep it, how fast they tell you about a fault, what you can see without a ticket, what you can export, what help you get on your last day, how much warning you get before the model changes, and who is at fault when output is wrong. Features come after that.
What are good questions to ask an AI?
Good questions test the edges, not the sales pitch. Ask the tool to cite sources, then check two of them yourself. Ask it the same thing twice and compare. Ask what it does not know, and give it a task where you know the right answer. Steady results and honest refusals matter more than smooth prose, and that habit is basic AI literacy.
What should you ask an AI vendor before buying a company?
Ask what the AI stack is licensed to do after a change of control, because many contracts do not transfer and some model licences bar use by a new owner. Check who owns the data, what duties you would inherit around keeping it, and whether client data was ever used to train a model. Then ask for the fault log for the past two years.
How quickly should an AI vendor report an incident?
Fast enough that you can still meet your own deadlines. A controller has 72 hours from learning of a personal data breach to alert its regulator, and a processor working for you must report to you without undue delay. If your vendor handles personal data for you, their clock has to be much shorter than yours. Ask for a fixed number of hours.
What does data portability mean in an AI contract?
It means four things travel with you: your content, the shape around it, your settings, and the record of what the system did. A vendor offering only the first has handed you a text file, not an exit. Ask for a real sample export during the trial, in an open, machine-readable format, plus a written window for pulling it out after the deal ends.
How do you check whether a vendor trains on your data?
Read the terms, not the sales page. Look for the default setting, whether an opt-out exists, and whether the promise reaches their suppliers and support chats too. Ask for it as a contract clause instead of a screenshot of a settings page, because settings change without warning. Then ask what happens to data they hold already if you switch that setting off.
What are red flags in an AI vendor’s answers?
Vague timing is the clearest one. Words like promptly, best efforts, and soon fill the space where a number belongs. Watch for a refusal to run a pilot on your own records, a certificate whose scope is never shown, and an export nobody will demo live. Bold claims of perfect accuracy are another, because every current AI system has known limits.
Where should you start if this feels like a lot?
Start with one tool and one contract rather than a full review. Pull the current deal, mark which of the eight questions it answers today, and send the gaps to the vendor in a single dated email. Most teams find three or four unanswered, and the replies alone reshape a shortlist. To have a second pair of eyes on yours, schedule a consultation.
Executive Summary
Vendor choice goes wrong in the places a demo never visits. These eight questions lean hard on data handling and exit: training use, where data sits and how long, the notice clock, what you can see for yourself, export scope, hand-over help, model change warning, and who is at fault when output is wrong. Each carries a reason it matters and a good answer to listen for. Vendors who have planned for failure hand you a written process, while vendors who have not hand you adjectives. Ask early, while an unsigned deal still gives you room.
What Should You Do Next?
This week, pull the contract for the AI tool your team leans on hardest and mark which of the eight questions it answers in writing. Send the gaps to your account manager as one dated email. Do the same for anything you are about to sign before the trial ends, and file the replies with your change management notes.
AI Smart Ventures offers AI advisory for growing businesses comparing AI tools without an in-house technical team. Schedule a consultation to turn a vendor shortlist into a scored sheet your leadership can sign off.
People Also Read
- AI Marketing for Owner-Operators With No Marketing Team
- How to Evaluate an AI Marketing Agency Before You Hire
About the Author
Nicole A. Donnelly is the Founder of AI Smart Ventures and an AI Adoption Specialist with 20 years of experience as a founder and CEO and over a decade leading AI adoption initiatives. She helps businesses integrate artificial intelligence with clarity and confidence, driving innovation and sustainable growth. Nicole has trained over 20,217 professionals in Applied AI, delivered 624 workshops, and worked with close to 1,000 organizations across diverse industries.
Expertise: AI Transformation, AI Strategy, AI Implementation, AI Adoption, Applied AI, Marketing, Business Operations
Disclaimer: This content is for informational purposes only and does not constitute professional business or technology advice. Results vary based on industry, existing systems, and implementation commitment. Contact AI Smart Ventures for a consultation regarding your specific situation.
