July 14, 2026 · 14 min read
The AI security question bank: every question the AI section actually asks
Enterprise security reviews grew an AI section, and it does not look like the rest of the questionnaire. It asks what your agent did last Tuesday, who can edit the record of it, and whether a document a user uploads can trick the model into calling a tool. This page collects those questions the way reviewers actually phrase them: 21 questions across six categories, with the reason behind each one, the evidence that closes it, and the honest fallback when you do not have the control yet.
Two things pushed this from ad hoc to standard. The Cloud Security Alliance published the AI-CAIQ, roughly 320 AI-specific questions, revised again in mid-2026, and procurement teams are adopting it as their template. And the EU AI Act's enforcement deadline of August 2, 2026 gave every buyer with European exposure a date to point at. The NIST AI Risk Management Framework supplies the vocabulary reviewers use when they escalate. Your SOC 2 answers none of this. SOC 2 covers your organization's controls, not your model's behavior or your agent's actions.
Before the questions, a note on format. Everything listed under "what satisfies it" is a thing you can hand over and let the buyer check on their own. Build these artifacts before the questionnaire arrives, because building them during the review is how deals spend two months in security. For the CSA questionnaire specifically, we wrote a separate field guide to answering the AI-CAIQ.
1. Agent audit trail and tamper evidence
Most reviews open here, for a practical reason. After something goes wrong, the first question is what the agent did, and the second is how anyone knows the record is real.
1. “Show us a log of every action your AI took on our data, and prove nobody edited it after the fact.”
Why they ask: Post-incident forensics is worthless if the log can be rewritten. Plenty of vendors have produced 'audit logs' that turned out to be a database table anyone with admin access could alter. A log that admins can edit after the fact proves very little.
What satisfies it: A per-action log sample where each entry includes a hash of the previous entry, so any edit breaks the chain, plus a verification endpoint or command the buyer can run themselves. Write-once (WORM) storage for the archive scores extra. The reviewer should be able to check integrity without trusting you.
If you don't have it: Say: 'Logs are append-only at the application layer, write access is restricted to the service role, and cryptographic tamper evidence ships in [date].' Do not say 'immutable' if a database admin can edit the table.
2. “Pick one request from six months ago and reconstruct exactly what the agent saw, decided, and did.”
Why they ask: Disputes and audits arrive months late. When a customer challenges an automated decision, the buyer's auditor wants the inputs, the model version, every tool call, and the output, in order. 'We log errors' reconstructs nothing.
What satisfies it: A worked example: one past request, replayed end to end, with timestamps, the model and version used, each tool call with its arguments, and hashes tying the steps together. State your retention window in plain numbers.
If you don't have it: Give your real retention ('90 days hot, then gone') and what a reconstruction includes today. A short window stated plainly does better in a review than a long one nobody can verify.
3. “When your agent acts, who shows up in the log as the actor?”
Why they ask: Most agent deployments run under one shared service account, so every action attributes to something like svc-ai-prod. Then nobody can tell which agent, which workflow, or which human's request caused the write.
What satisfies it: A log sample showing a distinct identity per agent, plus the originating user or API key on every entry. Attribution should survive the whole chain of tool calls, past the first hop.
If you don't have it: Admit the shared account, show how request IDs correlate actions back to a user, and commit to per-agent identity with a date.
4. “Can we stream these events into our SIEM?”
Why they ask: The buyer's security team lives in their own console. If your events never reach their Splunk or Sentinel, then from where they sit, your logging does not exist.
What satisfies it: A working export: an HTTP event collector integration, syslog, or a pull API, plus one sample event in a standard schema such as OCSF. Include your retention numbers so they know the export has time to run.
If you don't have it: Offer a scheduled export to their bucket now, and put the native connector on a dated roadmap.
2. PII and data boundaries
The fear is specific: their customer data lands in your prompts, your prompts land in a third-party model API, and somewhere down that chain it becomes training data or a breach notification.
5. “Where exactly in the request path does PII get redacted, and what happens when redaction misses?”
Why they ask: Everyone in the room knows pattern matching misses things. The question long ago shifted from whether you redact to what happens when a miss gets through, because a vendor with no answer for the failure mode probably has not gone looking for failures.
What satisfies it: A data-flow diagram with the redaction point marked before any external model call, a documented behavior for uncertain detections (block, route to a model inside the boundary, or alert), and a PII test corpus you run in CI so regressions surface before customers do.
If you don't have it: Name your method honestly: 'Detection is pattern-based. It catches structured identifiers like SSNs and card numbers. It can miss names in free text. Response scanning alerts when a miss surfaces downstream.' That answer passes more reviews than 'we redact all PII,' which nobody believes.
6. “List every third party that sees our raw prompts.”
Why they ask: A prompt can pass through your API, a model provider, a reranker, and a logging vendor before anyone notices the chain got long. The buyer carries subprocessor obligations to their own customers, and they will read your provider agreements.
What satisfies it: A complete flow list: every service that touches a prompt or output, whether it sees raw or redacted text, the region it runs in, and the agreement covering it. One page. No omissions.
If you don't have it: If the list embarrasses you, present it complete anyway. A subprocessor they discover after signing turns into a contract-breach conversation.
7. “Do you train on our data? Prove the opt-out is real.”
Why they ask: Consumer AI terms taught every buyer to assume yes. A toggle in your settings page convinces no one, because toggles change.
What satisfies it: You need three layers here. Start with the no-training clause in your own contract, add the model provider's enterprise terms (zero retention where offered), and back both with a technical statement that no fine-tuning pipeline can reach tenant data. Each layer covers a different way the promise could fail.
If you don't have it: If you fine-tune on aggregate usage, say exactly what goes in, how it is de-identified, and offer per-tenant exclusion in writing.
8. “After a request finishes, what do you still hold: prompts, outputs, embeddings, caches?”
Why they ask: Deletion promises usually break in the stores teams forget they have. Embeddings and semantic caches persist user content in a form classic deletion scripts never touch, and they widen the blast radius of any breach.
What satisfies it: A retention table per artifact type: prompt, completion, embedding vector, cache entry, log line, each with a TTL and a deletion path. Then proof that a deletion request purges the vector store and the cache along with the primary database.
If you don't have it: If embeddings currently persist past deletion, say so, provide the manual purge procedure, and date the automated propagation work.
3. Tenant isolation
Cross-tenant leakage through AI infrastructure is a newer bug class than the ones classic questionnaires cover, and reviewers know it. The probing concentrates on shared vector stores and semantic caches, plus any model you have fine-tuned across accounts.
9. “How is our data isolated from other tenants, in the database, the vector store, and the cache?”
Why they ask: The buyer's security team has learned that 'logical isolation' can mean a tenant_id column and optimism. AI systems add stores that older reviews never covered, and each one is a separate chance to leak.
What satisfies it: An isolation map that names every store. Database: per-tenant schema, or encryption keyed per tenant with the tenant ID bound into the encryption context. Vector store: per-tenant collections or namespaces. Cache: tenant-scoped keys. Per-tenant encryption keys are the strongest single line in this answer.
If you don't have it: Describe the real isolation level per store, including which layers share infrastructure, and offer a dedicated instance as the option for buyers who need more.
10. “Can another tenant's data ever influence the answers we get?”
Why they ask: A semantic cache that serves tenant B an answer generated from tenant A's documents is a real failure mode. So is a fine-tune trained across tenants that repeats one customer's phrasing to another.
What satisfies it: Show the configuration itself: cache keys scoped by tenant, retrieval filters applied when the query is built rather than after results return, and a plain statement that no model is fine-tuned across tenant boundaries.
If you don't have it: If you share caches for cost reasons, offer to disable shared caching for their tenant and price accordingly.
11. “Show us the test that proves tenant A cannot read tenant B's documents through the agent.”
Why they ask: This is the practitioner's version of the isolation question. A diagram only shows what you designed, and leaks live in what actually runs. More review teams now want a check they can rerun after each of your releases.
What satisfies it: An automated cross-tenant test: create two tenants, plant a marker document in one, then attempt retrieval from the other through every path, search, chat, and agent tool calls. It runs in CI, the results are shareable, and ideally the buyer can rerun it against their own instance.
If you don't have it: Run the test live on the review call. It takes twenty minutes and converts skeptics faster than any document.
4. Tool and action security, credentials
The moment your product stopped only generating text and started calling tools, the review changed. Reviewers used to worry about what the model might say. Once it can call tools, the worry shifts to what it can do inside your customer's systems, and on whose instruction.
12. “What can your agent actually do in our systems? Show us the allowlist.”
Why they ask: Prompt injection means the model's intent can be attacker-controlled. If the agent can reach every tool that exists, one poisoned input equals one arbitrary action in the buyer's environment.
What satisfies it: A deny-by-default tool allowlist as reviewable configuration: each tool marked allow, deny, or require-approval. Show that enforcement lives outside the model, in a proxy or policy layer, because a model cannot be trusted to police itself.
If you don't have it: If the allowlist is a hardcoded constant today rather than per-customer policy, show the constant. It proves deny-by-default exists. Then commit to configurable policy.
13. “Do agents hold standing API keys, or scoped short-lived tokens?”
Why they ask: A leaked standing key means compromise until someone notices, and prompt injection is a leak vector aimed at whatever secrets the model can see. Reviewers ask because they know how often agent credentials end up pasted into an env var and forgotten.
What satisfies it: Short-lived, scoped credentials issued per task, or a broker pattern: the agent sees a placeholder token and your platform swaps in the real credential at the network edge, so the model never holds a secret it could be tricked into repeating. For anything long-lived, a rotation schedule with dates.
If you don't have it: Standing keys in a vault with rotation every N days, stated plainly, plus per-task tokens on a dated roadmap.
14. “Which actions require a human to approve before they execute?”
Why they ask: Some actions cannot be undone: wire transfers, record deletion, emails to the buyer's customers, permission changes. Nobody wants to explain the one an agent ran unattended at 3am.
What satisfies it: A human-approval gate on named action classes, shown as configuration, plus an approval log sample: who approved, what they saw, when, and what executed after. The pattern to describe: the AI proposes, a human gates, then it runs.
If you don't have it: If everything runs autonomously today, enumerate what your product can actually do, mark which of those actions are irreversible, and offer approval gating as pilot configuration before their data goes live.
15. “A document we upload contains instructions aimed at the model. What stops that from triggering a tool call?”
Why they ask: Indirect prompt injection is the best-documented attack on agent systems and no complete defense exists. The people asking know this. Nobody expects a complete defense here. What gets scored is whether you understand where your own setup is exposed and what limits the damage.
What satisfies it: Give a layered answer: scanning inbound content for known injection patterns, tool policy enforced outside the model so a fully hijacked model can still only invoke allowed tools within scope, human approval on the dangerous ones, and audit logging that catches what got through. None of these layers is complete on its own, which is why you run all four.
If you don't have it: Never claim you block all injection. Say detection is best-effort and the real control is limiting what a compromised model can reach. That answer earns trust, because it is the same one security teams give their own leadership.
5. Model and training-data provenance
This section exists because the buyer's lawyers ask their security team, and the security team forwards the question to you. It is also where behavioral surprises hide: a model version changes, and the system the buyer approved is no longer the system that runs.
16. “Which models do you run, which versions, and where do the weights execute?”
Why they ask: The model inventory is usually the first artifact a reviewer requests, and a vendor who needs a week to produce one signals they do not control their own stack. Silent model swaps also change system behavior underneath the buyer's sign-off.
What satisfies it: Put it in one table: model name, version, where it runs (your infrastructure, a provider API, the customer's environment), what it is used for, and a change log of past swaps. Keep it current, because every other answer gets checked against it.
If you don't have it: Build the table during the review if you have to. It costs about a day, and the review will not move until some version of it exists.
17. “What rights cover the training data of the models you use?”
Why they ask: Training-data lawsuits flow downhill. If a base-model provider loses one, the buyer wants to know their exposure through your product, and whether anyone indemnifies anyone.
What satisfies it: For third-party models: the provider's published terms, the license for open-weight models, and any indemnity language, quoted rather than paraphrased. For your own fine-tunes: records of what data went in and your rights to it.
If you don't have it: You cannot attest to training you did not perform. Say exactly that: 'We run [model] under [terms]. We did not train it, and we pass through the provider's representations. Our own fine-tuning data is documented here.'
18. “What happens when a model update changes behavior?”
Why they ask: The buyer approves a system, then the provider retires that model version, and behavior drifts under a compliance sign-off that now describes a system that no longer exists.
What satisfies it: Pinned model versions, an evaluation suite that runs before any switch, and a change notice to customers when a swap ships. None of this is exciting to build, which is part of why it lands so well in a review.
If you don't have it: If you track 'latest' today, say so, describe what monitoring would catch a regression, and then pin. This is one of the cheapest gaps on this page to close.
6. Deployment and isolation options
For regulated buyers this category decides the deal. Every question above gets easier when your answer starts with 'it runs inside your network.'
19. “Can it run in our VPC, or fully on-prem, without talking to your infrastructure?”
Why they ask: Data that never leaves the buyer's network deletes most of the review at once. The subprocessor chain and the residency questions disappear, and nobody has to argue about which third parties see prompts. For banks, healthcare, and defense it is usually the opening question.
What satisfies it: A deployment artifact their platform team can run without you: a container image or Helm chart, an architecture diagram showing zero required connections back to your infrastructure, and installation docs that have survived contact with a real customer environment.
If you don't have it: A single-tenant dedicated instance in a region they choose is the middle option that survives verification. Do not call a dedicated SaaS instance 'on-prem,' because the buyer's platform team will trace where it actually runs.
20. “What does it phone home? Telemetry, license checks, model downloads?”
Why they ask: Their egress monitoring will find every connection you did not disclose, and an undisclosed connection reads as concealment even when it is only a version check.
What satisfies it: The complete outbound-connection list with the purpose of each, plus a documented zero-egress mode: models loaded from local disk, licensing that works offline, telemetry off by default. Completeness is the pass condition, not the size of the list.
If you don't have it: List everything, including the analytics ping you are not proud of. Letting them find it themselves costs you the review.
21. “If this runs self-hosted in our network, how do we get your security patches?”
Why they ask: A self-hosted deployment that goes stale becomes the buyer's liability, and platform teams have been burned by appliances running three-year-old dependencies nobody could patch.
What satisfies it: Signed releases the buyer can verify, a software bill of materials per release, a published process for CVE response, and an upgrade path that does not break their configuration. Show that the last upgrade actually shipped.
If you don't have it: At minimum, ship versioned releases and a changelog, and put in writing that you will notify on security fixes within a stated window.
How to use this bank
Work through it before a questionnaire lands. For each of the 21 questions, produce the artifact or write the fallback sentence, and put both in one folder your sales team can send within a day. The people scoring these packs read dozens of them, and the vendors who move fast tend to do the same few things. Every claim traces to something inspectable. Gaps come with dates. And the same facts appear in every answer, because contradictions between sections are what trigger the follow-up call.
If you want the shorter list first, we also published the 20 questions the AI section will ask you as a one-page reference, with a guide on how to answer each one.
FAQ
How many questions does the AI section of a security review actually ask?
The full CSA AI-CAIQ runs about 320 questions across governance, security, privacy, and resilience, but most buyers send a subset. The 21 questions in this bank are the ones that decide outcomes in practice: audit trail, data boundaries, tenant isolation, tool security, provenance, and deployment.
Does SOC 2 cover AI and agent controls?
No. SOC 2 attests to your organization's security controls: access, change management, availability. It says nothing about what your model was trained on, what your agent can do, or whether its action log is tamper-evident. That gap is exactly why buyers bolt an AI section onto their reviews.
What is the most common AI security review question?
Some version of 'show us what the agent did, and prove nobody edited the record.' Audit trail and tamper evidence come up first because incident response depends on them, and because an editable log fails silently until the day it matters.
What should we say when we don't have a control?
Say what the current control actually is, name the stronger control you are building, and give a date. Security teams accept roadmaps every day. What kills reviews is an overstatement they discover later, because then every other answer needs re-verification.
What evidence format do reviewers prefer?
Artifacts they can verify without trusting you: a log sample whose integrity they can check, a data-flow diagram with the redaction point marked, a tenant-isolation test they can rerun, a complete outbound-connection list. A policy document only tells them what you intended to build.
Why are these reviews getting stricter in 2026?
It comes down to two dates. The CSA's AI-CAIQ landed and keeps being revised, and procurement teams are adopting it as their AI-section template. The EU AI Act's enforcement deadline hits August 2, 2026, which gives every buyer with European exposure a regulator to answer to. NIST's AI Risk Management Framework supplies the shared vocabulary.
Want to know where you stand before a buyer asks? The free 2-minute readiness checker scores you against these questions. If a review is blocking a deal right now, the AI Security Review Kit turns a 30-minute intake into the full answer pack.
Written by Dmitrii Karataev. Twenty years in infrastructure and security. Corrections welcome: hello@daylite.ai.