The answer first
Do not launch an AI assistant because its policy document looks complete or its demo behaves well. Launch only when the release candidate has reviewable evidence for bounded capabilities, downstream authorization, meaningful approval of high-impact actions, adversarial testing, protected operational logs, incident response, deactivation, and accountable residual-risk decisions. If critical evidence is missing, pause for remediation or stop the launch.
This checklist is a practical go, hold, or stop framework. It does not certify a system, establish legal compliance, or promise that a control will prevent a breach. Prompt injection retains residual risk, finite tests cover only tested behavior, and appropriate thresholds depend on users, data, tool effects, threat actors, and consequences.
Use it after a specific assistant and release configuration exist. For earlier design work, review chatbot development services or the distinction between assistants and agentic workflows. Here, every pass must point to an artifact tied to the candidate that would actually launch.
Key Takeaways
- Treat model output, retrieved content, and tool requests as untrusted; enforce permissions and authorization in deterministic application and downstream systems.
- Require release-specific evidence: manifests, policy exports, negative tests, approval recordings, sanitized logs, incident exercises, and named risk decisions.
- Hold launch when high-impact actions, tenant boundaries, indirect injection paths, observability, deactivation, or release identity remain materially untested.
- Record residual risk and change triggers. A checklist pass is a point-in-time decision aid, not security assurance, certification, or a universal requirement.
Define the release candidate and decision authority
Freeze the object being reviewed before collecting evidence. Record the model and provider version, system instructions, policy version, retrieval corpus or index, tool definitions, credentials and scopes, application build, deployment configuration, and supported input types. Tests against an earlier prompt, a different model, or a staging tool set do not directly describe the launch candidate. When any material element changes, identify which evidence must be rerun.
Name one release authority and owners for application security, identity, data, operations, incident response, and business risk. The authority should see unresolved findings, accepted exceptions, expiration dates, and the consequences of failure. CISA’s Secure by Design guidance supports executive ownership and security as a product-lifecycle requirement. It is general software guidance, not an AI-specific certification or test plan.
Decide what go, hold, and stop mean for this release. Go means the defined evidence threshold is met and residual risks have owners. Hold means a remediable gap blocks release. Stop means the proposed capability or exposure is unacceptable without redesign. These are organization-specific governance decisions, not thresholds supplied universally by OWASP, NIST, or CISA.
Map trust boundaries before testing behavior
Draw the complete path from user input through orchestration, retrieval, model processing, validators, tools, downstream systems, logs, and user-visible output. Mark where instructions enter, where untrusted content can be retrieved, which identities cross each boundary, and where side effects occur. Include supported documents, web content, email, images, fallback routes, administrative consoles, and human review queues rather than diagramming only the model call.
NIST’s Generative AI Profile distinguishes direct injection supplied through an interface from indirect injection embedded in data likely to be retrieved. Test both paths for every input mode the product actually supports. Clearly separate retrieved material from system instructions and pass structured outputs through deterministic validation before downstream use. Validation can enforce format and selected invariants; it cannot establish factual truth or detect every harmful meaning.
Classify data in prompts, retrieved records, outputs, traces, feedback, and support artifacts. Document necessity, retention, access, deletion, and tenant boundaries. A deployment choice changes evidence ownership but is not itself a security result. If architecture placement is still unresolved, compare private LLM on LAN article and privately hosted LLM responsibilities separately from this release gate.
Use a control-and-evidence decision table
Review controls as claims that require evidence, not checkboxes satisfied by design intent. The following table synthesizes controls aligned to the cited sources. Tailor severity and approval thresholds to the workflow. A low-impact retrieval assistant and an assistant that sends messages, changes accounts, executes code, or moves money should not share one release threshold.
| Control area | Evidence to inspect | Hold or stop signal | Residual-risk boundary |
|---|---|---|---|
| Role and task boundaries | Versioned system policy, allowed and denied task list, and tests for refusal or safe routing outside scope | The assistant accepts consequential tasks outside its approved purpose or the tested policy differs from release | Instructions can constrain behavior but cannot make prompt injection impossible |
| Output enforcement | Schemas and tests for malformed, oversized, missing, unexpected, and semantically prohibited values before side effects | Unvalidated model output can reach a tool or downstream system, or failures default to execution | Deterministic checks cover encoded rules, not all factual or harmful semantics |
| Untrusted content | Trust-boundary diagram, provenance labels, and direct and indirect injection tests for every supported input channel | Retrieved or uploaded content is treated as trusted instruction, or a supported channel lacks adversarial coverage | Segregation and filtering may reduce influence while novel injections remain possible |
| Tools and permissions | Tool inventory, business justification, granular functions, assistant-specific credentials, scope export, and negative authorization tests | Unnecessary write, delete, admin, arbitrary shell, URL fetch, or broad database access remains enabled | Least privilege narrows potential impact but does not validate model intent |
| Downstream authorization | Per-user security context, policy-enforcement architecture, object-level tests, and cross-user or cross-tenant denial results | The model decides whether an action is allowed or any execution path bypasses complete mediation | Authorization is effective only for paths and policies implemented and tested correctly |
| High-impact approval | Risk classification, approval recording, visible final parameters, and reject, timeout, replay, and changed-parameter tests | A consequential action can execute without informed approval or after approved parameters change | Approval reduces autonomy risk; fatigue, misleading summaries, and compromised approvers remain concerns |
| Adversarial evaluation | Dated results tied to release versions, findings, remediation, retests, and a named owner for accepted residual risk | Injection, privilege escalation, exfiltration, tool misuse, abnormal sequences, or tenancy attacks are materially untested | Passing a finite corpus demonstrates tested behavior, not absence of unknown attacks |
| Logging and response | Redacted event samples, access and retention rules, alert tests, incident runbook, contact roster, tabletop, and after-action template | Tool calls, approvals, denials, or errors are not observable, or no owner can investigate and contain an incident | Logging supports detection and investigation; excessive collection can create privacy and security risk |
| Deactivation and recovery | Successful assistant disablement, credential revocation, tool shutdown, safe-mode, and rollback exercises | Operators cannot promptly contain the assistant or restore a defined safe operating mode | Prepared response can reduce response time and impact but does not prevent incidents |
| Supply chain and change control | Inventory of models, datasets, retrieval stores, libraries, providers, tools, licenses, assumptions, and change owners | A critical dependency or tested release identity is unknown, unreviewed, or has no response owner | Inventory supports accountability; it does not show that a component is defect-free |
Constrain agency outside the model
OWASP’s Excessive Agency guidance identifies excessive functionality, permissions, and autonomy as root causes. Offer only necessary tools, make each operation granular, separate read from write and delete, and avoid open-ended shell, generic SQL, or arbitrary URL capabilities where narrower functions meet the need. Issue assistant-specific credentials and record scope, rotation, ownership, and revocation procedures.
Carry the initiating user’s identity and security context to downstream systems. Authorization belongs at a deterministic policy-enforcement point, not in a model’s judgment or prose. Test object-level and tenant-level denial directly, including guessed identifiers, stale sessions, changed roles, retries, alternate tool paths, and race conditions relevant to the implementation. A successful happy-path action says nothing about whether forbidden actions are denied.
For high-impact actions, show the approver the actual target, scope, side effects, and final parameters immediately before execution. Test rejection, timeout, duplicate submission, replay, and parameter mutation. If a workflow needs explicit handoffs or exception queues, the workflow automation service can frame implementation work, but an approval screen alone is not evidence that permissions and downstream authorization are correct.
Test injection and failure paths adversarially
OWASP’s Prompt Injection guidance states that fool-proof prevention methods are unclear and presents controls as impact mitigations. Build a corpus around the real threat model: direct overrides, concealed instructions in retrieved material, tool-call manipulation, data extraction, privilege escalation, cross-tenant access, encoded or multilingual variants, malformed structured output, and abnormal action sequences. Include multimodal inputs only if the release supports them.
Tie results to exact release versions and preserve prompts, expected policy decisions, observed behavior, logs, severity, remediation, and retest status. Exercise trust boundaries with the model treated as an untrusted user. Record false positives and usability effects as well as successful attacks; controls that block legitimate work may create bypass pressure or unsafe manual workarounds.
Do not convert a passing test suite into a broad security claim. Tests provide evidence about sampled conditions. Unknown attacks, model variability, dependency changes, and operational mistakes remain possible. Re-run affected tests when prompts, models, tools, permissions, retrieval content, validators, or deployment settings materially change.
Prepare monitoring, incident response, and deactivation
Capture security-relevant requests, policy decisions, tool calls, approvals, denials, errors, and release identifiers in a form that supports investigation. Protect logs with access controls and retention rules, minimize sensitive payloads, and test alerts. Logging is not automatically beneficial: unrestricted prompts, secrets, retrieved records, or model output can turn observability into an additional data exposure.
Write an incident runbook with detection sources, triage roles, escalation contacts, containment steps, credential revocation, disclosure decision points, evidence preservation, recovery criteria, and after-action review. NIST’s GenAI Profile includes suggested actions for incident review, adversarial exercises, and protocols that allow a system to be deactivated when necessary. NIST describes the broader AI Risk Management Framework as intended for voluntary use; these actions are not a compliance certificate or mandatory universal launch threshold.
Exercise the actual production architecture. Disable the assistant, revoke its credentials, stop tool execution, isolate retrieval, and return users to a documented safe mode. Measure who receives the signal and whether queued or retried work still executes. A paper runbook or untested switch remains a hold signal because it does not demonstrate operability.
Make the launch decision from an evidence packet
Assemble one review packet containing the release manifest, data-flow and trust-boundary diagram, capability and permission manifest, action-risk register, adversarial test report, findings and residual-risk ledger, protected log samples, incident and deactivation exercise results, third-party inventory, and signed decision record. Link each pass to its artifact, owner, date, environment, and release identity.
Hold the launch when high-impact execution lacks meaningful approval or downstream authorization; cross-user or cross-tenant denials are untested; untrusted inputs lack direct or indirect injection coverage; malformed outputs can cause effects; security events are not observable; deactivation is unexercised; or critical findings have no owner and disposition. A narrowly scoped, time-bounded exception records governance; it does not erase the risk.
Approve only the bounded release described by the evidence. Record review dates and triggers such as model changes, new tools, broader permissions, new data classes, retrieval-source changes, provider changes, or material incidents. For help turning the packet into an implementation review, contact InfoSpectrum Engineering. This checklist reports no assessment or product-specific security result for an InfoSpectrum or client system.
A launch gate is credible only when every pass points to evidence from the system that will actually run, and every unresolved risk still has a visible owner.
InfoSpectrum Engineering
Frequently Asked Questions
Can a team completely prevent prompt injection?
The cited evidence does not support that guarantee. OWASP says fool-proof prevention methods are unclear and frames role constraints, untrusted-content handling, deterministic validation, least privilege, human approval, and adversarial testing as mitigations. Layered controls can reduce likelihood or impact while residual risk remains.
Is a human approval button enough for high-impact actions?
No. Meaningful approval should display the actual target, scope, side effects, and final parameters, while downstream systems independently enforce user-context authorization and minimum permissions. Reject, timeout, replay, and changed-parameter paths also need testing.
What missing evidence should hold an AI assistant launch?
Hold when consequential actions lack approval or downstream authorization, tenant denials or injection paths are untested, malformed outputs can trigger effects, security events cannot be investigated, deactivation is unexercised, release identity is unclear, or critical findings lack owners and disposition. These are recommended gates, not universal requirements.
Does passing this checklist make an assistant secure or compliant?
No. A pass means a team has evidence for a defined set of controls at a point in time. It is not a warranty, certification, breach-prevention promise, or legal conclusion. Applicable obligations require separate, scoped analysis using current authoritative sources and qualified review.
Sources
- LLM01:2025 Prompt InjectionOWASP Gen AI Security Project, Open Worldwide Application Security Project (OWASP)
Supports the claim that fool-proof prompt-injection prevention is unclear and that role constraints, deterministic output validation, untrusted-content separation, least privilege, human approval, and adversarial testing can mitigate impact. It does not support claiming that any single defense eliminates injection.
- LLM06:2025 Excessive AgencyOWASP Gen AI Security Project, Open Worldwide Application Security Project (OWASP)
Supports minimum tools, functions, and permissions; user-context execution; downstream authorization; and human approval for high-impact actions. It does not establish that a confirmation interface alone makes actions safe.
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence ProfileNational Institute of Standards and Technology (NIST), U.S. Department of Commerce
Supports direct and indirect prompt-injection threat modeling, adversarial exercises, incident after-action review, deactivation protocols, third-party component governance, and the voluntary-use scope of the AI RMF. It does not provide a universal mandatory launch threshold or product certification.
- Secure by DesignCybersecurity and Infrastructure Security Agency (CISA), U.S. Department of Homeland Security
Supports executive ownership and security as a lifecycle and business requirement. This general software guidance is not an AI-specific test plan, compliance standard, or proof that a particular assistant is secure.
Bring the release manifest, risk register, and open evidence gaps to InfoSpectrum Engineering for a bounded AI assistant launch-readiness review.
