An LLM Security Checklist for Teams Shipping AI Features

Adding an LLM to a product changes its attack surface in a way that's easy to underestimate: the model reads untrusted input and can be steered by it, sometimes in ways the people who built the guardrails didn't anticipate. The OWASP Top 10 for LLM Applications is the closest thing the industry has to a shared baseline. Here's what it looks like translated into things to actually check.

Prompt injection (direct and indirect)

• Have you tested what happens when a user directly instructs the model to ignore its system prompt?
• If your application feeds the model content from external sources (web pages, documents, emails), have you tested injection embedded in that content? Indirect injection is frequently the more dangerous variant, because the user never has to write anything malicious themselves.

Excessive agency

• If your model can call tools or APIs, does it have exactly the permissions the task needs, or broader access "just in case"?
• Is there a human approval step before any action with real-world consequences (sending an email, making a purchase, modifying data)?

Insecure output handling

• Is model output ever rendered directly into HTML, executed as code, or passed into a database query without the same validation you'd apply to any other untrusted input?

Sensitive information disclosure

• Could the model be prompted into revealing system prompt contents, other users' data, or information it was trained or fine-tuned on that shouldn't be exposed?

RAG & vector store security

• If you're using retrieval-augmented generation, is access to the vector store scoped per tenant or per user, or could one customer's query retrieve another customer's data?

Model & supply chain

• Do you know the provenance of any third-party models, fine-tuning data, or embeddings you depend on?
• Are you pinned to specific model versions, or could an upstream model update silently change your application's behaviour?

Monitoring

• Can you detect unusual patterns in prompts or outputs after launch, or is monitoring limited to uptime and latency?

None of this replaces a proper threat model for your specific application, but if you can't confidently answer most of the above, that's a reasonable signal it's time for one.