Applying the OWASP Top 10 for LLM Applications in production

Large language models are moving from experimentation into customer-facing services, internal assistants, code generation workflows, and retrieval-augmented generation platforms. That shift matters because the security problem changes. In a prototype, the main concern is whether the model behaves as expected. In production, the concern is whether prompts, tools, retrieved content, and downstream systems can be influenced in ways that create business risk.

The OWASP Top 10 for LLM Applications is useful here because it gives engineering and security teams a shared way to prioritise the most common failure modes. It is not a compliance framework and it is not a guarantee of safety. It is a practical starting point for deciding where to put controls, what to test, and what to monitor once the system is live.

For UK SMEs, that is important because AI adoption is often happening alongside lean engineering teams, third-party model services, and fast-moving product deadlines. The right approach is to treat LLM security as part of secure software delivery, not as a separate exercise. If you already use threat modelling, logging, access control, and release gates, the same disciplines apply here, just with different trust boundaries.

Why the OWASP Top 10 for LLM applications matters in production

The value of the OWASP Top 10 is prioritisation. Most teams do not have the time or budget to solve every theoretical AI risk. They need to know which issues are most likely to affect their own deployment and which controls will reduce exposure fastest. That is where the framework helps. It focuses attention on prompt injection, data exposure, insecure tool use, model supply chain issues, and output handling, all of which are common in real systems.

It also fits well alongside other security practices. A threat model helps you understand the system boundary and abuse cases. Threat modelling LLM systems for prompt injection is a useful companion when you are mapping how prompts, retrieval, and tools interact. MITRE ATT&CK can help you think about adversary behaviour and detection coverage, especially where an attacker is trying to exfiltrate data or manipulate tool calls. Secure SDLC practices then turn those findings into review gates, test cases, and release criteria.

In practice, the OWASP Top 10 works best as a control selection aid. Use it to decide which risks need preventive controls, which need detective controls, and which need operational guardrails. That gives you a more balanced design than relying on model filters alone.

What changes when an LLM moves from prototype to production

Prototype environments are usually narrow. A small group of users interacts with a limited prompt set, there may be no sensitive data, and the model often has no real side effects. Production is different. The system now has user traffic, authentication, role-based access control, logging, integrations, and business data. The model may also be connected to search, ticketing, CRM, source control, or internal knowledge bases.

That creates new trust boundaries. The prompt is no longer just a string. It may contain user input, system instructions, retrieved documents, conversation history, and tool outputs. Each of those inputs can be manipulated. If the model can call tools, then the model is effectively making decisions about actions in other systems. If the model can retrieve documents, then the content store becomes part of the attack surface. If the model output is consumed by another service, then the output becomes an input to a downstream system.

Operationally, production also introduces change. Model versions get updated, prompt templates evolve, retrieval corpora change, and tool permissions drift. A model that was safe enough in staging can become risky after a seemingly minor configuration change. This is why production control needs to be continuous rather than one-off.

The OWASP LLM risk areas most relevant to production systems

Several OWASP LLM risk areas matter more than others in day-to-day production work. Prompt injection is the most obvious. It occurs when untrusted text influences the model to ignore instructions, reveal context, or take unintended actions. Indirect prompt injection is especially relevant in retrieval-augmented generation, where malicious instructions are hidden inside documents, web pages, emails, or tickets that the model later consumes.

Model abuse is another practical concern. That includes attempts to force the model into disallowed behaviour, extract hidden prompts, or use the assistant as a proxy for policy bypass. For customer-facing systems, abuse often shows up as repeated jailbreak attempts, unusual prompt patterns, or adversarial phrasing designed to trigger unsafe output.

Sensitive data exposure is equally important. LLMs can surface secrets, personal data, internal policy text, or confidential business information if those values are present in prompts, context windows, logs, or retrieved content. Output handling matters too. If the model generates HTML, code, commands, or structured data, the receiving system must treat that output as untrusted until validated. A useful general principle here is that external input cannot be trusted, even when it came from your own model. The same discipline used in input validation, encoding, and output sanitisation techniques applies to LLM outputs as well.

Finally, supply chain dependency risk is easy to underestimate. Many LLM applications depend on hosted model APIs, embedding services, vector databases, plugin frameworks, and orchestration libraries. Each dependency adds change risk, availability risk, and potential data handling issues. If you have not already done so, it is worth treating the AI stack as a supply chain in the same way you would for any other software component.

How the OWASP Top 10 applies to code generation use cases

Code generation deserves special attention because developers are inclined to trust output that looks plausible. That is a security mistake. Generated code can contain insecure defaults, weak cryptography, unsafe deserialisation, command injection, hard-coded secrets, or incorrect access control assumptions. The risk is not only that the model writes bad code. It is that the team may copy it into the codebase without enough review.

For production teams, the control pattern should be straightforward. Treat generated code as untrusted draft material. Require human review before merge. Run static analysis, dependency scanning, and secret scanning on any code produced or modified with model assistance. If the model suggests a library or package, verify it before adoption. If the model produces infrastructure as code, check for overly broad network rules, permissive IAM policies, and missing encryption settings.

It is also sensible to add review gates for patterns that are especially risky in AI-assisted development. Examples include authentication logic, access control checks, payment flows, data export routines, and any code that handles secrets or personal data. In those areas, a model suggestion should never be treated as authoritative. It should be treated as a starting point for engineering review.

Threat modelling an LLM application before rollout

Before you put an LLM into production, map the data flows. A simple data-flow diagram should show where prompts originate, how system instructions are stored, what context is added, which retrieval sources are queried, which tools can be called, and where outputs go next. If the system uses multiple models, include each one separately. If the application has different user roles, show how those roles affect access to prompts, documents, and actions.

This is where abuse cases become useful. Ask what happens if a user tries to override instructions, if a document contains malicious content, if a tool returns unexpected data, or if an internal user attempts to extract sensitive context. Also consider what happens when the model is wrong but still confident, because that can create business process failures even when there is no adversary involved.

For SMEs, the aim is not to build an elaborate model. It is to identify the few places where trust crosses a boundary. Those are the places where you need validation, access control, logging, and monitoring. If you already use architecture review checkpoints, this is the right point to include them. Applying data-flow diagrams to security threat modelling is a good reference point for structuring that exercise.

Production controls that reduce LLM risk

The most effective controls are usually the boring ones. Start with least privilege. If the model can call tools, give it only the permissions it needs for the specific workflow. Separate read-only and write-capable actions. Use scoped service accounts rather than shared credentials. Where possible, put a human approval step in front of destructive or external-facing actions.

Input filtering and output validation should be layered rather than absolute. Do not assume a filter will catch every malicious prompt. Instead, use it to reduce obvious abuse, then validate outputs before they are consumed by another system. For example, if the model returns JSON, validate the schema. If it returns code, route it through review and scanning. If it returns a command, do not execute it automatically.

Segmentation matters too. Keep the model runtime, retrieval store, secrets, and business systems separated. The model should not have broad network reach just because it is part of an internal service. If the application uses retrieval-augmented generation, allowlist the sources that can be indexed and queried. Do not let the model browse arbitrary internal shares or unvetted web content unless you have a clear control and monitoring model in place.

Secrets handling deserves special care. Secrets should never be placed in prompts unless there is a very specific and justified reason, and even then the exposure should be tightly controlled. Prefer short-lived credentials, vault-backed retrieval, and service-to-service authentication. If you need to protect the transport layer, use TLS and, where appropriate, mutual authentication between services. The same principles used in designing secure communications using TLS and mutual authentication apply here.

Finally, treat prompt templates and policy files as controlled configuration. Version them, review changes, and separate development, test, and production values. If your team already manages secure baselines and environment isolation, extend those practices to the AI stack. Secure configuration management and environment isolation is directly relevant when you are deciding how to separate model environments and protect sensitive settings.

Guardrails, monitoring, and detection engineering for LLM workloads

Production security is incomplete without visibility. At minimum, log prompts, tool calls, retrieval hits, policy decisions, model version identifiers, and high-level response outcomes. Avoid storing unnecessary sensitive content in logs, but keep enough context to reconstruct what happened during an incident or abuse investigation. If you already have a central logging pipeline, feed LLM events into it rather than creating a separate silo.

Detection engineering for LLM systems should focus on behaviour rather than just signatures. Look for repeated jailbreak attempts, unusually long prompts, rapid retries, high-volume retrieval access, tool call bursts, and requests that try to coerce the model into revealing hidden instructions. Also watch for patterns that suggest data exfiltration, such as prompts asking for full documents, system prompts, or internal policy text.

It is useful to define alert thresholds around workflow context. A single long prompt may be normal for a summarisation use case, but not for a customer support bot. A burst of tool calls may be expected in a research assistant, but not in a simple FAQ bot. That is why baseline behaviour matters. If you can define what normal looks like, you can detect deviations more reliably.

For teams with a SIEM, the practical approach is to send structured events from the orchestration layer, not just application logs. Include user ID, tenant or workspace, model name, tool name, retrieval source, policy outcome, and request ID. That makes it easier to correlate suspicious activity across identity, application, and infrastructure telemetry. The same operational mindset used in achieving centralised visibility across endpoints, identity, and network applies well to AI workloads.

Testing and validation before and after release

Testing should cover both the model interaction and the surrounding application logic. Abuse case testing is a good starting point. Build a small set of repeatable prompts that try to override instructions, extract hidden context, trigger unsafe tool calls, or cause the model to ignore policy. Run those tests in a controlled environment and repeat them after prompt, model, or retrieval changes.

Red teaming is useful, but it should be focused. You do not need a large-scale exercise to find value. A small set of realistic attack paths, tested by someone who understands the workflow, often exposes the most important weaknesses. For example, test whether a malicious document in the retrieval store can influence output, whether the model can be tricked into exposing system instructions, and whether tool permissions are too broad.

Open-source tooling can help with repeatability. Use it to automate regression tests in CI/CD so that prompt changes, model upgrades, and retrieval changes are checked before release. The goal is not to prove the system is invulnerable. The goal is to stop obvious regressions from reaching production. If you already run security regression tests for web applications, the same pattern works here. The difference is that the test cases are conversational and context-aware rather than purely request-response based.

After release, keep the tests running. LLM systems drift as content changes and models are updated. A control that worked last month may fail after a new document source is added or a model provider changes behaviour. Continuous validation is therefore part of operations, not just development.

Governance and operating model for UK SMEs

Governance does not need to be heavy, but it does need to be clear. Someone should own the LLM service, someone should own the security controls, and someone should approve changes that affect risk. If the system handles customer data or internal confidential data, define an exception process for any deviation from the standard control set.

For SMEs, the most practical model is usually a small cross-functional group that includes engineering, product, and security. That group can review new use cases, approve tool access, decide what data may be used for retrieval, and agree what gets logged. If the business is moving quickly, keep the process lightweight but documented. A short decision record is often enough if it captures the risk, the control, and the owner.

This is also where alignment with broader security management helps. If your organisation already uses risk treatment, change control, and management review, extend those routines to AI services. That keeps the AI programme from becoming an isolated exception. It also makes it easier to explain why certain controls exist and when they should be revisited.

Practical rollout checklist for production teams

Before expanding LLM use across the business, check that you have the basics in place. The minimum set should include scoped access to tools, logging of prompts and actions, retrieval source allowlisting, output validation, a review process for prompt and policy changes, and a test suite for prompt injection and abuse cases. If any of those are missing, the system is probably not ready for broad production use.

It is also worth asking a few practical questions. Can the model access more data than it needs? Can it take actions without human review? Are logs sufficient to investigate misuse? Do you know which model version is live? Can you disable a tool or retrieval source quickly if something goes wrong? If the answer to any of those is unclear, that is a useful signal that the operating model needs work.

For teams that are still early in their AI journey, it is often better to start with a narrow use case and a small set of controls than to launch a broad platform with weak governance. That gives you time to learn how the model behaves in your environment and to tune the controls before exposure increases.

Applying the OWASP Top 10 for LLM Applications in production is therefore less about ticking off a list and more about building a repeatable security pattern. Map the trust boundaries, constrain the model’s authority, validate what goes in and out, and keep enough telemetry to see when behaviour changes. If you do that consistently, you will have a much better chance of scaling AI safely without slowing delivery.

If you would like help translating these ideas into a practical control set for your own environment, speak to a consultant.

Frequently asked questions

What is OWASP Top 10 for LLM applications?

It is a risk-focused list of common security issues seen in large language model systems, such as prompt injection, data exposure, insecure tool use, and supply chain dependency risk. It helps teams decide which controls and tests to prioritise.

How does the OWASP LLM Top 10 apply to code generation?

Treat generated code as untrusted output. Review it before merge, scan it for insecure patterns and secrets, and apply the same controls you would use for any externally influenced code or configuration.