Introducing Advai's guide to the OWASP Top 10
As AI systems rapidly move from experimentation into real-world deployment, they are being connected to enterprise data, external tools and business processes. Meanwhile, the emergence of AI agents is giving systems greater autonomy over the actions they can take. Much like these systems are moving into reality, we are already seeing the proof of concept security risks that will become reality emerge alongside.
That makes understanding how these systems can fail, and how those risks can be identified and mitigated, increasingly important.
The Open Worldwide Application Security Project (OWASP) Top 10 for Large Language Model LLM Applications provides a useful framework for understanding some of the most significant security vulnerabilities associated with LLM-based systems, from prompt injection and sensitive information disclosure, to excessive agency.
Over the coming weeks, Advai’s technical team will take a closer look at each of these vulnerabilities and, importantly, how we test for them in practice.
Each article will explore what the vulnerability is, why it matters in real-world AI deployments and how effective testing can identify weaknesses in the systems and controls designed to manage them.
For Advai, this distinction is important. AI assurance cannot stop at evaluating an individual model or running a generic benchmark. Testing needs to reflect the system in which the model operates: what it can access, how it interacts with users and other systems, what actions it can take and what happens when controls fail.
The series will draw on the experience of Advai’s researchers and security specialists testing real-world deployed AI systems, with individual authors bringing their own perspective to each vulnerability.
We start with the number one vulnerability on the OWASP list: prompt injection, and why effective testing means testing the system, not just the model.
Follow us on LinkedIn to keep up-to-date as these are released.
How Advai Test for Prompt Injection Risk
Prompt injection is still #1 on the Open Worldwide Application Security Project (OWASP) Top 10 for LLM Applications, 2026. Better understood as a threat vector than a risk itself, it maintains its spot due to the breadth of risks that it can enable. Sensitive information disclosure, generation of harmful content, automated spear phishing, resource exhaustion – we’ve seen a wide range of consequences from successful prompt injections.
As a challenge it’s matured quickly. In 2023, a Chevrolet dealership's chatbot got talked into "selling" a $76,000 Tahoe for a dollar; the chatbot hand no authority to complete the transaction, so the consequences were largely limited to embarrassment. By 2025, researchers were demonstrating more serious attack chains against deployed agentic products.
Cybersecurity company Brave showed that Perplexity's Comet could be hijacked by a hidden instruction in a Reddit comment and made to leak a user's login. EchoLeak used a crafted email to evade Microsoft 365 Copilot prompt-injection classifiers, exfiltrating enterprise data without requiring a user to click a link. GeminiJack did much the same, demonstrating how a poisoned document, email or calendar invite could cause Gemini Enterprise to search connected data sources and transmit information.
These were responsibly disclosed security findings, rather than confirmed attacks. Even so, they show that prompt injection has moved beyond simple chatbots and semantic retrieval systems. Researchers can now demonstrate complete attack chains against agents with access to authenticated sessions and external tools.
Effective testing has to stop treating prompt injection as a single category and start treating it as a means to an end. An instruction can arrive directly, from a user, or indirectly, buried in an email, a document, a web page, or a tool's output. The entry point does not determine the severity. What matters is determining what the system can access and what it is permitted to do next. That is why, at Advai, our testing focuses on the behaviour and potential impact of the system as a whole, not simply whether a model can be induced to follow a malicious instruction.
Test the system, not the model.
There’s a reason we don’t run a generic benchmark against an individual model. Instead we start by asking: what can this system access, who can talk to it, what does it ingest from the outside world, and what would an actual failure look like? An attacker’s means, methods and objectives can vary wildly depending on the system architecture and use case. A financial assistant's worst day looks nothing like a coding agent's worst day.
To attack a system, Advai combines expert red-teamers with semi-automated generation, developing custom fixed test sets for repeatable coverage and adaptive attacks that adjust depending on the system response. This has been echoed in the Five Eyes guidance, issued by the intelligence and security alliance comprising Australia, Canada, New Zealand, the UK and US, on careful adoption of Agentic AI. Evaluations need to be agent specific.
Know which security layer succeeded.
Standard use of Attack Success Rate (ASR) is a good starting point for understanding the likelihood component of a risk. However, production systems have input guardrails, orchestration logic, the model itself, tool permissions, secondary judges, and output filters. We’re deliberately aiming for defence in depth. At Advai, we therefore record where an attack was detected, not only whether it succeeded. That evidence is what lets teams compare systems and find controls that are ineffective or unnecessarily expensive. An unsuccessful attack was not necessarily handled efficiently, nor did it necessarily produce an appropriate alert.
A blocked attack that nobody can see isn't a success. A wall of text saying "I can't help with that" might stop an attacker, but if it doesn't produce a structured, identifiable event, the Security Operations Centre (SOC) may never know that the attempt occurred. Effective testing therefore needs to answer both "can they get in" and "would we know."
References:
OWASP GenAI LLM Top 10 2026, OWASP GenAI LLM Top 10 2026 - OWASP Gen AI Security Project
AI Incident Database, Chevrolet Dealear Chatbot Agrees to Sell Tahoe for $1, Incident 622: Chevrolet Dealer Chatbot Agrees to Sell Tahoe for $1
Brave, Indirect Prompt Injection in Perplexity Comet, Agentic Browser Security: Indirect Prompt Injection in Perplexity Comet | Brave
National Vulnerability Database, CVE-2025-32711 (EchoLeak), NVD-CVE-2025-32711
Noma Security, GeminiJack, GeminiJack: the Google Gemini zero-click vulnerability leaked Gmail, Calendar and Docs data | Noma Security
ASD, Careful Adoption of Agentic AI Services, Careful adoption of agentic AI services | Cyber.gov.au
