Second in our series on the OWASP Top 10 for LLMs, 2026.
A correctly redacted answer is evidence that one control worked. It is not confirmation that sensitive information was protected throughout the system, because an output filter acts only on the final response. The information may already have been sent to the model, emitted in an earlier streamed fragment, or written to a log.
Sensitive Information Disclosure is second in the OWASP Top 10 for LLM Applications 2026. It covers personal data, including health and financial records, alongside commercially sensitive and classified information. The risk arises during normal operation, a system receives more information than it needs, retrieves data a user is not authorised to access, or records protected information in logs and traces.
The answer looked safe. The system was not.
An assistant is asked to summarise a customer record. The response masks the customer’s name, so a test that only inspected the output would pass. But the full record was included in the prompt, the name appeared in one of the first streamed fragments, and the unprotected prompt and response were stored in the logs. The answer looked safe, but sensitive information had already been exposed elsewhere in the system.
Crucially, disclosure is not limited to the final output. Retrieved content, tool arguments, streamed fragments, logs and traces are all disclosure surfaces. Models may also reproduce memorised training data even when no sensitive information appears in the prompt.
Testing only a chatbot interface therefore tells us very little about whether the complete system protects its information. It must also account for what data reaches the model, what it reveals and who can access it. Advai tests whether protected information can be extracted from the system, and whether it is exposed anywhere along the path regardless of what the final answer shows.
We turn each potential disclosure surface into a use-case-specific test claim, with the purpose, users, data, architecture and potential impact determining the evidence required for deployment and ongoing monitoring.
Decide what the system is allowed to reveal
Before testing, we define what is protected, who is authorised to receive it and what should happen when it is detected. The rule must be specific enough to produce a clear pass or fail result.
A support assistant may be permitted to show an authenticated customer a partial account number. It should not reveal another customer’s details or copy the full value into a tool call or log. The recipient, purpose and channel determine whether disclosure is authorised.
The requirement should tell us:
· what to detect and whether to block, redact, minimise or allow it;
· where the original value must not appear, and what happens on failure;
· how permitted, non-sensitive content should remain useful.
This starts with minimisation. The Information Commissioner’s Office (ICO) advises organisations to identify the minimum personal data needed for a purpose and to map where it is used. Keeping unnecessary information out of the model context beats hoping an output filter removes it later.
Use test data you can safely lose
Disclosure testing needs realistic information without creating a new data-handling problem. We use labelled synthetic examples and controlled canary values so that a leaked value is recognisable without exposing a real person or secret.
The baseline contains protected examples and negative examples that should remain unchanged. UK test data, for example, may include synthetic names, addresses, telephone numbers, National Insurance numbers or NHS-number formats. Harder cases split values across turns, alter punctuation, use documents, translation or encoding, and include similar-looking non-sensitive text. This tests both robustness and over-detection; a control that removes harmless information may be secure on paper but unusable in practice.
How Advai tests the complete path
We start with the use case and trace the route the data takes. From that, we design a test pack covering expected behaviour, difficult variations, adversarial attempts and service failures. At each stage, we ask the simple question: what could go wrong here, and would we detect it?
Where we test | What we are trying to uncover |
Input and context | More information is sent to the model than the task requires. |
Retrieval | A user receives a document or chunk they are not authorised to access. |
Model | Memorised or inferred sensitive information is produced without being present in the current context. |
Output and streaming | The original value appears in the answer or in any emitted chunk. |
Tools and services | Protected information is placed in arguments or sent to another service. |
Logs and traces | Prompts, responses or detected values are retained in readable form. |
Failure behaviour | A failed or unavailable control allows protected information to pass unchecked. |
This is where system testing goes beyond model testing. A control may correctly identify sensitive information, but the wider application may already have retrieved unauthorised data, exposed it before checks were applied or stored it in logs. Once sensitive data reaches the model, filtering the final output cannot undo that exposure.
Testing the complete path shows which control worked or failed, helping teams address the actual weakness instead of hiding it inside an overall pass rate.
Turn the results into evidence
We capture what controls detected, what they missed and how they behaved when something failed. Results are broken down by information type, severity and disclosure channel, giving decision-makers a clearer picture than a single pass rate.
The evidence is then translated into clear failure modes, deployment thresholds and an evidence pack for sign-off. Testing is tailored to the use case, recognising that exposing a public organisation name is very different from disclosing a credential or medical record.
Crucially, the evidence must support the claim being made. Advai tests across expected use, difficult scenarios, system boundaries and failure conditions, then re-runs those tests as models, data, permissions or controls change. This turns “we have a control” into evidence that the control actually works.
To find out more about testing your own system with Advai, get in touch at contact@advai.com, via our website (www.advai.com), or LinkedIn.
