AI security · Deep dive
Your AI found a bug. Where is the witness?
Google’s PageBreak puts the spotlight on a less glamorous breakthrough: a finding has to survive a test outside the model.
Kinwilder research desk · · AI-assisted analysis
Code specimen
Kinwilder illustration · evidence contract
{"claim": "tenant A cannot read tenant B fixture",
"environment": "isolated staging / pinned revision",
"expected": "deny",
"observed": null,
"artifact": null,
"status": "unverified"}- Line 3: Decide before you run — The expected policy is specified independently of the model’s preferred result.
- Line 4: Null means unknown — The harness supplies the observation. An empty result is never evidence of either safety or exploitation.
- Line 6: Honest state — A hypothesis remains unverified until evidence supports a narrower, reproducible claim.
A model can narrate an exploit beautifully and still be wrong about the single precondition that makes it possible. The next useful step in AI-assisted security is often an unglamorous question: what observation would prove this claim false?
What the public report says
Google's September 24, 2026 PageBreak report describes an internal agent that tests Google's own web applications. It pairs model-generated hypotheses with specialized validators written separately from the AI. Google reports more than 500 XSS findings and says unverified candidates are kept out of the normal product-team reporting stream. These are Google's reported results, not an independent benchmark or a result Kinwilder reproduced. The PageBreak report.
Our reading: separate the storyteller from the witness
A convincing report often contains three different things: an observed program behavior, an assumption about the deployed environment, and a statement about business impact. Models can blend them into one smooth paragraph. An evidence record should keep them separate.
Take a suspected cross-tenant access issue in your own staging application. Seeing an identifier in a URL is not evidence. Seeing a successful HTTP response is still insufficient: it might be a login page, cached public content, or an error document with status 200. A meaningful witness is that a specifically identified principal, under a known authorization policy, received a fixture belonging to a different tenant when that policy prohibited access.
The difference is not rhetorical. It changes what the test records. Keep the principal, resource owner, expected decision, observed result, application revision, configuration, and isolation conditions. Use a synthetic marker that exists only in the intended fixture. Keep secrets out of the report.
A small evidence contract
The example is our own illustrative data contract. It is not PageBreak's internal schema. Its purpose is to make missing evidence visible before prose can hide it. observed should come from the test harness. A model's assertion that it ran a test is not an equivalent artifact.
What a validator can still miss
A passing validator proves a bounded proposition under particular conditions. It does not automatically prove internet exposure, exploit reliability, severity, or all affected versions. A failed validator may mean the hypothesis was false, or that authentication expired, the environment was different, or the harness never reached the suspected path.
Record that distinction as not reproduced or inconclusive, rather than quietly converting every failed attempt into “secure.” Include negative controls: a principal that should be allowed, one that should be denied, and a resource that does not exist. Those controls tell you whether the harness is measuring authorization or merely a generic response template.
Put the idea to work
For your next authorized test, ask the AI to produce both a claim and the smallest observable evidence that could support or refute it. Have a separate harness enforce the scope and collect results. Require a reviewer to explain the remaining preconditions before assigning impact.
This is also our editorial rule. We can publish analysis of public disclosures without pretending to have rerun their experiments. Original vulnerability hypotheses stay private until validation and responsible disclosure. More intelligent speculation can improve a research queue. It cannot turn the queue into a list of discoveries.