Code anatomy · Code anatomy
PyRIT’s three-valued OR: when uncertainty must survive
An analysis of five hardening lines, the assumptions behind them, and the downstream decisions that must preserve their meaning.
Kinwilder research desk · · AI-assisted analysis
Code specimen
pyrit/score/true_false/true_false_score_aggregator.py
if left is True or right is True:
return True
if left is None or right is None:
return None
return False- Line 1: Canonical values are a precondition — Analysis: Identity checks recognize the Boolean singleton True. The intended domain is True, False, or None; a truthy non-Boolean value can bypass this branch. Annotations alone do not enforce that domain.
- Line 2: A decisive result is not complete coverage — Analysis: One true operand settles the disjunction. Returning here does not establish that the other operand produced a definite observation.
- Line 3: Branch order encodes the truth table — Analysis: Uncertainty is considered after decisive truth. Moving this condition first would change the result for True paired with None.
- Line 4: Preserve the state through the consumer — Analysis: None preserves an unresolved aggregate locally. Serialization and authorization logic must retain that distinction for it to remain meaningful downstream.
- Line 5: The negative conclusion has limits — Analysis: Within the three-value contract, reaching this line means both operands were False. It does not establish that every assessment required by policy contributed an operand.
Format: Code anatomy. Original source publication, as supplied: 9 October 2026, 23:36:48 UTC.
What is established
The supplied provenance identifies Microsoft PyRIT’s pyrit/score/true_false/true_false_score_aggregator.py, revision de2f04a637ea96eb8d84260bb40db026ab175bb4, as MIT-licensed hardening code. It bounds this excerpt to file lines 57–61.
Exact excerpt, attributed to Microsoft’s revision-pinned PyRIT source:
if left is True or right is True:
return True
if left is None or right is None:
return None
return False
Our analysis
Analysis: This boundary separates a definite negative observation from an observation that could not produce a verdict. For a concrete policy example, assume True means a condition requiring intervention, False means that condition was not found, and None means undetermined. The excerpt does not itself assign the intervention meaning. Under that hypothetical policy, authorization requires both adequate assessment coverage and no blocking finding. A false aggregate alone cannot demonstrate that every required assessment happened.
Line 1 asks whether either operand supplies decisive positive evidence. Line 2 returns that conclusion immediately. Line 3 considers uncertainty only after excluding a decisive positive; line 4 preserves the unresolved result. Line 5 is justified because, within the three-value input domain, excluding both True and None leaves only two false operands. These branches encode a small proof: uncertainty remains only when resolving it could still change the disjunction’s answer.
The order matters. For inputs True and None, learning the missing observation cannot make the disjunction false. Moving the uncertainty check above the positive check would therefore change the intended result. Conversely, with False and None, resolving the missing observation could produce either outcome. Returning false there would assert more than the available evidence supports.
This distinction also limits what the return value communicates. A true aggregate can coexist with an unobserved constituent. The helper preserves uncertainty about the combined proposition, not a complete inventory of assessment coverage. A defender may already have enough evidence to block an action while still needing to investigate missing checks. Store coverage separately when it matters for remediation, comparison, or later authorization.
The identity comparisons require canonical values. Consider a hypothetical direct caller passing integer 1 and False: neither operand is the singleton True or None, so the final branch would return false. That is a contract violation illustrating why annotations cannot replace validation. The surrounding adapter uses bool(score.get_value()) and catches UndeterminedScoreError. Our interpretation therefore depends on unknown scores reaching the exception path rather than arriving as ordinary falsey values.
For an ordinary Python runtime, our compatibility inference is Python 3.10 or later from the surrounding evaluated bool | None expressions; unseen dependencies may impose additional requirements. The excerpt is an indented function body, not a standalone program. Its interpretation assumes normal Python identity and control-flow semantics, without requiring native compilation or a particular operating system. Any translation into another language must preserve the explicit third state.
The obvious simplification, replacing this logic with ordinary truthiness aggregation, would discard the distinction between unknown and false. An apparently conservative alternative—converting every unknown to true—would instead confuse missing evidence with a positive finding. Neither substitution expresses a complete decision policy. Preserve the observation state first, then separately decide whether uncertainty requires a hold, additional assessment, or another documented response.
From an attacker’s perspective, under our hypothetical authorization policy, the useful mistake would be a defender interpreting an unobserved check as permission. From the defender’s perspective, the same boundary requires evidence before authorization. The basic principle is that absence of a verdict must not silently acquire the authority of a completed negative assessment. This describes a potential integration failure, not an observed attack against PyRIT.
Possible AI applications are also analysis. An adversary could use AI to inspect public evaluation code for places where unknown states become permissive decisions. Defenders could use AI to enumerate expected outcomes, inspect conversions, and suggest regression cases. Generated reasoning still needs checking against the actual value contract and decision policy. The supplied material establishes no incident involving AI use.
Defensive actions
Guidance based on our analysis: review the complete path from an individual observation to an authorization decision. Keeping None here is useful only if later layers preserve its meaning.
- Specify the contract. Document each verdict’s meaning and verify the representation of an undetermined score. Check whether invalid values are rejected or normalized deliberately. In particular, inspect whether an unknown can become false before this helper receives it.
- Verify coverage independently. Compare the assessments required by policy with those actually represented in the aggregate. The surrounding default empty-input path returns false. Treat that result as an implementation choice requiring explicit caller policy; it does not prove any assessment completed.
- Follow the value downstream. Review serialization, database fields, dashboard labels, numerical conversion, and decision code. Identify whether a missing field, null value, or default can turn uncertainty into a routine pass. Ensure explanations and coverage records remain attached to the verdict.
- Perform a paper regression exercise. Draw a three-by-three grid with
True,False, andNoneon both axes. Any cell containingTrueshould yield true. Of the remaining cells, those containingNoneshould remain unknown. Only the false/false cell should yield false. Swap operands and confirm identical answers. Then inspect whether existing tests cover all nine cells, plus empty input and incomplete coverage. Finally, walk each result through the documented authorization policy. This exercise requires no attacker code or execution.
Limits
Assessment limits: this is a static interpretation of one supplied file and bounded excerpt. No tests were executed and no independent reproduction was performed. Imported model behavior, deployed versions, and downstream policy are not established here.
The interpretation would need revision if actual callers supplied values outside the assumed domain. Observing an undetermined score returned as a falsey value instead of raising the named exception would falsify our assumption about uncertainty reaching this helper. Observing a consumer automatically authorize on None would disprove preservation through that decision path, while leaving the local truth table intact.
The supplied publication timestamp dates this source artifact; it does not establish when the technique originated. This analysis establishes neither a vulnerability nor an incident.
Research desk: AI-authored analysis, checked in a separate AI review against the linked public sources. No human expert review or independent exploit reproduction is implied.