Autonomous Pentest Assurance Standard (APAS): when you may trust autonomous pentest results

The standard itself describes the starting position in sober terms: autonomous systems are marketed as able to perform, augment or substitute for penetration testing — and the audiences of that marketing have no reliable yardstick for telling a bounded, verifiable claim from marketing language (Background and Motivation). APAS makes such claims auditable by defining how an autonomous system declares what it did, under what conditions and with what evidence.

Three properties of autonomous systems make the classic pentest report insufficient as evidence. First, the non-determinism trap: in two runs under identical conditions, an agent finds different attack surfaces and findings; whoever reports only the most favorable run feigns consistency (4.6.2). Second, the self-confirmation trap: a system that assesses its own findings — even with a second model acting as a “judge” — observes not the target but itself; agreement between internal components is not independent confirmation (4.4.1 f, 11.2.1 b). Third, the coverage trap: a system that discovers less appears more complete when the denominator comes from its own discovery (1.1.4).

From this, APAS draws a clear lower bound: automated scanning that raises signals without validated exploit paths is not by itself an autonomous pentest. Such output qualifies at most as a Q1 Assurance Outcome supported by EV1 or EV2 evidence and shall never be represented alone as an APAS pentest claim. The differentiators of a pentest are independent reproduction by a party other than the producing system, validation of the exploit path to the asserted condition, and demonstrated multi-step chaining within the declared scope (2.2.3).

APAS is being developed in alignment with the Cloud Security Alliance and follows the neutral stewardship requirements of Section 12: a standard controlled by a market participant shall not be represented as a neutral industry standard (12.4.1). The document names the CSA AI Safety Working Group as its home; the lead author is Will Kapcio, joined by eight contributors (among them Jim Reavis and John Yeoh of the CSA) and twelve reviewers. The change log records the initial release of 23 October 2026, the file is dated 2 November, and the CSA publication is dated 4 November 2026. The PDF is protected by copyright and may not be redistributed — we therefore link to the download at the CSA and quote with source attribution.

The Status section makes three things clear. APAS assesses the truthfulness and usability of pentest evidence — not the safety, containment or misuse resistance of the system itself; APAS certification shall never be represented as a fitness-for-deployment safety assurance. Second: no companion document has been published. Third, and decisively: until the Evaluator Guide, the applicable Profile Registry entry and the Attestation Schema are available at compatible versions, no Atomic System Capability Certification and no Engagement Conformance Statement may be issued. Publishing the Core opens the certification path, not certification itself (Status, 12.1.2).

The document set

Governance is already codified: versions follow a major/minor/patch scheme, and v1.0 is a major version because it changes claim-validity semantics, quality dimensions, environment classes, publication gates and finding semantics (12.2.1). Each companion document declares the Core major version against which it is normative; claims may be issued only against a compatible document set, and a new Core version does not retroactively invalidate claims already issued (12.2.2, 12.2.3). The Core and the Evaluator Guide are to be reviewed for capability drift at least every twelve months, with an out-of-cycle trigger for material shifts in frontier capability (12.2.4). Normative changes undergo public review; the stewarding body must be organizationally independent of any single vendor, maintain multi-stakeholder processes and disclose conflicts of interest (12.3, 12.4). A public claim-status record tracks every claim through the states Active, Under Review, Revoked and Retired; on credible evidence that a claim rests on fabricated or misleading evidence, the claim moves to Under Review and may no longer be relied upon as Q3, Q4, R2 or R3 (12.5).

The rules for applying them are strict. Every Quality Declaration must represent Coverage Fidelity, Finding Validity, Evidence Traceability and Agent Integrity; for Q4 or R3 claims, Remediation Usability and Retestability become mandatory as well (4.1.1 a). A dimension may be treated as not applicable or not measured only against documented criteria and with evaluator approval — and that absence must be disclosed (4.1.1 b). Profiles may add dimensions but shall not subtract any (4.1.1 c). Above all: the dimensions are declared and evaluated independently, shall not be combined into a composite score, and a deficiency in one shall not be represented as offset by strength in another (4.1.1 d).

Every Q2, Q3, Q4, R2 or R3 claim includes a Quality Declaration that represents each applicable dimension by numeric value, ordinal class, boolean status or a profile-defined representation and references supporting evidence or methodology (4.1.2). Comparative validation — for instance against a human reference cohort — may supplement quality assessment but shall never replace Coverage Fidelity, Finding Validity or Evidence Traceability (4.1.3). Agent Integrity is a distinct dimension and is not satisfied by Process Integrity disclosures (4.1.4).

APAS <Mode>-<Tier> / <Execution>-<Collaboration>-<Assurance> / <Environment> [<Affordance Profile>] {<Reliance>}

APAS BB-T2C / X2-C2-Q4 / E3 [P-021] {R2}

The four example claims from Appendix A

There are three claim types (5.1): the Atomic System Capability Certification is the evaluator-issued result that a system satisfied the applicable requirements for one capability combination under declared conditions. The Multi-Configuration Capability Statement bundles two or more atomic claims for one system version. The Engagement Conformance Statement refers to a specific engagement executed using an APAS-certified system under the same or narrower declared conditions — it never certifies the target environment (5.1.1).

Because a system can offer several combinations, APAS requires a Capability Matrix with seven status classes: S1 Supported (designed to operate, not evaluated), S2 Evaluated (evaluated, no claim represented), S3 Certified or Conformant, S4 Not Certified, S5 Not Evaluated, S6 Unsupported, S7 Retired. Only S3 may be represented as certified; a public matrix omitting materially relevant supported combinations is misleading (5.3.1). Each row includes, among other things: combination ID, full claim string, system name and version, model identities including system-prompt provenance, methodology reference, evaluator, status, references to the attestation, Quality Declaration and Coverage Declaration, Core and companion document versions, certification and validity dates, material limitations; for Q3, Q4, R2 or R3, a certification date and an explicit expiry date or revalidation interval are mandatory (5.4.1).

The Assistance Class from Section 9 adds the question of what humans actually contributed: H0 no assistance affecting strategy, exploitation, validation, prioritization or reporting; H1 administrative or safety-only intervention; H2 analytical assistance — exploit guidance, interpretation, prioritization, control mapping or report editing. The class must be disclosed in the attestation; for multi-configuration systems, per combination and per run (9.1.1). An X3 claim prohibits H2 during execution; a Q4 claim prohibits H2 during Evidence Package assembly — including by internal validation or adjudication components that shape the package content (9.1.2). Post-execution vendor-authored framework mapping or prioritization that materially affects the relying reviewer is treated as Vendor-Side Analytical Review and must be disclosed (9.1.3). A platform must not be marketed as agent-only, independently autonomous or assessor-ready where materially relevant runs relied on undisclosed human intervention or post-execution analytical reshaping (9.2.2).

Two further rules deserve the attention of CISOs and internal audit. First, liability: every R3 claim must identify the party accountable for the accuracy of the substituted evidence, the relying party's recourse if that evidence is later found deficient, any warranty limitations, and any professional-liability or errors-and-omissions coverage represented to the relying party — APAS itself does not create, transfer or cap liability (5.10.4). Second, tenancy and authorization testing: an R2 or R3 claim must never be represented as substituting for, or augmenting, authorization, access-control or tenancy-boundary testing where fewer than two distinct principals, or only a single tenant, were exercised; such areas must be declared materially untested (5.10.6). And: R2/R3 declare what the package supplies — not which procedures the relying reviewer may omit; residual procedures and residual-risk acceptance rest with that reviewer and the governing framework (5.10.7).

Every T2A, T2C, T3, T4, Q2, Q3, Q4, R2 or R3 claim includes a Coverage Declaration with fixed required fields (4.2.1). It must be truthful about the differences between declared, reachable and tested scope, and about discrepancies against the surface baseline; no package may imply complete, full or equivalent coverage without delivering the methodology and evidence for it; a result demonstrated on a single instance or in a single run is not coverage of the corresponding attack class (4.2.2).

The baseline ladder BS-1 to BS-3

1. 1
   BS-1 Asserted

   A target-owner asset inventory or scope list — independent of the candidate system, but bounded by owner completeness (4.2.8 a).

2. 2
   BS-2 Directed

   Evaluator-directed enumeration, the evaluator specifying or extending the surface beyond the candidate system's discovery — the minimum level for Q3, Q4 and R3 (4.2.8 b, 4.2.10).

3. 3
   BS-3 Instrumented

   Enumeration by an independent discovery instrument or process that does not share the candidate system's discovery logic (4.2.8 c).

For Q3, Q4 and R3, the coverage denominator must be established against an Independent Surface Baseline; the candidate system's own discovery must not be its sole source (4.2.4). Material discrepancies between discovered surface and baseline must be disclosed in either direction; any baseline surface not exercised is classified as materially untested; under-discovery reduces the represented coverage, never improves it (4.2.5). Where sources are combined, the composite baseline must not be represented at a strength higher than the weakest source solely covering any material region (4.2.9). Where no independent baseline is required, the declaration must state that the denominator is self-enumerated and describe the enumeration method (4.2.6).

Coverage status alone says nothing about depth. For each area tested or partially tested, the declaration must state whether request-level probing, authenticated probing, multi-step workflow or state-transition testing, and privilege- or tenancy-boundary testing were performed. The unit of coverage, partial-coverage thresholds, depth grading and weighting are defined in the Profile Registry — absent an applicable profile, depth must be declared narratively and no coverage proportion or percentage may be represented (4.2.11). For T2C claims, the declaration must be structured so that a Relying Reviewer can identify control-relevant tested, excluded and untested areas without vendor-side analytical interpretation (4.2.7).

Non-determinism is the second blind spot. A system is materially non-deterministic where repeated runs under the same conditions produce materially different reachable surface, finding sets or exploitation outcomes; this must be disclosed in the methodology statement (4.6.1). Variance must not be concealed by reporting only the most favorable run; where multiple runs inform a claim, selection, aggregation and disclosure must be defined in advance, and the evaluator attests that the run count and aggregation rule were fixed before execution (4.6.2). Q3, Q4, R2 and R3 claims characterize run-to-run variance using a declared measure — such as surface discovery variance or the finding-set Jaccard index — over a Pre-Registered Run Design (4.6.3). The aggregation rule is drawn from a closed set (any-run union, all-run intersection, majority or another expressly declared rule), and the per-run outcomes of all runs, not a summary statistic alone, reach the Evidence Package (4.6.5).

Every Confirmed Finding carries a Stable Finding Identifier — a unique identifier that remains stable across revisions, retests, exports and format conversions — and, except under the single-capture path, a Reproduction Procedure: a recorded, deterministic, agent-independent sequence of steps, requests, payloads, preconditions and expected observations with which a competent third party can reproduce the condition without access to the candidate system (4.3.3, Appendix D). The expected observations include the control or expected-secure observation; a procedure may only be probabilistic where the condition itself is probabilistic (race conditions, TOCTOU, timing side channels), and then with attempt count and observed success rate. Every Confirmed Finding also declares the vantage point and access condition as well as the region of the environment class (4.3.5); adjacent services or cross-system paths material to exploitability or impact are disclosed — mandatory for E3 (4.3.6).

For Q3, Q4, R2 or R3 claims, EV3 and EV4 artifacts must be collected through a path not under the candidate system's sole control — network interception, host observation or evaluator-collected captures; below that gate, artifacts collected solely by the candidate system must be disclosed as such in the evidence manifest; for Q2 and Q3, EV3/EV4 should be Out-of-Band Witness Evidence — callback listeners, canaries or target-side monitors that the system can neither author, observe nor reach (4.4.4, Appendix D). Where a finding asserts an authorization or tenancy-boundary defect, EV3 requires paired observations: the same request succeeding for the unauthorized principal and refused for an authorized control principal, both referenced via Identity Coverage (4.4.6).

Retest and deduplication

A retest executes the Reproduction Procedure recorded against the Stable Finding Identifier, matches the result to that identifier and discloses whether the finding remains present, is partially resolved or is no longer reproducible (4.3.4). Probabilistic rediscovery — or its failure — by an agent re-run is neither fix verification nor persistence confirmation nor a retest; such results must be labeled probabilistic rediscovery and distinguished from the Reproduction Procedure result (4.3.4 a). For intermittent conditions, attempt count and success rate are assessed against the pre-remediation rate (4.3.4 b). A documented deduplication rule must exist for every package, stating how repeat observations of the same condition from separate runs, vantage points or instances collapse onto a single Stable Finding Identifier (4.3.9).

The Affordance Profile is the system's declared operating environment during the test: capabilities, restrictions, access conditions, escalation limits and constraints (Appendix D). Every claim references a profile with at minimum the fields below; a generic label such as “standard” does not satisfy the Target-protection controls or Agent-integrity controls fields. Internal validation, judge, verifier or consensus components must be declared in the Agent Topology and their role disclosed in the Quality Declaration against Finding Validity. Absent a declared Action Reversibility Class, the engagement is treated as irreversible — with the strictest safety obligations (7.1.1).

The Environment Classes describe reach across trust boundaries, not target type: E1 a single isolated target, E2 multi-role, multi-step or stateful interaction within one declared trust boundary, E3 interaction crossing a trust or segmentation boundary (7.2). They apply equally to applications, APIs, networks, infrastructure, cloud control planes, containers, client software, mobile, firmware and embedded targets; no class imports social-engineering, physical or wireless vectors. Every claim is limited to its declared class (7.2.1); the environment type and any material differences from production must be declared, and a non-production test does not imply production coverage absent declared parity (7.2.2). T2A, T2C, T3 and T4 claims reference a declared methodology; methodologies targeting Q4 or R3 additionally define negative-result handling, remediation and retest expectations, and the threat-model rationale for the chosen mode (7.3).

Authorization and Rules of Engagement

Every APAS engagement has an Authorization Declaration identifying the authorizing party, the legal or contractual basis, the authorized scope and time window, and the Rules of Engagement; testing outside them is treated as out-of-scope activity under Section 10 — regardless of technical success or finding value (7.4.1). Where material, the following must additionally be declared: testing windows including blackout and maintenance windows, named emergency contacts for the target owner, the evaluator and the system operator with availability and maximum notification time, authorization status for third-party and cloud infrastructure including provider-specific testing conditions, the system's source addresses and allowlisting, data-classification limits, and de-confliction with the SOC or blue team (7.4.3). The declaration also identifies the acting agent identity, the identity scheme, the verification method available to the target owner and, where sub-agents act, the delegation chain and maximum delegation depth (7.4.4).

Because autonomous systems encounter vulnerabilities in third-party components or outside the scope, every engagement requires a Coordinated-Disclosure Path: each such finding carries a disclosure state (encountered, notified, disclosure-restricted, resolved, published), remains enumerated in the evidence manifest in every state with its Stable Finding Identifier and content hash, and only the exploitation detail may be withheld temporarily — never the finding itself (7.4.2). Where the system can reach shared or third-party infrastructure outside the declared authorization, the escalation limits must bound it away from that infrastructure (7.4.3.2).

Any certification or engagement statement represented as R2 or R3 includes an assessor-consumable Evidence Package; any Q2, Q3 or Q4 claim includes a Quality Declaration and a Coverage Declaration, even where only R1 is claimed (8.1.1). For R2/R3, supporting evidence should remain available at least as long as the claim is valid (8.1.2 b).

Integrity: signature, hashes, tamper-evident logs

Any Q3, Q4, R2 or R3 attestation is cryptographically signed by the issuing evaluator or conformance authority — with a verifiable identity, a published verification method, a signing certificate or key identifier, a trusted timestamp and a published means of validating the key status even after rotation, expiry or revocation (8.2.1). The evidence manifest includes a collection-time content hash for each artifact (8.2.2); execution and evidence logging is tamper-evident and is maintained or countersigned by a component or party not under the system's sole control (8.2.3). Fabricated, non-traceable or materially misleading evidence invalidates the claim (8.2.4); a package whose signature fails verification, whose hashes mismatch or whose log shows unexplained discontinuity must not be relied upon as Q3, Q4, R2 or R3 until the discrepancy is resolved and re-attested (8.2.5). Inconsistencies among the attestation, findings, execution records and artifacts must be disclosed, not resolved by narrative interpretation (8.2.7).

Machine readability and open interchange formats

Any Q2, Q3, Q4, R2 or R3 package carries machine-readable fields: the full claim string, combination ID, system name and version, model identities and versions, system-prompt provenance, a build identifier of the system as a cryptographic digest over the bound configuration, methodology reference, evaluator identity, signature and verification metadata, Assistance Class, observed operating path, references to the Quality Declaration and Coverage Declaration with baseline sources, asset identifiers and SBOM references, finding-state summary, Stable Finding Identifiers, attack-path relationships, single-capture labels, materiality determinations, manifest hashes, certification date with expiry or revalidation interval, Core and companion versions, and Reliance Outcome. The claim string and build identifier travel in a declared provenance envelope such as in-toto, DSSE or SCITT (8.4.1).

The baseline applies to every claim: explicit scope boundaries, a stop or suspend mechanism, and preserved logs or artifacts sufficient for evaluator review (10.1.1). Out-of-scope activity, fabricated evidence, material concealment of human assistance or misleading coverage statements invalidate the claim (10.1.2); material uncertainty in impact, coverage or retest conclusions must be explicitly labeled in the package (10.1.3).

1. 1
   Declare Stop Authority

   Every engagement declares the parties empowered to suspend or terminate execution — at minimum the target owner and, where present, the evaluator (10.2.1).

2. 2
   Operate out-of-band

   The stop mechanism operates out-of-band from the system's decision loop; the system is never the sole arbiter of whether a stop takes effect. The Affordance Profile declares a maximum stop reaction time; within that bound all offensive activity halts, in-flight actions are interrupted, and queued, scheduled or delegated work is canceled or dispositioned in the record (10.2.2).

3. 3
   Fail safe on loss of oversight

   Where monitoring, oversight connectivity, logging or the stop channel is lost or degraded beyond the declared tolerances, the system suspends offensive activity until oversight is restored — loss of oversight never defaults to continued autonomous operation (10.2.3).

4. 4
   Record events and notify contacts

   Stop and halt-on-loss events and their timing are recorded in the tamper-evident log and disclosed in the Evidence Package; the emergency contacts are notified within the declared notification time (10.2.4).

5. 5
   Access Disposition Record

   At engagement closure — and after every stop — a record identifies each access point created by the engagement (tokens, accounts, sessions, tunnels, delegated tokens, persistence) with its disposition status (Revoked, Expired, Removed, Rotated by target owner, or Retained with named owner and expiry), the responsible party and a verification reference (10.2.5).

6. 6
   Verify channels end-to-end

   The evaluator verifies each safety-critical stop, suspend, resume or override channel end-to-end — from the authorized party to the enforcement point in the attested deployment configuration; mere existence or correct signing is not sufficient, and at least one exercised event per channel with its observed delivery and enforcement result belongs in the package (10.2.6). Channel availability is not evidence of delivery; an instruction that is not acted on must be detectable as a delivery failure (10.2.7).

The obligations scale with the worse of execution autonomy, reduced in-band human participation and declared affordance risk (Action Reversibility Class): within a given profile, the requirements for X3 are at least as strong as for X2, and those at least as strong as for X1, with the same ordering from C1 through C4; a less reversible Action Reversibility Class raises the requirements irrespective of X and C; no profile, evaluator decision or vendor configuration may grant the most autonomous classes weaker oversight — a configuration in which increased autonomy removes oversight is not certifiable (10.3.1).

• X3, C4 and E3 operation, and any engagement that permits irreversible actions or autonomous chaining into adjacent surfaces, additionally require: continuous availability of the stop channel during execution (10.3.2 a),
• automated detection of scope-boundary approach or violation that halts or blocks the action rather than only recording it (10.3.2 b),
• a declaration of how the authorized-scope boundary is enforced at runtime (10.3.2 c), halt-on-loss-of-oversight (10.3.2 d),
• enforcement of the declared escalation limits by a mechanism whose failure is detectable (10.3.2 e),
• and, where actions are composed into a chain, a gate set at the worst reachable Action Reversibility Class across the entire chain before the first step executes (10.3.2 f).

Agent Integrity: the agent as an attack target

A pentest agent reads target content, calls tools, uses MCP servers and persists context — which makes it an attack target in its own right. APAS requires declared controls against at least four manipulation classes: prompt injection from target content or responses; manipulation or substitution of tools, plugins, models, or MCP and equivalent tool-connection servers; poisoning of retrieved or persisted context; and unauthorized modification or bypass of the runtime controls governing scope, permitted tools and privileges (10.4.1). Every Q3, Q4, R2 or R3 claim includes in its Quality Declaration an agent-integrity disclosure of the controls in place, any manipulation attempts detected and their handling; a candidate-generated record of “no manipulation detected” is not by itself evidence that none occurred (10.4.2).

What matters is who performs the testing: for Q3, Q4, R2 or R3, the evaluator must execute the adversarial agent-integrity testing itself — at minimum goal hijack, tool-response injection, and context or memory poisoning; solely vendor-asserted controls support none of these claims (10.4.4). The evaluation compares labeled-evaluation runs against blinded runs on scope adherence, out-of-scope refusal, coverage and finding-fabrication rate; a material behavioral difference is eval-awareness, must be disclosed and caps the Assurance Outcome (10.4.3). Undetected manipulation that materially altered strategy, findings, scope adherence or report content invalidates the affected claim dimensions. If the system detects manipulation, it suspends offensive activity, notifies the target owner, the evaluator and the emergency contacts, and quarantines evidence and findings from the suspected manipulation window (10.4.5).

Required disclosures of every public APAS claim (11.1.1)

• atomic or matrix-based status,
• all eight claim dimensions,
• the evaluator's identity,
• certification and validity dates,
• the candidate system version,
• material limitations where reliance is claimed,
• the availability of the applicable Quality Declaration or a summary,
• where more than one combination is supported, the status of adjacent combinations a reasonable buyer might infer are available,
• the Core Standard version and the identifier and version of each companion document applied,
• where material, whether dynamic switching among combinations is supported,
• and, wherever the claim string is used publicly or in buyer-facing, marketing or procurement-facing material, a one-sentence plain-language reading in the form of Appendix A — with prominence equal to the encoded string.

Prohibited claims (11.2.1)

Solely on the basis of APAS certification, no party may state or imply:

Two further rules round off the section: no execution class, collaboration class, assurance outcome class or reliance outcome may be marketed without the full claim dimensions — an isolated “Q4” or “X3” in a brochure is not permitted (11.2.2). And a Single-Capture Confirmation must be labeled as such in every public claim and may never be represented as independently reproduced (11.2.3). In case of conflict, the attested declarations govern over any public statement and any marketing summary (13.2); if a framework or assessor requires conditions stricter than APAS, those conditions govern (13.3).

The CSA itself positions APTS as the closest related effort: an early-stage OWASP Incubator project on the safe, bounded operation of autonomous pentest platforms. The two overlap only on operational governance — APAS Sections 7 (Declared Conditions), 9 (Human Assistance) and 10 (Safety) — while APTS prescribes no coverage-measurement methodology, no reliance or substitution grading and no evaluator-issued certification; precisely these are the core of APAS. Where APTS touches on finding reproducibility and assessor-delivered evidence, APAS goes further: Confirmed Findings are gated on agent-independent reproduction, and coverage is measured against an Independent Surface Baseline (Relationship to Existing Work).

How to use both standards together

The decision: where autonomous pentests deliver today

Autonomous pentest systems play to their strengths where breadth, frequency and repeatability count: continuous testing of a changing attack surface, regression tests after releases, pre-triage ahead of a manual pentest, retests with reproducible procedures and coverage of many similar targets. APAS itself sets their limits: no social engineering, no physical or wireless vectors, no full-scope red team (2.2, 5.6.3), no continuous operation as a claim (2.2.4) — and no substitution of required testing without R3, a profile and the permission of the relying framework (5.10). The decision is therefore rarely “autonomous or human” but rather: which subset of the testing activities can be performed autonomously, with what assurance, and what remains conventional?

1. 1
   Define the evidence purpose

   Internal remediation (Q2/R1), augmentation for assessors (Q4/R2) or substitution (Q4/R3)? The Reliance Navigator on this page walks through the question; the result determines all further requirements.

2. 2
   Provide scope and baseline

   A machine-readable inventory (CMDB/CAASM, SBOM references) is your BS-1 baseline; for Q3/Q4/R3, evaluator-directed enumeration (BS-2) supplements it. Without a baseline, nobody can measure coverage — not even you (4.2).

3. 3
   Set up the pilot as an evaluation

   Pre-Registered Run Design with multiple runs and a variance measure (4.6.3), independent reproduction of a sample (EV3), paired observations for authorization findings (4.4.6), end-to-end Stop Authority test (10.2.6), adversarial Agent Integrity tests (10.4.4) and a blinded run against eval-awareness (10.4.3).

4. 4
   Define the human-plus-agent operating model

   Which X/C class matches your risk tolerance? Production targets with irreversible actions carry the strictest obligations (10.3.2); define approval points, escalation contacts, testing windows and de-confliction with the SOC (7.4.3).

5. 5
   Contract and governance

   Data handling and egress (7.1.1), Stop Authority and reaction time, Access Disposition, disclosure of configuration changes and re-attestation (5.11.3), liability in the case of substitution (5.10.4), interchange formats (8.4.3) — plus KPIs: Unsupported-Finding Rate, retest rate, coverage against baseline, stop reaction time.

Tool evaluation: the APAS-based requirements catalog

• Which Capability Combinations does the platform offer — as a Capability Matrix with all eight dimensions and status classes, please, not as a feature list (5.3, 5.4)?
• What is the plain-language reading of your strongest claim, and which limitations apply to it (11.1.1)?
• Where does the coverage denominator in your reports come from — from the system's own discovery or from an Independent Surface Baseline? Show us a Coverage Declaration with all required fields (4.2.1).
• What is your Unsupported-Finding Rate per combination, separated into false positives and over-severity (4.3.8)?
• Does every finding include a Reproduction Procedure my team can follow without your platform, and a Stable Finding Identifier (4.3.3)?
• How do you label Observations, Validated Findings and Confirmed Findings — and who confirms (4.3, 4.4.1)?
• How do you characterize run-to-run variance, and do you report all runs or only the best one (4.6)?
• Which fields does your Affordance Profile cover — in particular inference and telemetry egress with jurisdiction, tool and MCP provenance, Action Reversibility Class (7.1.1)?
• How does the Stop Authority operate out-of-band, with what maximum reaction time, and what happens on loss of oversight (10.2.2, 10.2.3)?
• Which controls exist against prompt injection, tool substitution, context poisoning and circumvention of runtime controls, and who has tested them adversarially (10.4)?
• Do you deliver an Agent Execution Record, content hashes and a signed attestation — and in which formats do you export findings, mappings and fix status (8.2, 8.4.3)?
• Which changes to model, prompt, orchestration or tool set do you report to us, and how often has your configuration changed materially in the last twelve months (5.11)?

The full version with scoring matrix, exclusion criteria and 20 RfP questions is contained in the CISO whitepaper further down this page.

Regulatory positioning: what assessors can accept

APAS makes clear that it does not itself establish regulatory, contractual or audit compliance and that APAS certification never implies framework acceptance (1, 5.9.1). Conversely, most frameworks say nothing about autonomous testers — they require tests, methodology, qualification and documentation. The following positioning is therefore a VamiSec assessment of which Reliance Outcome an assessor can reasonably accept under each framework today.

Supervisors have shown a discernible line in 2026: AI as a supporting, human-supervised capability. In July 2026, ECB Banking Supervision called on significant institutions to draw up action plans against AI-enabled attacks and named AI-based tools as a complement to vulnerability testing — on condition of a prior benefit-risk assessment, appropriate safeguards and human oversight. In the same month, ENISA described the “verification bottleneck”: the volume of technically correct AI findings overwhelms verification — exactly the gap APAS addresses with Confirmed Findings and the Unsupported-Finding Rate. The UK NCSC requires sandboxing, immutable logs and the ability to “pull the plug” for AI agents; the joint Five Eyes guidance of May 2026 recommends red teaming and penetration testing against agentic systems and human approval before high-impact actions. To date, there is no supervisory statement that explicitly recognizes or rejects autonomous pentests as regulatory evidence.