← All updates

Engineering / Field note

A matching hash does not prove a test ran

An imported test report can match real code and output digests while its execution claim remains unverified. Established attestation guidance helps separate the bytes, the statement and the identity responsible for it.

By Zyel · Published · 4 min read

An imported test report can be internally consistent without establishing that its tests ran. Imagine a hypothetical report naming a real code revision and including a digest of its captured output. Someone could write the output themselves, calculate its digest and submit both. Every supplied byte could match while the claimed execution remains unsupported.

This November 11, 2026 field note applies established SLSA v1.2 and in-toto attestation guidance to imported test reports. The examples are hypothetical, and the reporting recommendations are Zyel’s interpretation. The useful distinction is between identifying the material, authenticating a statement about it and deciding whether to trust the particular claim.

A digest identifies the subject

The in-toto statement specification binds an attestation to its subject artifacts and identifies the type of its predicate, meaning the claim’s structured content. Each subject must have a digest. The specification says artifacts are matched by digest and assumes the subjects are immutable. [3]

That binding is valuable. In a hypothetical review, a result attached to one code tree should not silently become evidence for another. Likewise, changing a captured output file should cause its comparison with the recorded digest to fail. These checks can catch an accidental mix-up or a change to the material being reviewed.

The opening example presents a different problem. Its creator supplies a consistent collection of files and claims. There is no need to defeat the hash algorithm: the creator can calculate a perfectly valid digest of invented output. Internal consistency leaves the origin of the execution claim as an unanswered question.

A signature identifies an issuer

SLSA defines a software attestation as an authenticated statement about software artifacts. Its model separates the statement’s metadata from the signature that denotes who created the attestation. The artifact, the statement about it and the issuer therefore have distinct roles. [1]

Its artifact-verification guidance goes beyond checking a digest. It includes authenticating the provenance envelope, recognizing the builder through configured trust, and checking expectations such as the canonical source, build type and external parameters. [2]

That guidance concerns build provenance. Our application of the trust model to test reports is to ask which system is allowed to attest to a run, and what assertion it actually makes. A valid signature can authenticate the issuer of a statement; accepting that issuer for this purpose remains a policy decision.

For a hypothetical organization receiving a supplier’s test evidence, this might mean agreeing which runner or workflow is expected to produce the result and which delivered artifact the result concerns. A signature from an unexpected source would leave a different question open from a mismatched artifact digest.

Keep the verdict separate from the binding

Consider three hypothetical records. The first describes tests for a different code tree. Its result may be genuine, but the association with the code under review fails. Presenting a positive outcome for that code would lose the most important fact the comparison established.

The second has matching file digests, but its runner record reports a failing exit status. A captured sentence saying “tests passed” does not resolve that conflict. The report consumer needs a declared success rule and a clear way to show that the recorded outcome did not satisfy it.

The third has matching code and output digests but no accepted evidence of the producing execution. A useful description would preserve both observations: the bindings match, and execution remains unverified. Those statements can coexist. Collapsing them into a single green status would make the display claim more than the checks established.

These are suggested application-level distinctions. They help a reviewer see whether the question concerns the artifact association, the recorded verdict or the source of the assertion. Each uncertainty calls for different follow-up evidence.

The distinction needs to survive the dashboard

Our recommendation is to preserve those distinctions through generated pages, summaries and exports. If an imported record loses its trust label when a chart or outcome card is built, the final reader may receive a stronger assurance than the underlying record supports. Attractive formatting should make the evidence easier to assess.

A command recorded in an imported report is also a description to inspect. Re-execution belongs to an explicitly authorized workflow with reviewed inputs and an understood environment. Simply opening an evidence file should not grant its contents authority to start a process.

Even an accepted statement from a trusted runner has a scope. It can describe the code, conditions and outcome covered by that run. Whether the tests adequately represent the customer’s requirements is a further engineering judgment, supported by the test design and other evidence.

A useful report makes that scope visible: which bytes the claim concerns, who made it, what the recorded outcome was and which checks the reviewer accepted. Hashes remain an important part of that account. Their value is clearest when the report asks them to establish exactly what they can.

Sources + further reading

Source dates describe the original material. Publication above is the date this Zyel article went live.

  1. SLSA v1.2 software attestation modelSLSA · Checked Nov 11, 2026
  2. SLSA v1.2: verifying artifacts and provenanceSLSA · Checked Nov 11, 2026
  3. in-toto attestation statement specification v1in-toto project · Checked Nov 11, 2026

Reader discussion

What would you ask next?

Questions, useful corrections and thoughtful disagreement are welcome. Comments are reviewed before they appear.

No reader comments yet.

10–2,000 characters. Plain text. Leave out passwords, private project data and personal contact details. A blank display name appears as Anonymous.

Complete the CAPTCHA before sending. Verification runs on your device.