verify-action-build: verify in-tree binaries (attestation / SHA256SUMS)#819
Conversation
|
My approval is contingent on @gmcdonald confirming that If it still needs to be used then I think we will need an allow-list of actions that pass verify. Maybe these are ones we can wildcard. |
Note that failing the test does not mean rejection. It just means failing the test. It can still be approved. |
Also - if you look there runs-on/action#36 The maintainer had been very responsive and implemented attestation for building those binaries (cc: @ppkarwasz ) - so we still might allow the actions where attestations are present - they do not give us 100% reproducibility - but I think the trust in this case moves from the author to GitHub + review of the build steps. Am I right Piotr? |
potiuk
changed the title
Also ... I added (as a follow-up after the fix implemented by @crohr in runs-on/action#36 -> this check will now only fail if we "can't" verify the binary - wiht either binary attestations or SHASUM approch. This PR has been tested on #825 and it correctly verify the attested binaries - with SLSA attestations of build: 
Where it fails on previous version of the same bump, where the action did not have attestations: 
Which is probably exactly what we want. |
potiuk
force-pushed
the
reject-in-tree-binaries
branch
from
2e200bb to
3f2fc7e
Compare
…n-tree Add `analyze_in_tree_binaries` to flag actions that commit native binaries (Go cross-compile, .exe/.dll/.so/.dylib/.jar/.wasm, etc.) directly in their repo and exec them from a small launcher. The JS-rebuild check verifies the launcher; the binaries are opaque executable code that no part of our pipeline can reconcile with source. runs-on/[email protected] (apache#809) is the case in point — ~10 MB of UPX-packed Go binaries run as root. The check is path-only (cheap, no fetch): known binary extensions plus the <name>-<os>-<arch>(.exe)? cross-compile suffix. Add the new failure path to the verification summary and overall result. Also document the criterion in the security review checklist and add case F (in-tree native binaries) + a runs-on/action precedent to the analyze-action-pr skill.
…6SUMS The previous In-tree binary check was a flat reject of any pre-compiled native binary in an action's tree. That blocked runs-on/action and similar Go-based actions outright, with no path forward except "wait for upstream to fix it". The runs-on/action maintainer responded to our upstream issue (runs-on/action#36) by wiring actions/attest-build-provenance into the release workflow and shipping a SHA256SUMS asset (runs-on/action#37) — exactly the chain we asked for. v2.1.2's binaries are now verifiable end-to-end. Extend analyze_in_tree_binaries to use that chain. Each detected binary is verified through a cascade: 1. gh attestation verify --owner <org>: queries GitHub's attestation transparency log; ties the binary's SHA256 to the workflow run that produced it. 2. SHA256SUMS release asset: fetches the release's SHA256SUMS, compares each binary's filename + SHA256 hash. 3. Neither: hard fail (the original behaviour, kept for actions with no provenance chain). Verified binaries are reported ✓ with the verification mechanism. Unverified binaries fail the action with the offending paths so reviewers can confirm. 8 new tests covering each branch of the cascade plus 6 tests for the SHA256SUMS parser. Update SKILL.md case F + README review checklist to reflect the new "verified ✓ / unverified ✗" semantics; add a runs-on/action v2.1.2 precedent showing the verification chain in action.
potiuk
force-pushed
the
reject-in-tree-binaries
branch
from
3f2fc7e to
a374d05
Compare
2 participants
Summary
verify-action-build. Detects actions that ship pre-compiled native binaries directly in the repo (Go cross-compiled likemain-linux-amd64, plus*.exe/*.dll/*.so/*.dylib/*.jar/*.wasm, etc.) and exec them from a small launcher.gh attestation verify --owner <org>— queries the GitHub attestation transparency log (populated byactions/attest-build-provenance).SHA256SUMSrelease asset — fetches the release's checksum file and SHA256-compares each binary.analyze-action-prskill, withruns-on/[email protected](rejected — no provenance) andv2.1.2(verified — both attestation and SHA256SUMS) as precedents.Test plan
uv run pytest utils/tests/— 236/236 (15 new for in-tree binaries + 6 new for SHA256SUMS parser).prek run --all-filesclean.runs-on/[email protected]): no provenance — 3 binaries flagged as unverified, action fails.runs-on/[email protected]): upstream landedactions/attest-build-provenance+SHA256SUMSafter our issue runs-on/action#36 — all 3 binaries verified viagh attestation verify, action passes (exit 0).Background
Closes the gap surfaced by #809 (
runs-on/[email protected]). The maintainer (@crohr) responded to our upstream ask by adding both verification mechanisms in runs-on/action#37; v2.1.2 is now fully verifiable, which is what #825 (the v2.1.2 dependabot bump) needs from us.Generated-by Claude Code.