[Idea] A tool to download artifacts by checksum

[justaugustus]

I wonder if we have any interest in creating a simple go tool that can download things by checksum? It feels very much in sig-release scope.

I was thinking about something like kubernetes/kops#17240

The idea would be that you could pass hget sha256:val urls... and it would go download from the canonical URLs, check the sums etc. It's a trivial program to write and I figure artifact distribution would be so much better (particularly if we one day got it into the distros)

The "killer feature" here I think would be to also support extensions, possibly things like hget-gcs or hget-s3, or maybe a directory of extensions, so that we could fetch from local mirrors easily.

Another thing I think would be awesome is if we supported transparency logs (rekor?), so we could go from artifact name -> sha -> download from mirrors (including user configured ones and ones in rekor).

cc @cpanato

Originally posted by @justinsb in #1543

[justaugustus]

@justinsb — my first thought was "Surely someone has already created this. I swear I remember something like this!"

...and so, some minor archeology began!

The first thing I recall is @philips' (Hi Brandon!) rget: https://merklecounty.substack.com/p/rget-a-secure-download-user-story / https://github.com/merklecounty/rget

rget was archived in 2020:

Archived Project Warning: rget is archived. Architectual issues made the project unmaintainable longterm (see issue). The rearchitected spiritual successor is under development: see transparencylog/btget.

btget redirects to transparencylog/tl, which was archived in 2021 (this was around the time that sigstore was emerging):

Project Archived
This project is archived at the transparency log service at https://beta.transparencylog.net will be shutoff on November 9th 2021.

If you are looking for an assest signing and verification system that is backed by a transparency log checkout the sigstore/cosign project. In particular they have a subcommand called sget that offers a sort of similar guarantee although not across any arbitrary URL.

A search of the cosign repo eventually gets you to https://github.com/sigstore/cosign/blob/6431af15a8066c4b33c7232fc2dba3f9278a16a5/CHANGELOG.md?plain=1#L1017:

Add deprecation warning for sget CLI and packages (sigstore/cosign#2019)

(uh oh)

From there, we head to https://github.com/sigstore/sget, which was archived in 2023.
A deprecation issue in that repo leads us to sigstore/docs#29 (comment):

@imjasonh are we deprecating github.com/sigstore/sget too? Maybe I misunderstood.

I think that would be best. I don't think there's any plan to continue developing it at this time.

---

So yes, this is a great idea because lots of smart folks I know have had it in the past!

It feels very much in sig-release scope.

Yes, I think this reasonably falls within the scope of SIG Release as a helper util for some of the work we do (I could see intersections w/ the following SIGs as well: Security, K8s Infra, Testing).

BUT:
Before we do anything here, we should solicit more info from these lovely folks to identify reasons that the various tools were deprecated.

@cpanato — Can you weigh in from the Sigstore side?

---

While this is in-scope for us within the Kubernetes project, it is probably even more appropriate for this to be considered in the Open Source Security Foundation (OpenSSF), perhaps around the Security Tooling Working Group.

@SecurityCRob @ware — have you seen [active] projects around secure download command-line utilities in your travels?

[justaugustus]

More in "have people wanted to do this before", curl edition:

• https://curl.se/mail/archive-2025-01/0019.html
• [feature] Checksum when downloading file curl/curl#1399
• Checksum of file while downloading curl/curl#6851