Upload artifacts as tagged release

[sandyspiers]

This is a continuation of #22 .

I have updated the code to use gh release create and gh release upload, and has some unit tests.

The way we get the repo name has change slightly, and instead uses whatever is set default for gh, i.e., gh repo set-default owner/repo.

These unit tests will pollute the releases of this repo, so I am happy to change the test strategy if preferred by the maintainers.

Thanks :)

[sandyspiers]

Looks like CI is failing because there is no GH_TOKEN for gh. I'll avoid adding it just yet, in case it pollutes the releases of this repo.

[simeonschaub]

Thank you, this looks great! I wonder whether we could perhaps use another dummy repo for testing release uploads to avoid polluting the releases here? Not sure whether we'd need a separate bot account for that

[sandyspiers]

I guess bot's could work, but I'm afraid I have no experience besides very basic github workflows.

As an alternative, I have updated the tests so that they run gh release delete (tag) -y --cleanup-tag, which effectively removes the release of tag test. So assuming you never want to publish a release with test (since it would be deleted on CI), this should work okay.

I ran it locally and confirmed it will create & delete the release.

If you're okay with this, we can add back GH_TOKEN, otherwise I may need some advice on the bot account solution :)

[simeonschaub]

It doesn't look like the previous job worked, yet it still passed: https://github.com/JuliaPackaging/ArtifactUtils.jl/actions/runs/23304720443/job/67775420671?pr=28#step:5:188 I think the test is insufficient, I suspect the artifact was already created locally, so Pkg just used that instead of downloading it from the release.

[sandyspiers]

Oops, my bad. I forgot to make sure that tests 'fail' if the try/catch fails on gh operations. This is fixed now, and the issue with gh is this:

86
HTTP 403: Resource not accessible by integration (https://api.github.com/repos/JuliaPackaging/ArtifactUtils.jl/releases)

I suspect this has got to do with the fact we cant create releases on pull requests, although I can't find anything to back that up.

[simeonschaub]

That's my suspicion as well, so we might need to go with the bot account approach after all. Unfortunately, I don't have much experience with that either. There is some documentation here, but not sure how applicable that is

[giordano]

What's the problem specifically? If it's a token limitation in the CI job, later today I can create an app which would allow overcoming that

[simeonschaub]

That might be a solution too. I haven't really looked that much into apps before, is that something we can trigger using just our regular CI job or does it require some extra setup?

[giordano]

Yes, during the CI job the app would create a powerful enough token

[simeonschaub]

That sounds like it could work! If you are able to set that up, I think that might be the solution

[giordano]

Uhm, thinking about it, activating the app would require reading secrets, which can't be done for PRs from forks, so that wouldn't be much helpful here 😞

[simeonschaub]

Going through https://discourse.julialang.org/t/reviewing-github-actions-workflows-tokens/, I feel less and less comfortable giving every PR here permission to mess with releases, so I think we need to come up with a better, more contained approach. (Ironically, the pull_request_target: trigger mentioned there might be exactly what we have been looking for.)

One other approach I have been thinking about is whether we can somehow mock either the gh command or the GitHub rest API for testing, but maybe the bot approach on a separate repo is easier?

[giordano]

(Ironically, the pull_request_target: trigger mentioned there might be exactly what we have been looking for.)

Yes, I ignored it as a possible option in my message above because pull_request_target is a security threat in the waiting.

[sandyspiers]

I have had a go at writing a fake gh, just for purpose of testing and it seems to work. Its just a script in test/, that when called like julia fake_gh.jl ARGS it accepts arguments like repo view, release create and release upload. In practice, all this does is create and move around some local files, so it never hits github. The minor downside is that now the gh binary path is passed around the code base as a keyword argument, allowing us to inject our fake gh.

The CI now passes, and the core functionality has stayed the same.

This did make me wonder whether we should add a dry-run feature? So users can check everything is going to run fine before publishing a release.