[BUG] caching causes action to hang #117
Description
Version
redhat-actions/openshift-tools-installer@v1
Describe the bug
When installing the oc tool, using the cache causes the action to take 3 minutes.
When we set skip_cache=true the action will take 5 seconds.
Steps to reproduce, workflow links, screenshots
To test, I created two actions and set the skip_cache flag to false and true
name: Lets speed up OC install
on:
workflow_dispatch:
jobs:
openshift-cd:
name: I only install OC
runs-on: ubuntu-22.04
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Install oc
uses: redhat-actions/openshift-tools-installer@v1
id: install-cli
with:
oc: 4
skip_cache: true
- name: Print installed OC info
run: |
echo "OC installation details:"
echo "${{ toJSON(steps.install-cli.outputs.installed) }}"
Running with skip_cache: false
Logs:
Run redhat-actions/openshift-tools-installer@v1
Installing oc matching version "4.18.17"
ℹ️ Tools will be installed from "mirror".
🔎 Searching for a version of oc satisfying the range "4.18.17" that was input as "4.18.17"
Download directory for oc is https://mirror.openshift.com/pub/openshift-v4/clients/ocp/
Max oc version satisfying 4.18.17 is 4.18.17
Current operating system is linux
Warning: Multiple files were found for oc that matched the current OS and architecture: openshift-client-linux-4.18.17.tar.gz, openshift-client-linux-amd64-rhel8-4.18.17.tar.gz, openshift-client-linux-amd64-rhel9-4.18.17.tar.gz, openshift-client-linux-arm64-4.18.17.tar.gz, openshift-client-linux-arm64-rhel8-4.18.17.tar.gz, openshift-client-linux-arm64-rhel9-4.18.17.tar.gz, openshift-client-linux-ppc64le-4.18.17.tar.gz, openshift-client-linux-ppc64le-rhel8-4.18.17.tar.gz, openshift-client-linux-ppc64le-rhel9-4.18.17.tar.gz, openshift-client-linux-s390x-rhel8-4.18.17.tar.gz, openshift-client-linux-s390x-rhel9-4.18.17.tar.gz. Selecting the first one.
Selecting openshift-client-linux-4.18.17.tar.gz
Using temporary directory for storage
📁 CLIs will be downloaded to /home/runner/work/_temp/openshift-bin
Added /home/runner/work/_temp/openshift-bin to PATH
Checking the cache for oc 4.18.17...
Current architecture is amd64
Warning: Failed to check cache for oc 4.18.17: Error: getCacheEntry failed: Request timeout: /ohs3j6NMCAK3t8wXRHu9kr[2](https://github.com/bcgov/ecc-test-iac/actions/runs/15473140512/job/43562365179#step:3:2)t0lHk4EwXuYxIpWP5bubFVVe0EL/_apis/artifactcache/cache?keys=oc_4.18.17_linux_amd64&version=e3[9](https://github.com/bcgov/ecc-test-iac/actions/runs/15473140512/job/43562365179#step:3:10)322eccce1c32bdc0745272ae86f4de6f05b6e8476db9aa8e3eacb226df9ba
⬇️ Downloading 69.95MB https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.18.17/openshift-client-linux-4.18.17.tar.gz ...
Downloaded openshift-client-linux-4.18.17.tar.gz in 1.1s
⬇️ Downloading hash file https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.18.17/sha256sum.txt
sha256 verification of openshift-client-linux-4.18.17.tar.gz succeeded.
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/184e462d-6004-4f3c-817e-350ca40198e7 -f /home/runner/work/_temp/0256cdbe90cb4197434ab5769609b0dc-openshift-client-linux-4.18.17.tar.gz
Removed /home/runner/work/_temp/0256cdbe90cb4197434ab5769609b0dc-openshift-client-linux-4.18.17.tar.gz
oc installed into /home/runner/work/_temp/openshift-bin/oc
/home/runner/work/_temp/openshift-bin/oc version --client
Client Version: 4.18.17
Kustomize Version: v5.4.2
💾 Saving oc 4.18.17 into the cache
Warning: Failed to cache oc 4.18.17: Error: reserveCache failed: Request timeout: /ohs3j6NMCAK3t8wXRHu9kr2t0lHk4EwXuYxIpWP5bubFVVe0EL/_apis/artifactcache/caches
✅ Successfully installed oc 4.18.[17](https://github.com/bcgov/ecc-test-iac/actions/runs/15473140512/job/43562365179#step:3:18).
✅ Successfully installed 1/1 client:
- oc 4.[18](https://github.com/bcgov/ecc-test-iac/actions/runs/15473140512/job/43562365179#step:3:19).17
Wrote out installed versions to /home/runner/work/_temp/openshift-bin/openshift-clients-installed.json
skip_cache: true
Logs
Run redhat-actions/openshift-tools-installer@v1
Installing oc matching version "4"
ℹ️ Tools will be installed from "mirror".
🔎 Searching for a version of oc satisfying the range ">=4.0.0 <5.0.0-0" that was input as "4"
Download directory for oc is https://mirror.openshift.com/pub/openshift-v4/clients/ocp/
Max oc version satisfying >=4.0.0 <5.0.0-0 is 4.18.1[7](https://github.com/bcgov/ecc-test-iac/actions/runs/15473206407/job/43562583521#step:3:8)
Current operating system is linux
Warning: Multiple files were found for oc that matched the current OS and architecture: openshift-client-linux-4.1[8](https://github.com/bcgov/ecc-test-iac/actions/runs/15473206407/job/43562583521#step:3:9).17.tar.gz, openshift-client-linux-amd64-rhel8-4.18.17.tar.gz, openshift-client-linux-amd64-rhel[9](https://github.com/bcgov/ecc-test-iac/actions/runs/15473206407/job/43562583521#step:3:10)-4.18.17.tar.gz, openshift-client-linux-arm64-4.18.17.tar.gz, openshift-client-linux-arm64-rhel8-4.18.17.tar.gz, openshift-client-linux-arm64-rhel9-4.18.17.tar.gz, openshift-client-linux-ppc64le-4.18.17.tar.gz, openshift-client-linux-ppc64le-rhel8-4.18.17.tar.gz, openshift-client-linux-ppc64le-rhel9-4.18.17.tar.gz, openshift-client-linux-s390x-rhel8-4.18.17.tar.gz, openshift-client-linux-s390x-rhel9-4.18.17.tar.gz. Selecting the first one.
Selecting openshift-client-linux-4.18.17.tar.gz
⏩ Input skip_cache is set; skipping cache.
⬇️ Downloading 69.95MB https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.18.17/openshift-client-linux-4.18.17.tar.gz ...
Downloaded openshift-client-linux-4.18.17.tar.gz in 0.9s
⬇️ Downloading hash file https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.18.17/sha256sum.txt
sha256 verification of openshift-client-linux-4.18.17.tar.gz succeeded.
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/7fad2327-5[11](https://github.com/bcgov/ecc-test-iac/actions/runs/15473206407/job/43562583521#step:3:12)b-4041-b396-3627a6b83808 -f /home/runner/work/_temp/f216acbc4dd0e8fb06c9071dd7b62b65-openshift-client-linux-4.18.17.tar.gz
Removed /home/runner/work/_temp/f216acbc4dd0e8fb06c9071dd7b62b65-openshift-client-linux-4.18.17.tar.gz
Current architecture is amd64
Using temporary directory for storage
📁 CLIs will be downloaded to /home/runner/work/_temp/openshift-bin
Added /home/runner/work/_temp/openshift-bin to PATH
oc installed into /home/runner/work/_temp/openshift-bin/oc
/home/runner/work/_temp/openshift-bin/oc version --client
Client Version: 4.18.17
Kustomize Version: v5.4.2
⏩ Input skip_cache is set; skipping cache.
✅ Successfully installed oc 4.18.17.
✅ Successfully installed 1/1 client:
- oc 4.18.17
Wrote out installed versions to /home/runner/work/_temp/openshift-bin/openshift-clients-installed.json
additional observations.
Seems like the time spike happened starting this week (week of June 2)
Last week - saw some caching errors, but it gave 422 html code response and still finished within 30 seconds. I put the logs of that action run below.
logs
Run redhat-actions/openshift-tools-installer@v1
Installing oc matching version "4"
ℹ️ Tools will be installed from "mirror".
🔎 Searching for a version of oc satisfying the range ">=4.0.0 <5.0.0-0" that was input as "4"
Download directory for oc is https://mirror.openshift.com/pub/openshift-v4/clients/ocp/
Max oc version satisfying >=4.0.0 <5.0.0-0 is 4.18.12
Current operating system is linux
Warning: Multiple files were found for oc that matched the current OS and architecture: openshift-client-linux-4.18.12.tar.gz, openshift-client-linux-amd64-rhel8-4.18.12.tar.gz, openshift-client-linux-amd64-rhel9-4.18.12.tar.gz, openshift-client-linux-arm64-4.18.12.tar.gz, openshift-client-linux-arm64-rhel8-4.18.12.tar.gz, openshift-client-linux-arm64-rhel9-4.18.12.tar.gz, openshift-client-linux-ppc64le-4.18.12.tar.gz, openshift-client-linux-ppc64le-rhel8-4.18.12.tar.gz, openshift-client-linux-ppc64le-rhel9-4.18.12.tar.gz, openshift-client-linux-s390x-rhel8-4.18.12.tar.gz, openshift-client-linux-s390x-rhel9-4.18.12.tar.gz. Selecting the first one.
Selecting openshift-client-linux-4.18.12.tar.gz
Using temporary directory for storage
📁 CLIs will be downloaded to /home/runner/work/_temp/openshift-bin
Added /home/runner/work/_temp/openshift-bin to PATH
Checking the cache for oc 4.18.12...
Current architecture is amd64
Warning: Failed to check cache for oc 4.18.12: Error: Cache service responded with 422
⬇️ Downloading 69.94MB https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.18.12/openshift-client-linux-4.18.12.tar.gz ...
Downloaded openshift-client-linux-4.18.12.tar.gz in 2.3s
⬇️ Downloading hash file https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.18.12/sha256sum.txt
sha256 verification of openshift-client-linux-4.18.12.tar.gz succeeded.
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/5f3ab2f2-0cc2-49dc-985e-29e87af64f67 -f /home/runner/work/_temp/af76[16](https://github.com/bcgov/ECC-ECER/actions/runs/14912213414/job/41889147169#step:3:17)9780d49da308e65efeb6fdab2c-openshift-client-linux-4.18.12.tar.gz
Removed /home/runner/work/_temp/af76169780d49da308e65efeb6fdab2c-openshift-client-linux-4.[18](https://github.com/bcgov/ECC-ECER/actions/runs/14912213414/job/41889147169#step:3:19).12.tar.gz
oc installed into /home/runner/work/_temp/openshift-bin/oc
/home/runner/work/_temp/openshift-bin/oc version --client
Client Version: 4.18.12
Kustomize Version: v5.4.2
💾 Saving oc 4.18.12 into the cache
Warning: Failed to cache oc 4.18.12: ReserveCacheError: Unable to reserve cache with key oc_4.18.12_linux_amd64, another job may be creating this cache.
✅ Successfully installed oc 4.18.12.
✅ Successfully installed 1/1 client:
- oc 4.18.12
Wrote out installed versions to /home/runner/work/_temp/openshift-bin/openshift-clients-installed.json
The last successful OC install and cache run happened on or about Apr 2 (very approximate guess since we don't always run this action).
The issue with the action taking up to 3 minutes to complete with caching was observed this week.