[RFE] Restraint should allow user to specify the action upon git fetch task failure

[jirka-h]

Hello,

we have migrated the Linux Kernel Install task to the "git fetch" flow. When testing, we have run into this problem:

1. When "git fetch" URL is inaccessible, for example, due to the service outage, the kernel_install task is marked as Aborted.
2. However, the Beaker job has continued, executing the follow-up tasks with the wrong kernel

In such a situation, we need the Beaker job to be aborted immediately. We are looking for a way to specify for the git fetch task an action when it fails to download the test.

Thanks a lot

Jirka

[jirka-h]

Hi Vector, Thanks for looking into it! The issue is that we have nowhere where to put this command. It's a chicken egg problem. When git fetch fails, there is no code to be executed. This has to be handled by the Restraint itself. Jirka

…

On Wed, Feb 22, 2023 at 4:57 AM Vector Li ***@***.***> wrote: Hi @jirka-h <https://github.com/jirka-h>, please have a try to use cki_abort_recipe <https://gitlab.com/redhat/centos-stream/tests/kernel/kernel-tests/-/blob/main/cki_lib/libcki.sh#L88> to abort a recipe. # From: https://gitlab.com/redhat/centos-stream/tests/kernel/kernel-tests/-/blob/main/cki_lib/libcki.sh#L65 65 # 66 # When a serious problem occurs and we cannot proceed any further, we abort 67 # this recipe with an error message. 68 # 69 # Arguments: 70 # $1 - the message to print in the log 71 # $2 - 'WARN' or 'FAIL' 72 # 73 function cki_abort_recipe() 74 { 75 typeset failure_message="$1" 76 typeset failure_type=${2:-"FAIL"} 77 78 echo "❌ ${failure_message}" 79 if [[ "$failure_type" == 'WARN' ]]; then 80 rstrnt-report-result "${RSTRNT_TASKNAME}" WARN 99 81 else 82 rstrnt-report-result "${RSTRNT_TASKNAME}" FAIL 1 83 fi 84 rstrnt-abort -t recipe 85 exit $CKI_STATUS_ABORTED 86 } Please be noted that *L84 and L85 are required when aborting a recipe*. For details, please refer to the comments of mine in kernel-tests/-/issues/1576 <https://gitlab.com/redhat/centos-stream/tests/kernel/kernel-tests/-/issues/1576> . — Reply to this email directly, view it on GitHub <#288 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2U5OKF7H2JXX32RVHA2C3WYWFC5ANCNFSM6AAAAAAVC4JHJY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[idorax]

Hi @jirka-h, you're welcome. I'll discuss with @jbastian about this RFE since it's really needed.

[wackrat]

The request here should be to abort the recipe in which the task fails, not the entire job (which might have other recipes which don't have the same problem; if they do, they can abort themselves).

[jirka-h]

Hi Eirik,

yes, you are right! We want to abort the recipe here. Thanks for pointing this out!

Jirka

[lulinqing]

+1 on adding this "kill-switch", even per recipe (if per task is too much to ask).
We are relying on fetch-url to perform some key/core tasks instead of legacy task rpm, so this is critical.

Thanks in advance!

[lulinqing]

+1 on adding this "kill-switch", even per recipe (if per task is too much to ask). We are relying on fetch-url to perform some key/core tasks instead of legacy task rpm, so this is critical.

Thanks in advance!

BTW it also makes sense to increase the number of fetch-url retries, as well as longer delay (incrementally) before each retry.
It's a common practice to mitigate temporary connection issues - with minimum change needed in restraint code.