Skip to content

Commit d817507

Browse files
wind57wind57
authored
fix github actions build failure (#1054)
* test * add timeout * trigger * add timeout to 60 min * test * test * test * fix what tests each instance is running * github fix for abstract tests * remove file * removed echo * test circleci * escape * single? * test * bigger timeout * add a bit more debugging * fix circleci * split in half * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * circleci should work now * circleci should work now * drop splitting as it seems un-needed * remove file when done * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * trigger * trigger * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * trigger * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * test * trigger * test * test * test * test * added README * refactor load images * enable reuse * increaase timeout * increaase timeout * test * test * test * test * no return * test * test * test * test * test * test * test * test * test * test * test * test * test * disable test * pom * enable test back
1 parent 3b9bde7 commit d817507

42 files changed

Lines changed: 496 additions & 411 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.circleci/config.yml‎

Lines changed: 70 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,11 @@ jobs:
4343
# needed for .withReuse(true) to work
4444
echo "testcontainers.reuse.enable=true" > ~/.testcontainers.properties
4545
- checkout
46+
- restore_cache:
47+
keys:
48+
- spring-cloud-kubernetes-{{ .Branch }}-{{ checksum "pom.xml" }}
49+
- spring-cloud-kubernetes-{{ .Branch }}
50+
- spring-cloud-kubernetes
4651
- attach_workspace:
4752
at: /tmp/docker
4853
- run:
@@ -60,19 +65,74 @@ jobs:
6065
###########################################################################################################################
6166
################################################# Build test support dependency ###########################################
6267
cd spring-cloud-kubernetes-test-support
63-
.././mvnw clean install
68+
.././mvnw clean install
6469
cd ..
6570
6671
###########################################################################################################################
6772
##################################################### Split and run tests #################################################
68-
CLASSNAMES=$(circleci tests glob "**/src/test/**/**.java" | grep -v 'Fabric8IstioIT' \
69-
| xargs grep -l '@Test' \
70-
| sed 's/.*src.test.java.//g' | sed 's@/@.@g' \
71-
| sed 's/.\{5\}$//' \
72-
| circleci tests split --split-by=timings)
73-
echo $CLASSNAMES
74-
TEST_ARG=$(echo $CLASSNAMES | sed 's/ /,/g')
75-
echo $TEST_ARG
73+
74+
# - find all tests
75+
# - exclude Fabric8IstionIT
76+
# - only take classes that have @Test inside them
77+
# - ignore the ones that have 'abstract class'. we do this because otherwise we would pass
78+
# to -DtestsToRun an abstract class, and it will not run anything.
79+
# - drop the "begining" xxx/src/test/java
80+
# - replace / with .
81+
# - drop last ".java"
82+
# - replace newline with space
83+
84+
PLAIN_TEST_CLASSNAMES=($(find . -name '*.java' \
85+
| grep 'src/test/java' \
86+
| grep -v 'Fabric8IstioIT' \
87+
| xargs grep -l '@Test' \
88+
| xargs grep -L 'abstract class' \
89+
| sed 's/.*src.test.java.//g' \
90+
| sed 's@/@.@g' \
91+
| sed 's/.\{5\}$//'))
92+
93+
# classes that have @Test and are abstract, for example: "LabeledSecretWithPrefixTests"
94+
# - exclude Fabric8IstionIT
95+
# - only take classes that have @Test inside them
96+
# - only take classes that are abstract
97+
# - drop everything up until the last "/"
98+
# - drop ".java"
99+
100+
ABSTRACT_TEST_CLASSNAMES_COMMAND="find . -name '*.java' \
101+
| grep 'src/test/java' \
102+
| grep -v 'Fabric8IstioIT' \
103+
| xargs grep -l '@Test' \
104+
| xargs grep -l 'abstract class' \
105+
| sed 's/.*\///g' \
106+
| sed 's/.java//g'"
107+
108+
# find classes that extend abstract test classes
109+
DERIVED_FROM_ABSTRACT_CLASSES_COMMAND="find . -name '*.java' \
110+
| grep 'src/test/java' \
111+
| grep -v 'Fabric8IstioIT' \
112+
| xargs grep -l 'extends replace_me ' \
113+
| sed 's/.*src.test.java.//g' \
114+
| sed 's@/@.@g' \
115+
| sed 's/.\{5\}$//'"
116+
117+
while read class_name; do
118+
replaced=$(echo ${DERIVED_FROM_ABSTRACT_CLASSES_COMMAND/replace_me/"$class_name"})
119+
result=($(eval $replaced))
120+
PLAIN_TEST_CLASSNAMES+=(${result[@]})
121+
done < <(eval $ABSTRACT_TEST_CLASSNAMES_COMMAND)
122+
123+
IFS=$'\n'
124+
SORTED_TEST_CLASSNAMES=( $(sort \<<< "${PLAIN_TEST_CLASSNAMES[*]}" | uniq -u) )
125+
unset IFS
126+
127+
printf "%s\n" "${SORTED_TEST_CLASSNAMES[@]}" > file.txt
128+
CLASSNAMES=( $(cat file.txt | circleci tests split --split-by=timings) )
129+
rm file.txt
130+
131+
TEST_ARG=$(echo ${CLASSNAMES[@]} | sed 's/ /,/g')
132+
echo '----------------------- tests ---------------------'
133+
echo ${TEST_ARG[@]}
134+
echo '---------------------------------------------------'
135+
76136
./mvnw -s .settings.xml -DfailIfNoTests=false -DtestsToRun=$TEST_ARG -e clean org.jacoco:jacoco-maven-plugin:prepare-agent install \
77137
-U -P sonar -nsu --batch-mode -Dmaven.test.redirectTestOutputToFile=true \
78138
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
@@ -110,7 +170,7 @@ jobs:
110170
- run:
111171
name: dependencies
112172
command: |
113-
./mvnw -s .settings.xml -U dependency:resolve-plugins dependency:go-offline -B -Dservice.occurence=${_SERVICE_OCCURENCE} || true
173+
./mvnw -s .settings.xml -U dependency:resolve-plugins dependency:go-offline -B || true
114174
- run:
115175
name: build
116176
command: |

‎.github/workflows/composites/build-controllers-project/action.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
name: build controllers project
2+
description: build controllers project
3+
runs:
4+
using: "composite"
5+
steps:
6+
- name: build controllers project
7+
shell: bash
8+
run: |
9+
cd spring-cloud-kubernetes-controllers
10+
.././mvnw -T 1C clean install
11+
cd ..

‎.github/workflows/composites/build-integration-tests-project/action.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
name: build integration tests project without tests
2+
description: build integration tests project without tests
3+
runs:
4+
using: "composite"
5+
steps:
6+
- name: build integration tests project without tests
7+
shell: bash
8+
run: |
9+
cd spring-cloud-kubernetes-integration-tests
10+
# build the images, but dont run the tests
11+
.././mvnw -T 1C clean install -DskipTests
12+
cd ..

‎.github/workflows/composites/cache/action.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
name: cache
2+
description: cache
3+
runs:
4+
using: "composite"
5+
steps:
6+
- uses: actions/cache@v2
7+
with:
8+
path: ~/.m2/repository
9+
key: ${{ runner.os }}-cache-${{ env.BRANCH_NAME }}-${{ hashFiles('**/pom.xml') }}
10+
restore-keys: |
11+
${{ runner.os }}-cache-${{ env.BRANCH_NAME }}-${{ hashFiles('**/pom.xml') }}

‎.github/workflows/composites/download-docker-images/action.yaml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# we are not going to use upload/download actions, since it is slower than cache usage
2+
# as key to the cache we are going to use DOCKER_IMAGES_KEY, which is the same as GITHUB_RUN_ID
3+
4+
name: download docker images
5+
description: download docker images
6+
runs:
7+
using: "composite"
8+
steps:
9+
- uses: actions/cache@v2
10+
with:
11+
path: /tmp/docker/images
12+
key: docker-images-cache-${{ env.DOCKER_IMAGES_KEY }}
13+
restore-keys: docker-images-cache-${{ env.DOCKER_IMAGES_KEY }}
14+
15+

‎.github/workflows/composites/echo-saved-images/action.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
name: echo saved images
2+
description: echo saved images
3+
runs:
4+
using: "composite"
5+
steps:
6+
- name: echo saved images
7+
shell: bash
8+
run: |
9+
VIEW=$(ls -l /tmp/docker/images)
10+
echo "${VIEW}"

‎.github/workflows/composites/env-variables/action.yaml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
name: sets environment variables
2+
description: sets environment variables
3+
runs:
4+
using: "composite"
5+
steps:
6+
- name: set env variables
7+
shell: bash
8+
run: |
9+
echo "BRANCH_NAME=$(echo $GITHUB_HEAD_REF)" >> $GITHUB_ENV
10+
echo "DOCKER_IMAGES_KEY=$(echo $GITHUB_RUN_ID)" >> $GITHUB_ENV

‎.github/workflows/composites/fabric8-istio-integration-test/action.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
name: run fabric8 istio integration test
2+
description: run fabric8 istio integration test
3+
runs:
4+
using: "composite"
5+
steps:
6+
- name: run fabric8 istio integration test
7+
shell: bash
8+
run: |
9+
docker load -i /tmp/docker/images/spring-cloud-kubernetes-fabric8-istio-it.tar
10+
cd spring-cloud-kubernetes-integration-tests/spring-cloud-kubernetes-fabric8-istio-it/
11+
../.././mvnw clean install -Dskip.build.image=true
12+
cd ../..

‎.github/workflows/composites/fabric8-istio/action.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# build a lightweight project so that dependencies for the
2+
# maven-surefire-plugin are downloaded and thus cached also
3+
4+
name: build fabric8 istio project
5+
description: build fabric8 istio projects
6+
runs:
7+
using: "composite"
8+
steps:
9+
- name: build fabric8 istio project
10+
shell: bash
11+
run: |
12+
cd spring-cloud-kubernetes-fabric8-istio
13+
.././mvnw -s ../.settings.xml clean install -U
14+
cd ..

‎.github/workflows/composites/load-docker-images/action.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: install docker images
2+
description: load docker images from tar
3+
runs:
4+
using: "composite"
5+
steps:
6+
- name: load docker images
7+
shell: bash
8+
run: |
9+
10+
# get the name of the images to load from children of spring-cloud-kubernetes-controllers
11+
cd spring-cloud-kubernetes-controllers
12+
while read controller_image; do
13+
docker load -i /tmp/docker/images/${controller_image}.tar
14+
done < <(mvn -Dexec.executable='echo' -Dexec.args='${project.artifactId}' exec:exec -q | grep -v 'spring-cloud-kubernetes-controllers')
15+
cd ..
16+
17+
# get the name of the images to load from children of spring-cloud-kubernetes-integration-tests
18+
cd spring-cloud-kubernetes-integration-tests
19+
while read integration_test_image; do
20+
docker load -i /tmp/docker/images/${integration_test_image}.tar
21+
done < <(mvn -Dexec.executable='echo' -Dexec.args='${project.artifactId}' exec:exec -q | grep -v 'spring-cloud-kubernetes-integration-tests')
22+
cd ..

0 commit comments

Comments
 (0)