Skip to content

Commit 70fc6fe

Browse files
committed
bare-metal -> qemu images
Signed-off-by: bakhtin <a@bakhtin.net>
1 parent a5815d0 commit 70fc6fe

File tree

10 files changed

+69
-56
lines changed

10 files changed

+69
-56
lines changed

.github/workflows/release.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ jobs:
7878
- name: Generate SHA256 checksums
7979
run: |
8080
cd mkosi.output
81-
sha256sum buildernet-*.{efi,tar.gz,vhd} | tee buildernet-${GITHUB_REF_NAME#buildernet-v}-${GITHUB_SHA::8}.sha256
81+
sha256sum buildernet-*.{efi,tar.gz,vhd,qcow2} | tee buildernet-${GITHUB_REF_NAME#buildernet-v}-${GITHUB_SHA::8}.sha256
8282
8383
- name: Sign artifacts
8484
env:
@@ -94,6 +94,6 @@ jobs:
9494
- name: Upload to R2
9595
run: |
9696
rclone copy -P --retries 3 --retries-sleep 20s --error-on-no-transfer \
97-
--s3-upload-concurrency=8 --transfers=8 --include "buildernet-*.{efi,tar.gz,vhd,minisig,sha256}" \
97+
--s3-upload-concurrency=8 --transfers=8 --include "buildernet-*.{efi,tar.gz,vhd,qcow2,minisig,sha256}" \
9898
mkosi.output r2-flashbots-public-artifacts:flashbots-public-artifacts/buildernet-images/${GITHUB_REF_NAME#buildernet-}/
9999

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@ mkosi -I buildernet.conf
1313
# What is this?
1414
This repository contains source code and configuration files for building Linux VM images for BuilderNet using [`mkosi`](https://github.com/systemd/mkosi).
1515

16-
The configuration supports building multiple flavors of the image for different platforms: Azure, GCP, and bare metal (see `mkosi.images` directory).
16+
The configuration supports building multiple flavors of the image for different platforms: Azure, GCP, and QEMU (see `mkosi.images` directory).
1717

1818
The entrypoint for the build is the main configuration file `buildernet.conf`. It defines basic parameters shared between the images. Each individual image may extend and override the main configuration (see [`mkosi` manual](https://github.com/systemd/mkosi/blob/main/mkosi/resources/man/mkosi.1.md#building-multiple-images)).
1919

20-
Final images depend on the successful build of the intermediate images. All of Azure, GCP, and bare metal images depend on `buildernet` image which in turn depends on `base` image.
20+
Final images depend on the successful build of the intermediate images. All of Azure, GCP, and QEMU images depend on `buildernet` image which in turn depends on `base` image.
2121

2222
BuilderNet images use Debian Trixie as a base. The images have low footprint and designed to be run fully in memory. One can still attach a persistent disk if needed (expected to mount at `/var/lib/persistent`).
2323

@@ -28,7 +28,7 @@ We build a custom Linux kernel using Debian's source kernel package with a bunch
2828
## Base image
2929
Located in `mkosi.images/base` is the base intermediate image that is shared between all downstream images.
3030

31-
This image builds a custom Linux kernel with configuration and patches overrides. We build two flavors of the same kernel: for the cloud and bare-metal. It uses `linux-config` Debian package for the base kernel configuration: `config.amd64_none_cloud-amd64` for the cloud version and `config.amd64_none_amd64` for bare metal.
31+
This image builds a custom Linux kernel with configuration and patches overrides. The kernel uses `linux-config` Debian package for the base kernel configuration: (`config.amd64_none_cloud-amd64` configuration file)
3232

3333
We further tweak the base config by disabling unnecessary modules/drivers (see `kernel/configs`). All the configuration fragments are merged and applied on top the default configuration.
3434

@@ -41,8 +41,8 @@ If you want to produce a new fragment you can use the tools that come with the L
4141
./scripts/kconfig/merge_config.sh -O . <path to config.amd64_none_cloud-amd64> <path to flashbots-images/mkosi.images/base/kernel/configs/*>
4242
```
4343
This will produce a file `.config` that you can tweak using `menuconfig` (e.g., `make nconfig` for the ncurses menuconfig)
44-
5. After you're done with tweaking, save the config under the new name, e.g., `.config_new`
45-
6. Produce a fragment by diff'ing the configs
44+
1. After you're done with tweaking, save the config under the new name, e.g., `.config_new`
45+
2. Produce a fragment by diff'ing the configs
4646
```
4747
./scripts/diffconfig -m .config .config_new > <path to flashbots-images/mkosi.images/base/kernel/configs/new-fragment>
4848
```

buildernet.conf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[Config]
22
Dependencies=buildernet-azure
3-
buildernet-bare-metal
43
buildernet-gcp
4+
buildernet-qemu
55
MinimumVersion=26~devel
66

77
[Distribution]
Lines changed: 35 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,60 +1,51 @@
11
#!/bin/bash
22
set -eu -o pipefail
33

4-
# Define flavors to build
5-
FLAVORS=("cloud" "bare-metal")
4+
FLAVOR="cloud"
5+
KERNEL_CONFIG_FILE="config.amd64_none_cloud-amd64.xz"
6+
FLAVOR_BUILDDIR="$BUILDDIR/linux-kernel-$KERNEL_VERSION-$FLAVOR"
67

7-
for FLAVOR in "${FLAVORS[@]}"; do
8-
echo "Building kernel flavor: $FLAVOR"
8+
mkdir -p "$FLAVOR_BUILDDIR"
99

10-
if [[ "$FLAVOR" == "cloud" ]]; then
11-
KERNEL_CONFIG_FILE="config.amd64_none_cloud-amd64.xz"
12-
else
13-
KERNEL_CONFIG_FILE="config.amd64_none_amd64.xz"
14-
fi
10+
tar xaf "/usr/src/linux-source-${KERNEL_VERSION}.tar.xz" -C /tmp
11+
SOURCE_DIR="/tmp/linux-source-${KERNEL_VERSION}"
1512

16-
FLAVOR_BUILDDIR="$BUILDDIR/linux-kernel-$KERNEL_VERSION-$FLAVOR"
17-
mkdir -p "$FLAVOR_BUILDDIR"
13+
cd "$SOURCE_DIR"
1814

19-
tar xaf "/usr/src/linux-source-${KERNEL_VERSION}.tar.xz" -C /tmp
20-
SOURCE_DIR="/tmp/linux-source-${KERNEL_VERSION}"
21-
22-
cd "$SOURCE_DIR"
15+
# 1. Apply base/common patches
16+
# These are patches located in mkosi.images/base/kernel/patches/<version>/
17+
find "$SRCDIR/mkosi.images/base/kernel/patches/$KERNEL_VERSION" -maxdepth 1 -type f -name "*.patch" 2>/dev/null || true | sort | while read patch; do
18+
echo "Applying base patch: $patch"
19+
patch -p1 < "$patch"
20+
done
2321

24-
# 1. Apply base/common patches
25-
# These are patches located in mkosi.images/base/kernel/patches/<version>/
26-
find "$SRCDIR/mkosi.images/base/kernel/patches/$KERNEL_VERSION" -maxdepth 1 -type f -name "*.patch" 2>/dev/null || true | sort | while read patch; do
27-
echo "Applying base patch: $patch"
28-
patch -p1 < "$patch"
29-
done
22+
# 2. Apply flavor-specific patches from ALL images
23+
# Looks for patches in: mkosi.images/*/kernel/patches/<version>/<flavor>/
24+
find "$SRCDIR/mkosi.images" -path "*/kernel/patches/$KERNEL_VERSION/$FLAVOR/*.patch" 2>/dev/null || true | sort | while read patch; do
25+
echo "Applying $FLAVOR patch: $patch"
26+
patch -p1 < "$patch"
27+
done
3028

31-
# 2. Apply flavor-specific patches from ALL images
32-
# Looks for patches in: mkosi.images/*/kernel/patches/<version>/<flavor>/
33-
find "$SRCDIR/mkosi.images" -path "*/kernel/patches/$KERNEL_VERSION/$FLAVOR/*.patch" 2>/dev/null || true | sort | while read patch; do
34-
echo "Applying $FLAVOR patch: $patch"
35-
patch -p1 < "$patch"
36-
done
29+
echo "Using kernel config file: $KERNEL_CONFIG_FILE"
3730

38-
echo "Using kernel config file: $KERNEL_CONFIG_FILE"
31+
unxz "/usr/src/linux-config-${KERNEL_VERSION}/$KERNEL_CONFIG_FILE" -c > /tmp/kernel.config
32+
./scripts/kconfig/merge_config.sh -O "$FLAVOR_BUILDDIR" /tmp/kernel.config "$SRCDIR/mkosi.images/base/kernel/configs/"*
33+
rm /tmp/kernel.config
3934

40-
unxz "/usr/src/linux-config-${KERNEL_VERSION}/$KERNEL_CONFIG_FILE" -c > /tmp/kernel.config
41-
./scripts/kconfig/merge_config.sh -O "$FLAVOR_BUILDDIR" /tmp/kernel.config "$SRCDIR/mkosi.images/base/kernel/configs/"*
42-
rm /tmp/kernel.config
35+
export DEB_BUILD_PROFILES='nodoc noudeb pkg.linux.nosource pkg.linux.notools pkg.linux.nokerneldbg pkg.linux.nokerneldbginfo pkg.linux.nometa'
36+
export MAKEFLAGS=-j$(nproc)
37+
export KBUILD_BUILD_TIMESTAMP="Fri Jun 5 15:58:00 CEST 1971"
38+
export KBUILD_BUILD_HOST="mkosi"
39+
export KBUILD_BUILD_USER="mkosi"
40+
export LOCALVERSION="${LOCALVERSION:-}-mkosi-$FLAVOR"
41+
rm -f "$FLAVOR_BUILDDIR/.version"
4342

44-
export DEB_BUILD_PROFILES='nodoc noudeb pkg.linux.nosource pkg.linux.notools pkg.linux.nokerneldbg pkg.linux.nokerneldbginfo pkg.linux.nometa'
45-
export MAKEFLAGS=-j$(nproc)
46-
export KBUILD_BUILD_TIMESTAMP="Fri Jun 5 15:58:00 CEST 1971"
47-
export KBUILD_BUILD_HOST="mkosi"
48-
export KBUILD_BUILD_USER="mkosi"
49-
export LOCALVERSION="${LOCALVERSION:-}-mkosi-$FLAVOR"
50-
rm -f "$FLAVOR_BUILDDIR/.version"
43+
make O="$FLAVOR_BUILDDIR" olddefconfig
44+
make O="$FLAVOR_BUILDDIR" bindeb-pkg
5145

52-
make O="$FLAVOR_BUILDDIR" olddefconfig
53-
make O="$FLAVOR_BUILDDIR" bindeb-pkg
46+
find "$FLAVOR_BUILDDIR/.." -type f -name "linux-image-${KERNEL_VERSION}*.deb" -exec cp {} "$PACKAGEDIR/" \;
5447

55-
find "$FLAVOR_BUILDDIR/.." -type f -name "linux-image-${KERNEL_VERSION}*.deb" -exec cp {} "$PACKAGEDIR/" \;
48+
rm -rf "$FLAVOR_BUILDDIR/.."/*.buildinfo "$FLAVOR_BUILDDIR/.."/*.deb "$FLAVOR_BUILDDIR/.."/*.changes
49+
unset LOCALVERSION
5650

57-
rm -rf "$FLAVOR_BUILDDIR/.."/*.buildinfo "$FLAVOR_BUILDDIR/.."/*.deb "$FLAVOR_BUILDDIR/.."/*.changes
58-
unset LOCALVERSION
59-
done
6051
#KDEB_PKGVERSION=someversion

mkosi.images/buildernet-bare-metal/README

Lines changed: 0 additions & 1 deletion
This file was deleted.

mkosi.images/buildernet-bare-metal/TODO

Lines changed: 0 additions & 1 deletion
This file was deleted.

mkosi.images/buildernet-bare-metal/mkosi.conf renamed to mkosi.images/buildernet-qemu/mkosi.conf

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
Dependencies=buildernet
33

44
[Output]
5-
ImageId=buildernet-bare-metal
5+
ImageId=buildernet-qemu
66
Format=uki
77

88
[Content]
@@ -12,4 +12,4 @@ KernelCommandLine=ro
1212
systemd.volatile=overlay
1313
console=ttyS0,115200n8
1414
systemd.firstboot=no
15-
VolatilePackages=linux-image-6.16.3-mkosi-bare-metal
15+
VolatilePackages=linux-image-6.16.3-mkosi-cloud
File renamed without changes.
File renamed without changes.
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
#!/bin/bash
2+
3+
set -eu -o pipefail
4+
5+
export SOURCE_DATE_EPOCH=0 # not propagated from the main config, needed for mkfs.vfat
6+
export SYSTEMD_REPART_MKFS_OPTIONS_VFAT="-i 12345678"
7+
mkdir -p ${OUTPUTDIR}/esp/EFI/BOOT
8+
cp ${OUTPUTDIR}/${IMAGE_ID}_${IMAGE_VERSION}.efi ${OUTPUTDIR}/esp/EFI/BOOT/BOOTX64.EFI
9+
rm -f ${OUTPUTDIR}/${IMAGE_ID}_${IMAGE_VERSION}.raw
10+
11+
systemd-repart --empty=create \
12+
--size=auto \
13+
--definitions=mkosi.images/buildernet/repart.d \
14+
--copy-source=${OUTPUTDIR} \
15+
--seed=630b5f72-a36a-4e83-b23d-6ef47c82fd9c \
16+
--dry-run=no \
17+
${OUTPUTDIR}/${IMAGE_ID}_${IMAGE_VERSION}.raw
18+
sgdisk --disk-guid "12345678-1234-5678-1234-567812345678" ${OUTPUTDIR}/${IMAGE_ID}_${IMAGE_VERSION}.raw
19+
20+
rm -rf ${OUTPUTDIR}/esp
21+
22+
cd ${OUTPUTDIR}
23+
qemu-img convert -f raw -O qcow2 ${OUTPUTDIR}/${IMAGE_ID}_${IMAGE_VERSION}.raw ${OUTPUTDIR}/${IMAGE_ID}_${IMAGE_VERSION}.qcow2
24+
rm -f ${OUTPUTDIR}/${IMAGE_ID}_${IMAGE_VERSION}.raw

0 commit comments

Comments
 (0)