feat: add a way to sync Kubernetes manifests in Omni by Unix4ever · Pull Request #2231 · siderolabs/omni

Unix4ever · 2026-01-30T16:01:37Z

Manifests support two modes:

FULL - Omni will keep the manifest in sync always.
ONE_TIME - Omni will apply the manifest only if it doesn't exist. If the manifest is removed by hand and then changed in Omni it will be applied too.

Manifests are applied using service side apply, Omni now has two inventories: omni-internal-inventory and omni-user-inventory:

User inventory will be used for user managed manifests.
Internal one will be used for the manifests which are created by Omni controllers (workloadproxy, advanced healtcheck service).

Manifests also support setting namespace to all namespaced resources. It might be useful for the huge manifest files which are supplied without the namespace (similar to kubectl apply -n namespace -f manifest.yaml).

Orzelius

Not as much a code review as just a bunch of questions from my part

client/api/omni/specs/omni.proto

client/pkg/omni/resources/omni/cluster_kubernetes_manifest_status.go

client/pkg/omni/resources/omni/kubernetes_manifest.go

internal/backend/runtime/omni/controllers/omni/kubernetes/cluster_manifest_status.go

client/api/omni/specs/omni.proto

client/pkg/omni/resources/omni/cluster_kubernetes_manifest_status.go

rothgar · 2026-02-04T17:37:56Z

How is someone intended to use this initially? Is this only part of cluster templates or can it be added to any cluster in Omni?

Unix4ever · 2026-02-05T11:38:31Z

How is someone intended to use this initially? Is this only part of cluster templates or can it be added to any cluster in Omni?

I'm going to make it part of the cluster templates in the next PR.
Also will move workloadproxy manifests to this system.

Unix4ever · 2026-02-06T13:50:11Z

Pretty much rewrote the controller. Made the status updates more frequent, fixed more issues there:

mappers for the LoadbalancerStatus and ClusterStatus were incorrect.
reduced the prune/rollout timeouts to make the controller retry and update the status.

The status now contains more phases as well as the information about the resource being created.

Example:

metadata:
    namespace: default
    type: ClusterKubernetesManifestsStatuses.omni.sidero.dev
    id: talos-default
    version: 34
    owner: ClusterKubernetesManifestsStatusController
    phase: running
    created: 2026-02-06T12:38:46Z
    updated: 2026-02-06T13:20:00Z
spec:
    manifests:
        default_nginx-deployment_apps_Deployment:
            phase: 3
            lasterror: ""
            mode: 1
            kind: Deployment
            name: nginx-deployment
            namespace: default
    errors: {}
    outofsync: 0
    total: 1

Manifests support two modes: - `FULL` - Omni will keep the manifest in sync always. - `ONE_TIME` - Omni will apply the manifest only if it doesn't exist. If the manifest is removed by hand and then changed in Omni it will be applied too. Manifests are applied using service side apply, Omni now has two inventories: `omni-internal-inventory` and `omni-user-inventory`: - User inventory will be used for user managed manifests. - Internal one will be used for the manifests which are created by Omni controllers (workloadproxy, advanced healtcheck service). Manifests also support setting namespace to all namespaced resources. It might be useful for the huge manifest files which are supplied without the namespace (similar to `kubectl apply -n namespace -f manifest.yaml`). Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>

utkuozdemir · 2026-02-09T10:28:30Z

client/api/omni/specs/kubernetes_manifest.go

+
+	compressedData := doCompress(data, config)
+
+	x.Data = ""


What if we compress them unconditionally? Our old compressing resources were done kinda that way for kind of backwards compatibility. Maybe we can just do this simpler.

It has more than that. It also doesn't compress if the manifest size is small. So I thought it kinda makes sense to use the same approach here.

utkuozdemir · 2026-02-09T10:31:22Z

client/api/omni/specs/omni.proto

+    string namespace = 6;
+  }
+
+  map<string, ManifestStatus> manifests = 1;


Curious, does this mean there is no order of apply between manifests? I guess it'd probably work anyway, our code would keep trying to apply them, but normally there can be an order in k8s manifests - an example is creating a CRD and a CR. If so, we could maybe turn these into slices instead?

This is the status resource.

While the manifests are ordered: separate KubernetesManifest resources are ordered by COSI using the ID, and it's possible to put many manifests to a single resource delimiting by ---, then it's a slice.

Unix4ever · 2026-02-09T12:59:45Z

client/pkg/omni/resources/omni/labels.go


+	// LabelSystemManifest marks the manifest as the system manifest, so it shouldn't be editable by the user.
+	// tsgen:LabelSystemManifest
+	LabelSystemManifest = SystemLabelPrefix + "system-manifest"


Show the system manifest flag inside the ClusterKubernetesManifestsStatus.

Unix4ever · 2026-02-09T13:06:28Z

internal/backend/runtime/omni/controllers/omni/kubernetes/cluster_manifest_status.go

+
+	if len(oneTimeManifests) > 0 {
+		if err = ctrl.syncOneTime(ctx, logger, oneTimeManifests, client, clusterKubernetesManifestsStatus); err != nil {
+			errs = errors.Join(errs, err)


wrap the error to add more context

Unix4ever · 2026-02-09T13:06:37Z

internal/backend/runtime/omni/controllers/omni/kubernetes/cluster_manifest_status.go

+
+	for inventory, objects := range manifestsByInventory {
+		if err = ctrl.sync(ctx, logger, objects, inventory, client, clusterKubernetesManifestsStatus); err != nil {
+			errs = errors.Join(errs, err)


wrap the error to add more context

Unix4ever · 2026-02-09T13:07:09Z

client/api/omni/specs/omni.proto

+    KubernetesManifestSpec.Mode mode = 3;
+    string kind = 4;
+    string name = 5;
+    string namespace = 6;


add last modified date.

Unix4ever · 2026-02-09T13:15:34Z

internal/backend/runtime/omni/controllers/omni/kubernetes/cluster_manifest_status.go

+	return apierrors.IsInvalid(err) || apierrors.IsBadRequest(err) || apierrors.IsForbidden(err) ||
+		apierrors.IsRequestEntityTooLargeError(err) || webhookError(err) || validationError(err) || strings.Contains(err.Error(), "reconcile failed")


Consider making it less restricted

Unix4ever · 2026-02-09T13:25:02Z

internal/backend/runtime/omni/controllers/omni/kubernetes/cluster_manifest_status.go

+
+			//nolint:exhaustive
+			switch result.Type {
+			case event.PruneType:


Try to use inventory to track prune/delete events.

Unix4ever · 2026-02-09T13:29:12Z

internal/backend/runtime/omni/controllers/omni/kubernetes/cluster_manifest_status.go

+			continue
+		}
+
+		_, err = dr.Create(ctx, manifest, v1.CreateOptions{FieldManager: constants.KubernetesFieldManagerName})


Suggested change

_, err = dr.Create(ctx, manifest, v1.CreateOptions{FieldManager: constants.KubernetesFieldManagerName})

_, err = dr.Apply(ctx, manifest, v1.CreateOptions{FieldManager: constants.KubernetesFieldManagerName})

Unix4ever deployed to Storybook preview January 30, 2026 16:01 — with GitHub Actions View deployment

Unix4ever force-pushed the manifests branch from 0e1a6b0 to 925d34e Compare January 30, 2026 18:19

Unix4ever deployed to Storybook preview January 30, 2026 18:19 — with GitHub Actions View deployment

Unix4ever force-pushed the manifests branch from 925d34e to 8ddc16f Compare January 30, 2026 18:52

Unix4ever deployed to Storybook preview January 30, 2026 18:52 — with GitHub Actions View deployment

Orzelius reviewed Feb 2, 2026

View reviewed changes

Unix4ever force-pushed the manifests branch from 8ddc16f to 56631f6 Compare February 3, 2026 11:41

Unix4ever deployed to Storybook preview February 3, 2026 11:41 — with GitHub Actions View deployment

Unix4ever force-pushed the manifests branch from 56631f6 to 094cdf0 Compare February 3, 2026 16:57

Unix4ever deployed to Storybook preview February 3, 2026 16:57 — with GitHub Actions View deployment

Unix4ever marked this pull request as ready for review February 3, 2026 16:57

talos-bot added the status/ok-to-test label Feb 3, 2026

talos-bot added this to Planning Feb 3, 2026

talos-bot moved this to In Review in Planning Feb 3, 2026

Unix4ever force-pushed the manifests branch from 094cdf0 to b316918 Compare February 3, 2026 17:03

Unix4ever deployed to Storybook preview February 3, 2026 17:04 — with GitHub Actions View deployment

Unix4ever force-pushed the manifests branch from b316918 to 2d1fe7a Compare February 3, 2026 17:25

Unix4ever deployed to Storybook preview February 3, 2026 17:25 — with GitHub Actions View deployment

oguzkilcan reviewed Feb 4, 2026

View reviewed changes

client/api/omni/specs/omni.proto Show resolved Hide resolved

oguzkilcan reviewed Feb 4, 2026

View reviewed changes

client/pkg/omni/resources/omni/cluster_kubernetes_manifest_status.go Outdated Show resolved Hide resolved

oguzkilcan approved these changes Feb 4, 2026

View reviewed changes

github-project-automation bot moved this from In Review to Approved in Planning Feb 4, 2026

Unix4ever force-pushed the manifests branch 2 times, most recently from 7a46b50 to d28cbd8 Compare February 6, 2026 13:47

Unix4ever deployed to Storybook preview February 6, 2026 13:47 — with GitHub Actions View deployment

Unix4ever force-pushed the manifests branch from d28cbd8 to dd977a8 Compare February 6, 2026 13:48

Unix4ever deployed to Storybook preview February 6, 2026 13:48 — with GitHub Actions View deployment

Unix4ever force-pushed the manifests branch from dd977a8 to 8f1e418 Compare February 6, 2026 14:47

Unix4ever deployed to Storybook preview February 6, 2026 14:47 — with GitHub Actions View deployment

utkuozdemir reviewed Feb 9, 2026

View reviewed changes

Unix4ever commented Feb 9, 2026

View reviewed changes

		return apierrors.IsInvalid(err) \|\| apierrors.IsBadRequest(err) \|\| apierrors.IsForbidden(err) \|\|
		apierrors.IsRequestEntityTooLargeError(err) \|\| webhookError(err) \|\| validationError(err) \|\| strings.Contains(err.Error(), "reconcile failed")

	_, err = dr.Create(ctx, manifest, v1.CreateOptions{FieldManager: constants.KubernetesFieldManagerName})
	_, err = dr.Apply(ctx, manifest, v1.CreateOptions{FieldManager: constants.KubernetesFieldManagerName})

Conversation

Unix4ever commented Jan 30, 2026

Uh oh!

Orzelius left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rothgar commented Feb 4, 2026

Uh oh!

Unix4ever commented Feb 5, 2026

Uh oh!

Unix4ever commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Unix4ever Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Unix4ever commented Feb 6, 2026 •

edited

Loading

Unix4ever Feb 9, 2026 •

edited

Loading