How to use the capsule project with the argocd gitops approach

[djkormo]

How to use the capsule project with the argocd gitops approach

Capsule was installed using helm chart with version 0.7.4

How to sync namespaces resources

1. Provide the Capsule Tenant YAML definitions

My capsule configuration

apiVersion: capsule.clastix.io/v1beta2
kind: CapsuleConfiguration
metadata:
  annotations: null
  labels:
    app.kubernetes.io/instance: capsule
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: capsule
    app.kubernetes.io/version: 0.7.4
    argocd.argoproj.io/instance: capsule
    helm.sh/chart: capsule-0.7.4
  name: default
spec:
  enableTLSReconciler: false
  forceTenantPrefix: false
  nodeMetadata:
    forbiddenAnnotations:
      denied: []
      deniedRegex: ''
    forbiddenLabels:
      denied: []
      deniedRegex: ''
  overrides:
    TLSSecretName: capsule-tls
    mutatingWebhookConfigurationName: capsule-mutating-webhook-configuration
    validatingWebhookConfigurationName: capsule-validating-webhook-configuration
  protectedNamespaceRegex: "^(kube-system|kube-public|kube-node-lease|kube-kube-.*|default|capsule-systems)$"
  userGroups:
    - projectcapsule.dev
    - system:serviceaccounts:argocd
    - system:admin
    - system:masters

My tenant configuration

apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  labels:
    argocd.argoproj.io/instance: capsule-systems
  name: destroyer
spec:
  cordoned: false
  forceTenantPrefix: false
  limitRanges:
    items:
      - limits:
          - default:
              cpu: 500m
              memory: 512Mi
            defaultRequest:
              cpu: 100m
              memory: 10Mi
            type: Container
  namespaceOptions:
    additionalMetadata:
      labels:
        application: destroyer
    quota: 3 # number of namespaces per tenant
  owners:
    - kind: User
      name: system:admin
    - kind: User
      name: my-user
    - kind: ServiceAccount
      name: system:serviceaccounts:argocd
  resourceQuotas:
    items:
      - hard:
          limits.cpu: '7'
          limits.memory: 16Gi
          requests.cpu: '7'
          requests.memory: 16Gi
      - hard:
          pods: '100'
    scope: Namespace

2. Provide all managed Kubernetes resources

My namespaces configuration

apiVersion: v1
kind: Namespace
metadata:
  name: destroyer-dev
  labels:
    capsule.clastix.io/tenant: destroyer
    purpose: development   
--- 
apiVersion: v1
kind: Namespace
metadata:
  name: destroyer-uat
  labels:
    capsule.clastix.io/tenant: destroyer
    purpose: testing   
--- 
apiVersion: v1
kind: Namespace
metadata:
  name: destroyer-prd
  labels:
    capsule.clastix.io/tenant: destroyer
    purpose: production   

My argocd application

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: capsule-systems
  namespace: argocd
spec:
  project: default
  source:
    repoURL: 'http://gogs-svc.default.svc.cluster.local:18080/gogs/argocd-apps.git' # my local git installed on kubernetes
    targetRevision: HEAD
    path: capsule-systems

  destination:
    server: https://kubernetes.default.svc
    namespace:  capsule-systems

  syncPolicy:
    syncOptions:
    - CreateNamespace=true
    - ApplyOutOfSyncOnly=true

How to reproduce

Argocd is used to synchronize tenant manifest and namespaces manifests

Expected behavior

After syncing resources (here namespace) in argocd they should be added to tenant with owner reference.

When I try to sync them (the namespace is not created yet) I see

admission webhook "owner.namespace.projectcapsule.dev" denied the request: Cannot assign the desired namespace to a non-owned Tenant

In the capsule controller logs I see

{"level":"Level(-5)","ts":"2025-03-09T14:20:32.095Z","logger":"admission","msg":"received request","object":{"name":"destroyer-uat","namespace":"destroyer-uat"},"namespace":"destroyer-uat","name":"destroyer-uat","resource":{"group":"","version":"v1","resource":"namespaces"},"user":"system:serviceaccount:argocd:argocd-application-controller","requestID":"eb29c740-7c3c-4e14-8787-aabe236a0398"}
{"level":"Level(-5)","ts":"2025-03-09T14:20:32.095Z","logger":"admission","msg":"wrote response","code":403,"reason":"Forbidden","message":"Cannot assign the desired namespace to a non-owned Tenant","requestID":"eb29c740-7c3c-4e14-8787-aabe236a0398","allowed":false}
{"level":"debug","ts":"2025-03-09T14:20:32.095Z","logger":"events","msg":"Namespace destroyer-uat cannot be assigned to the current Tenant","type":"Warning","object":{"kind":"Tenant","name":"destroyer","uid":"05a8ac6b-7556-4f55-b247-bbe3e89bf0bd","apiVersion":"capsule.clastix.io/v1beta2","resourceVersion":"21037528"},"reason":"NonOwnedTenant"}

When the namespace is created outside argocd and having owner reference added to a namespace I see

error when patching "/dev/shm/2783867173": admission webhook "owner.namespace.projectcapsule.dev" denied the request: Denied patch request for this namespace

In logs I see

{"level":"Level(-5)","ts":"2025-03-09T14:23:49.175Z","logger":"admission","msg":"received request","object":{"name":"destroyer-dev","namespace":"destroyer-dev"},"namespace":"destroyer-dev","name":"destroyer-dev","resource":{"group":"","version":"v1","resource":"namespaces"},"user":"system:serviceaccount:argocd:argocd-application-controller","requestID":"5261797d-9d90-4b9a-a946-3e6a61817a5c"}
{"level":"Level(-5)","ts":"2025-03-09T14:23:49.175Z","logger":"admission","msg":"wrote response","code":403,"reason":"Forbidden","message":"Denied patch request for this namespace","requestID":"5261797d-9d90-4b9a-a946-3e6a61817a5c","allowed":false}
{"level":"debug","ts":"2025-03-09T14:23:49.175Z","logger":"events","msg":"Namespace destroyer-dev can not be patched","type":"Warning","object":{"kind":"Namespace","name":"destroyer-dev","uid":"a30b4a8c-936f-472b-b999-9ea97508bbc5","apiVersion":"v1","resourceVersion":"21037533"},"reason":"OfflimitNamespace"}

Additional context

• Capsule version: ghcr.io/projectcapsule/capsule:v0.7.4
• Helm Chart version: 0.7.4
• Kubernetes version: 1.29.12

[prometherion]

In your Tenant owners there's an evident typo:

system:serviceaccounts:argocd

This is a group, not a specific user. If you want to use a Service Account, former must be system:serviceaccount:<namespace>:<name>, otherwise, that entry must be treated as a Group.

[djkormo]

After changing owner to

    - name: system:serviceaccount:argocd:argocd-application-controller 
      kind: ServiceAccount

Namespace is fully synced and owner reference is present

It works!. Thank you @prometherion

The next question is how to unlink namespace from tenant object

[viktoraws]

Hi @prometherion and @djkormo

I am experiencing same error "Cannot assign the desired namespace to a non-owned Tenant" in my test ArgoCD environment and not sure what I'm missing at this point. Could you please advise if possible?

1. I have successfully deployed capsule chart via ArgoCD. ArgoCD deployement here:
   https://github.com/viktoraws/app-argo/blob/master/capsule-app.yaml

2. Updated capsuleUserGroups configuration to support ArgoCD service accounts group here:
   https://github.com/viktoraws/app-argo/blob/master/helm/values.yaml#L132

3. Create another ArgoCD app to deploy my tenants-chart which creates Tenants and its namespaces:
   https://github.com/viktoraws/tenant/blob/master/tenant-app.yaml

4. Added system:serviceaccount:argocd:argocd-server to the tenants Owners here:
   https://github.com/viktoraws/tenant/blob/master/helm/templates/tenant.yaml#L8

But still getting above error with latest v0.7.4 Capsule image, K8s 1.31, ArgoCD v2.14.6.

ArgoCD view print screens:

Any help is appreciated!

[djkormo]

I was using this in tenant specification

    - name: system:serviceaccount:argocd:argocd-application-controller 
      kind: ServiceAccount

@viktoraws You are using this

    - name: system:serviceaccount:argocd:argocd-server
      kind: ServiceAccount

[viktoraws]

Yes, I tried both, with system:serviceaccount:argocd:argocd-application-controller and system:serviceaccount:argocd:argocd-server service accounts. Same error message unfortunately. Thanks for your reply @djkormo !

[viktoraws]

So, I just deleted everything and reinstalled from scratch with using system:serviceaccount:argocd:argocd-application-controller sa and it worked!! Looks like it had to re-create tenants objects instead of updating its ServiceAccount under Owners section. Thank you so much for your prompt response @djkormo and @prometherion !!