rke2spray and GitOps Operations Reference

Key facts

• rke2spray owns hosts, RKE2, cluster-foundation add-ons, and the Argo CD bootstrap
• The GitOps repository owns platform, data, observability, and storage resources after the root Application
• Cluster-specific RKE2, CNI, topology, and network values are declared under inventory/<cluster>/
• GitOps verifies 15 Argo CD Application resources including the root bootstrap, and 299Gi of application PVC requests
• The NFS endpoint, external L4 load balancer, and absence of external backups define the current availability and recovery boundaries

Scope and authority

• This page was verified on 2026-08-30 against the current main code in both repositories
  • jongminchung/rke2spray is the cluster source
  • The target repository's bootstrap/, apps/, and tests/ directories define the GitOps contract
  • When prose differs from executable code, inventory, Playbooks, manifests, and verification scripts are authoritative

Repository ownership

• Concurrent ownership of the same Kubernetes resource is not supported
  • An RKE2 manifest owner maps to /var/lib/rancher/rke2/server/manifests/<owner>
  • native-addons-handoff verifies owner, basename, and checksum during an ownership transfer
  • An ownership-only ledger entry transfers ownership without deleting the Kubernetes resource
  • A prune ledger entry is removable only with the exact confirmation value

rke2spray cluster declaration

Topology

Versions and networking

Storage and snapshots

• durable-rwo and shared-rwx express workload intent and currently share one NFS failure domain
• Retain and onDelete: retain restrict PV deletion and do not provide NFS data backups

RKE2 native add-on profile

rke2spray operator commands

• These targets are defined by the repository-root Makefile

• cluster-reset, cluster-recover, cluster-remove-node, add-on pruning, and failure probes can destroy or interrupt state
  • Each target requires a bounded target, confirmation value, or opt-in environment variable
  • Server removal requires prior removal from external LB backends on ports 80, 443, 6443, and 9345
  • Off-host snapshot transfer and retention are outside the rke2spray contract

GitOps handoff contract

Argo CD declaration

Bootstrap gates

• extra_playbooks/gitops_bootstrap.yml requires all of the following conditions

  • RKE2 API /readyz returns ok
  • Every RKE2 Node reports Ready
  • Cilium and CoreDNS rollouts are complete
  • Gateway API CRDs report Established and the Cilium GatewayClass reports Accepted
  • cert-manager and NFS CSI rollouts are complete
  • NFS migration-gate status, database status, StorageClass UID, and cluster UID match the live cluster

• Secrets move from Ansible Vault into Argo CD Secrets

  • Read-only SSH private key for the Git repository
  • GitHub known-hosts value
  • SOPS age private key
  • Neither repository stores plaintext private keys

GitOps applications

Application inventory

• Applications with pruning disabled do not use a finalizer
• Applications with pruning enabled use resources-finalizer.argocd.argoproj.io
• Every Application uses automated sync and selfHeal: true

Workload topology

• Application PVC requests total 299Gi and the deployment ceiling is 400Gi
• PostgreSQL, ClickHouse, Keeper, MongoDB, and SeaweedFS Master and Volume replicas spread across three distinct workers

Services and public routes

ClusterIP services

• Data services in observability, postgresql, and seaweedfs may not use NodePort or LoadBalancer

Gateway API routes

Verification contracts

rke2spray static verification

GitOps static verification

Live-cluster verification

• tests/integration/verify.sh defines the following completion state

  • All 15 Argo CD Applications report Synced and Healthy
  • Stateful and stateless Application prune and finalizer policies match
  • Three-replica workloads are Ready on three distinct workers
  • Services excluded from public exposure remain ClusterIP
  • No Pod is Pending and every Node root filesystem remains below 60% usage
  • durable-rwo uses nfs.csi.k8s.io, Immediate, Retain, and onDelete=retain
  • All target PVCs use durable-rwo and total 299Gi
  • Gateway HTTPRoute resources report Accepted=True and ResolvedRefs=True

• Verification that writes or deletes state requires an explicit opt-in variable

  • The NFS performance gate requires RUN_NFS_PERFORMANCE_GATE=true
  • The database gate requires RUN_NFS_DATABASE_GATE=true
  • Workload disruption checks require RUN_DISRUPTIVE_TESTS=true

Limits and recovery boundaries

Documentation update contract

• Update this page's updatedAt and verifiedAt when any of the following code changes
  • Version, topology, network, or storage inputs in inventory/<cluster>
  • Public operator targets or required variables in Makefile
  • Readiness gates in gitops_bootstrap.yml
  • Applications, waves, chart versions, or prune policies in bootstrap/templates/applications.yaml
  • Replicas, PVCs, Services, or public routes under apps/
  • Completion assertions in tests/integration/verify.sh