CKA Helm and Kustomize: Charts, CRDs, and Kubernetes Operators

Introduction

CKA Helm and Kustomize tasks test repeatable installation, manifest customization, and Kubernetes extension management. This lesson completes the Cluster Architecture, Installation & Configuration domain (25% of the CKA). Three official curriculum items are covered here: use Helm and Kustomize to install cluster components, understand extension interfaces (CNI, CSI, CRI, etc.), and understand CRDs, and install and configure operators. In the exam these show up as tasks such as "install a supplied chart as release web in namespace demo with 3 replicas", "apply the Kustomize overlay in ~/app/overlays/prod", or "install this CRD and create a custom resource". The commands are short, but you must know the flags and be able to inspect what a tool will produce before applying it.

After this lesson you will be able to install, upgrade, roll back and remove Helm releases, build and apply Kustomize overlays, explain what the CNI, CSI and CRI interfaces do and where their pieces live on a node, define and use CustomResourceDefinitions, and install an operator.

Learning objectives

• Add Helm repositories, search charts, install with overridden values, upgrade, roll back and uninstall releases.
• Render and inspect manifests without installing (helm template, --dry-run, helm get).
• Write a kustomization.yaml with a base and overlays, patches, name prefixes, namespaces and generators.
• Distinguish the roles of CNI, CSI and CRI and identify them on a running node.
• Create a CustomResourceDefinition and a custom resource, and explain how a controller turns it into action.
• Install and configure an operator, and verify that it works.

Helm: the package manager for Kubernetes

Helm packages Kubernetes manifests as a chart: a directory of templates plus default values. Installing a chart creates a release, a named instance stored in the cluster (by default, as Secrets of type helm.sh/release.v1 in the release namespace; alternate storage backends exist). Each upgrade creates a new revision, which is what makes rollback possible.

Helm 4 was released in November 2025 (the first major release in six years). Day-to-day commands used in the exam, repo, search, install, upgrade, rollback, uninstall, template, get, list, are the same in Helm 3 and Helm 4. Helm 4 mostly changes internals (a redesigned plugin system, server-side apply support, better --wait behaviour with kstatus, reproducible chart archives) and renames a few flags, so when a command from an old tutorial fails, run helm <command> --help. The exam environment tells you which version is installed; you do not need to guess.

Everyday workflow

The web-app examples assume the exercise provides a reviewed local chart at ./charts/webapp with replicaCount, service.type, and image.tag values and accessible approved images. Values are chart-specific. Repository commands below use the maintained Jetstack repository as a separate discovery example; pin a compatible chart version when installing third-party software.

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm search repo cert-manager                      # charts in added repositories
helm search repo jetstack/cert-manager --versions | head
helm show values ./charts/webapp | less       # default values you can override
helm show chart ./charts/webapp               # metadata

helm install web ./charts/webapp -n demo --create-namespace \
  --set replicaCount=3 --set service.type=ClusterIP
helm list -n demo
helm status web -n demo

Overriding values:

For an initial install, chart defaults apply unless overridden; later values files and repeated set values take precedence. Upgrade reuse/reset options determine how earlier release values participate.

Upgrade, history, rollback, uninstall

helm upgrade web ./charts/webapp -n demo --set replicaCount=5      # note: --set values are NOT remembered unless you add --reuse-values
helm upgrade --install web ./charts/webapp -n demo -f values.yaml  # install if absent; reruns still create upgrade revisions
helm history web -n demo
helm rollback web 1 -n demo                                      # creates a NEW revision with revision 1's content
helm uninstall web -n demo                                       # removes the release and its resources

Important behaviours:

• When supplying new values to helm upgrade, do not assume previous overrides will be reused. If you only pass --set replicaCount=5, earlier custom values are reset to chart defaults unless you use --reuse-values or repeat them. The safe habit is to keep values in a file and pass -f every time.
• helm rollback does not rewind history: it creates a new revision whose content equals the target revision.
• helm uninstall --keep-history retains release records. Resources the chart did not create (for example PVCs from a StatefulSet's volume claim templates) usually remain after uninstall.
• Charts can also come from an OCI registry: helm install web oci://registry.example.com/charts/nginx --version 1.2.3. Local charts work too: helm install web ./mychart.

Inspecting before and after

helm template web ./charts/webapp -n demo --set replicaCount=3 | less   # render locally, no cluster changes
helm install web ./charts/webapp -n demo --dry-run=client               # render and client-side checks; not proof of server acceptance or readiness
helm get values web -n demo            # user-supplied values only
helm get values web -n demo --all      # computed values
helm get manifest web -n demo          # manifests as installed
helm lint ./mychart

When a task says "use the chart's default values except X", helm get values is how you prove that only X was overridden.

Worked example: install, break, roll back

Scenario. Fabrikam installs the supplied web-app chart as release web in demo. Revision 1 uses replicaCount=2; revision 2 introduces a nonexistent image tag. The chart uses image.tag for that value.

Reasoning.

1. Inspect revisions with helm history web -n demo. A release can be marked deployed even while its Pods fail if readiness was not awaited.
2. Check kubectl get pods -n demo, the failing Pod’s Events, and helm get values web -n demo. Confirm ImagePullBackOff is caused by the changed tag.
3. Run helm rollback web 1 -n demo --wait. This creates a new revision based on the earlier release configuration; it does not undo external database or application side effects.
4. Verify release history, workload readiness, and service behavior.

Why not edit only the Deployment? A later Helm upgrade can overwrite the edit. Uninstall/reinstall loses the normal upgrade history and can disrupt service; rollout undo changes a workload without reconciling Helm’s release record.

Kustomize: template-free customization

Kustomize applies overlays to plain YAML without templating. It is built into kubectl: kubectl kustomize <dir> renders, and kubectl apply -k <dir> applies. A directory is a Kustomize base or overlay when it contains a kustomization.yaml.

Typical layout:

app/
  base/
    deployment.yaml
    service.yaml
    kustomization.yaml
  overlays/
    dev/kustomization.yaml
    prod/
      kustomization.yaml
      replica-patch.yaml

base/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml

overlays/prod/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: shop-prod
namePrefix: prod-
labels:
- pairs:
    env: prod
  includeSelectors: false
resources:
- ../../base
images:
- name: shop
  newTag: "2.4.1"
patches:
- path: replica-patch.yaml
configMapGenerator:
- name: shop-config
  literals:
  - LOG_LEVEL=warn

replica-patch.yaml (a strategic-merge patch; only the fields that change plus the identifying kind and name):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: shop
spec:
  replicas: 6

Core features and what they do:

Commands to remember:

kubectl kustomize overlays/prod            # render to stdout, do not touch the cluster
kubectl apply -k overlays/prod             # apply
kubectl diff -k overlays/prod              # preview changes
kubectl delete -k overlays/prod            # delete everything the overlay produces
kubectl get -k overlays/prod               # read the live state of those objects

Helm or Kustomize?

They combine well: helm template ... > base/all.yaml, then overlay with Kustomize, or a compatible Kustomize renderer can use helmCharts with explicit --enable-helm and a Helm binary. Do not assume plain kubectl apply -k enables chart inflation. Rendering Helm output into YAML and applying it separately does not create Helm release tracking.

Worked example: promote an app from dev to prod

Scenario. The shop Deployment in base has 1 replica and image shop:2.3.0. Production needs namespace shop-prod, 6 replicas, image tag 2.4.1, and every resource labelled env=prod.

Reasoning.

1. Do not copy the base files; create overlays/prod/kustomization.yaml that lists ../../base under resources.
2. Use namespace: shop-prod and labels for the label.
3. Use images for the tag. It is simpler and less error-prone than a patch.
4. Use a small patch for replicas: 6.
5. Render with kubectl kustomize overlays/prod | grep -E "namespace|replicas|image:" to verify before applying. Then kubectl apply -k overlays/prod.

Why not the alternatives? Copying and editing the base duplicates files and drifts. Using sed on YAML is fragile. Applying the base and then running kubectl scale leaves the replica count out of Git.

Extension interfaces: CRI, CNI and CSI

Kubernetes core does not run containers, set up pod networks or provision disks itself. It calls standard interfaces implemented by plugins.

Related extension points: device plugins (GPUs), admission webhooks, the cloud controller manager, aggregated API servers, and CustomResourceDefinitions.

Practical notes for the exam:

• A node reporting "cni plugin not initialized" needs inspection of runtime logs, CNI configuration, binaries, and the network add-on. An empty configured CNI directory may indicate an uninstalled or failed node component; repair the selected add-on rather than blindly installing a second network plugin.
• crictl ps, crictl pods, crictl logs talk to the CRI directly and work even when the API server is down. Set the endpoint in /etc/crictl.yaml or pass --runtime-endpoint.
• CNI is chosen at installation time and is not part of Kubernetes itself. Features such as NetworkPolicy only work when the CNI plugin enforces them (a later lesson covers this).
• A CSI-backed StorageClass references a driver provisioner such as ebs.csi.aws.com. Other provisioners exist; a local-path provisioner is not automatically a CSI driver. For prerequisites and behavior, see Kubernetes volumes, StorageClasses, and CSI.

CustomResourceDefinitions

A CustomResourceDefinition (CRD) teaches the API server a new resource type. After you create it, kubectl can create, list and delete objects of that kind (custom resources, CRs) exactly like built-in ones, with RBAC, watches and validation. A CRD alone only stores data; a controller watching those objects is what acts on them.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: backups.stable.example.com      # must be <plural>.<group>
spec:
  group: stable.example.com
  scope: Namespaced                     # or Cluster
  names:
    plural: backups
    singular: backup
    kind: Backup
    shortNames: ["bk"]
  versions:
  - name: v1
    served: true
    storage: true                       # exactly one version is the storage version
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            required: ["schedule"]
            properties:
              schedule:
                type: string
              retentionDays:
                type: integer
                minimum: 1

A custom resource:

apiVersion: stable.example.com/v1
kind: Backup
metadata:
  name: nightly
  namespace: data
spec:
  schedule: "0 2 * * *"
  retentionDays: 14

Key facts:

• In apiextensions.k8s.io/v1 a structural schema is mandatory; the API server rejects objects that violate it (for example retentionDays: 0).
• The CRD name must equal <plural>.<group>.
• Commands: kubectl get crd, kubectl explain backup.spec, kubectl api-resources --api-group=stable.example.com, kubectl get backups -A.
• Deleting a CRD deletes every custom resource of that kind. Treat it like dropping a table.
• Many installs (Helm charts for cert-manager, Prometheus operator, Gateway API) ship CRDs; Helm’s special crds/ directory does not provide automatic CRD upgrade or deletion. Some charts manage CRDs through templates or separate tooling instead, so follow that chart’s documented lifecycle.

Network-plugin installation and service behavior are covered in Pod connectivity, Services, and CoreDNS.

Worked example: "the server doesn't have a resource type"

Scenario. You run kubectl apply -f certificate.yaml and receive no matches for kind "Certificate" in version "cert-manager.io/v1".

Reasoning. The API server does not know the kind, so the CRD is missing or not yet established. Check with kubectl get crd | grep cert-manager. If absent, the operator or its CRDs were not installed. Install them (Helm chart with --set crds.enabled=true for cert-manager, or the vendor's CRD manifest), wait for kubectl wait --for=condition=Established crd/certificates.cert-manager.io, then re-apply. If the CRD exists but with a different version, fix apiVersion using kubectl explain certificate or kubectl get crd certificates.cert-manager.io -o jsonpath='{.spec.versions[*].name}'.

Why not the alternatives? Changing kind to a built-in type will not work; retrying the apply is pointless until the CRD exists; restarting the API server does nothing because CRDs are dynamic.

Operators

An operator is a controller (often with its own CRDs) that encodes the operational knowledge for an application: install, configure, back up, upgrade and heal it. You declare the desired state in a custom resource, and the operator reconciles reality towards it. This is the same control loop used by Deployments, applied to your own resources. Examples: Prometheus Operator, cert-manager, CloudNativePG, Strimzi (Kafka), Argo CD.

Installing an operator is normally a three-part job:

1. CRDs (the new API).
2. The operator Deployment plus its ServiceAccount and RBAC (Role/ClusterRole and bindings).
3. A custom resource that requests an instance.

Common installation methods: a Helm chart, a bundle manifest (kubectl apply -f install.yaml or kubectl apply --server-side -f for large CRDs), or Operator Lifecycle Manager (OLM) subscriptions.

# Select this reviewed version only when compatible with the target cluster.
helm install cert-manager oci://quay.io/jetstack/charts/cert-manager \
  --version v1.21.3 -n cert-manager --create-namespace --set crds.enabled=true
kubectl get pods -n cert-manager
kubectl get crd | grep cert-manager.io
kubectl wait --for=condition=Established crd/issuers.cert-manager.io --timeout=60s
# Wait for operator/webhook readiness using its installation verification steps.
kubectl apply -f issuer.yaml           # provided manifest in existing namespace demo
kubectl describe issuer selfsigned -n demo   # status/conditions are written by the operator

Verification pattern for any operator: Pods running, CRDs present and Established, a sample CR created, status or Events of the CR show progress, and the operator logs (kubectl logs deploy/<operator> -n <ns>) show reconciliation.

Worked example: install an operator-managed database

Scenario. Contoso wants a three-instance PostgreSQL cluster using an operator. The chart is cnpg/cloudnative-pg, which creates a CRD clusters.postgresql.cnpg.io.

Reasoning.

1. Add the documented repository (helm repo add cnpg https://cloudnative-pg.io/charts, then helm repo update), select a compatible chart version, and install the operator in its own namespace with Helm (helm install cnpg cnpg/cloudnative-pg -n cnpg-system --create-namespace).
2. Wait for the operator: kubectl rollout status deploy/cnpg-cloudnative-pg -n cnpg-system.
3. Confirm the CRD: kubectl get crd clusters.postgresql.cnpg.io.
4. Create the custom resource in the application namespace with spec.instances: 3, including its required storage configuration and existing namespace, then watch kubectl get clusters.postgresql.cnpg.io -n data and the Pods the operator creates.
5. Scaling later means editing spec.instances, not touching the Pods.

Why not the alternatives? Hand-writing a StatefulSet works but you then own failover, backups and upgrades; the operator encodes that logic. Applying the custom resource first fails because the kind is unknown.

Operators need explicit Kubernetes RBAC and ServiceAccount permissions. Inspect cluster-wide access before installation and verify controller readiness, not only the presence of the custom resource.

Common pitfalls and exam tips

• Forgetting the namespace. helm install without -n goes to the current context namespace; helm list also shows only that namespace unless you use -A.
• --set loss on upgrade. Re-supply all values (or use -f and --reuse-values knowingly).
• Rollback confusion. The rollback target is a revision number from helm history, not a chart version.
• Kustomize path. kubectl apply -k takes a directory; -f takes a file. Applying a base directly with -f ignores the overlay.
• Render first. helm template and kubectl kustomize show what will be created; use them to answer "which image will run?" questions without touching the cluster.
• Label selectors. Changing a Deployment selector via commonLabels on an existing object fails because spec.selector is immutable.
• CRD schema errors. In v1, a missing openAPIV3Schema or served/storage flags makes the CRD invalid.
• CRD deletion removes all instances. Never delete one casually.
• Exam tip: helm and kubectl kustomize are installed in the exam environment; kubectl explain and helm show values are your offline reference. The allowed documentation includes kubernetes.io, the Helm docs and the Kustomize docs listed in the candidate handbook, so search there rather than memorizing every flag.

Mini case study: platform bootstrap at Adventure Works

Adventure Works gives you an empty kubeadm cluster with a CNI already running. The tasks:

1. Install the supplied local web-app chart as release edge in namespace edge-apps, with 2 application replicas. Ingress NGINX maintenance ended in March 2026; it is not the default choice for a new installation in this lesson.
2. Deploy the inventory app to staging and production from a shared base, with production at 4 replicas in namespace inv-prod.
3. The operator team ships a CRD widgets.acme.io. A colleague gets "no matches for kind Widget". Explain and fix.

Solution.

1. With the reviewed chart available at ./charts/webapp:

helm install edge ./charts/webapp -n edge-apps --create-namespace --set replicaCount=2 --wait
helm list -n edge-apps
kubectl get deploy -n edge-apps
helm get values edge -n edge-apps

Check that the intended Deployment has two Ready replicas and that the values match the exercise. Chart rendering is separate from image availability and runtime readiness.

2. Create base/ with the Deployment and Service, overlays/staging and overlays/prod. The prod kustomization sets namespace: inv-prod, and a replicas patch (or the replicas field of the Kustomization) sets 4. Render with kubectl kustomize overlays/prod, check the namespace and replicas, then kubectl apply -k overlays/prod (creating the namespace first with kubectl create ns inv-prod, since Kustomize's namespace field does not create it unless a Namespace resource is included).

3. The CRD is not installed in this cluster. kubectl get crd widgets.acme.io returns NotFound; apply the CRD manifest, wait for Established, then apply the Widget. If the CRD exists, compare the apiVersion with kubectl get crd widgets.acme.io -o jsonpath='{.spec.versions[*].name}'.

Commentary. All three tasks are won by verifying: counting replicas, rendering overlays and listing CRDs. The common root cause of failures is acting without checking namespace or API availability.

Key terms

• Chart / release / revision: Helm package / installed instance / numbered version of a release.
• Values: configuration that overrides chart defaults.
• Base / overlay: shared Kustomize manifests / environment-specific customization.
• Patch: a partial manifest or JSON patch applied by Kustomize.
• CRI / CNI / CSI: container runtime, network and storage interfaces.
• CRD / CR: definition of a new API type / an instance of it.
• Operator: controller that manages an application through custom resources.
• Reconciliation: loop that moves actual state towards desired state.

Summary checklist

• helm repo add, repo update, search repo, show values, install -n --create-namespace --set/-f.
• helm upgrade --install, history, rollback <revision>, uninstall; keep values in a file.
• helm template or --dry-run and helm get values to inspect.
• kubectl kustomize to render; kubectl apply -k <dir>; overlays reference bases through resources.
• CRI = kubelet to runtime, CNI = Pod networking in /etc/cni/net.d, CSI = storage drivers.
• CRD name is <plural>.<group>, v1 needs a schema; deleting a CRD deletes its objects.
• Operator install = CRDs + controller Deployment/RBAC + a custom resource; verify Pods, CRD, CR status and logs.

Official sources

• Helm documentation (using Helm, charts, commands): https://helm.sh/docs/ (opens in a new tab)
• Helm 4 release announcement: https://helm.sh/blog/helm-4-released (opens in a new tab)
• Declarative management with Kustomize: https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/ (opens in a new tab)
• Network plugins (CNI): https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/ (opens in a new tab)
• Container runtimes (CRI): https://kubernetes.io/docs/setup/production-environment/container-runtimes/ (opens in a new tab)
• Volumes and CSI: https://kubernetes.io/docs/concepts/storage/volumes/#csi (opens in a new tab)
• Custom Resources and CRDs: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/ (opens in a new tab)
• Operator pattern: https://kubernetes.io/docs/concepts/extend-kubernetes/operator/ (opens in a new tab)
• CKA curriculum: https://github.com/cncf/curriculum/tree/master/cka (opens in a new tab)
• Helm upgrade value reuse: https://helm.sh/docs/helm/helm_upgrade/ (opens in a new tab)
• cert-manager Helm installation: https://cert-manager.io/docs/installation/helm/ (opens in a new tab)
• CloudNativePG chart installation: https://cloudnative-pg.io/charts/ (opens in a new tab)
• Ingress NGINX retirement notice: https://kubernetes.io/blog/2025/11/11/ingress-nginx-retirement/ (opens in a new tab)