Preparing Infrastructure and Installing Clusters with kubeadm

Introduction

This lesson belongs to the Cluster Architecture, Installation & Configuration domain of the CKA curriculum (Kubernetes v1.35), which carries 25% of the exam. Together with troubleshooting, it is where most of the practical, command-line work of the exam is concentrated. The CKA is a performance-based exam: you are given a terminal, one or more live clusters, and a list of tasks. You will rarely be asked to bootstrap a cluster from nothing, but you are expected to understand how a kubeadm cluster is built, because every later task (upgrading, backing up etcd, repairing a broken control plane) depends on that understanding.

By the end of this lesson you will be able to prepare a Linux node for Kubernetes, bootstrap a control plane with kubeadm init, join worker nodes, explain what each control plane component does and where its manifest lives, and work confidently with kubeconfig files and contexts.

Learning objectives

• List the prerequisites a node must meet before kubeadm is run: container runtime, swap, kernel modules, sysctl settings, open ports, unique identity.
• Run kubeadm init with the options that matter (--pod-network-cidr, --control-plane-endpoint, --apiserver-advertise-address, a configuration file) and interpret its output.
• Join worker and control plane nodes, regenerate expired tokens and discover the CA certificate hash.
• Describe the role of kube-apiserver, etcd, kube-scheduler, kube-controller-manager, kubelet, kube-proxy and the container runtime, and find the static Pod manifests that run the control plane.
• Read and edit kubeconfig files, switch contexts and namespaces, and create a kubeconfig for another user or cluster.

The big picture: what kubeadm does and does not do

kubeadm is the official tool for bootstrapping a conformant cluster on machines you already have. It is deliberately narrow in scope. It does:

• generate the cluster certificate authority (CA) and component certificates under /etc/kubernetes/pki;
• write kubeconfig files for the admin, the kubelet, the controller manager and the scheduler under /etc/kubernetes;
• write static Pod manifests for kube-apiserver, kube-controller-manager, kube-scheduler and (for a stacked topology) etcd under /etc/kubernetes/manifests;
• start the kubelet, wait for the control plane Pods to become healthy, and install two add-ons: CoreDNS and kube-proxy;
• create a bootstrap token so other nodes can join.

It does not: install a container runtime, install the kubelet and kubeadm packages, provision machines, or install a Pod network add-on (CNI plugin). Until a CNI plugin is installed, nodes stay NotReady and CoreDNS Pods stay Pending. This is the single most common observation after a first kubeadm init, and a common exam trap.

Preparing the nodes

Every node (control plane and worker) must be prepared in the same way before kubeadm is used.

Hardware, OS and identity

The Kubernetes documentation asks for a compatible Linux host with at least 2 CPUs and 2 GB of RAM per machine (control plane nodes need the 2 CPUs; kubeadm's preflight checks fail otherwise), full network connectivity between all machines, and a unique hostname, MAC address and product_uuid on each node. Duplicate identities lead to confusing, intermittent join problems. You can check the values with:

hostnamectl --static
ip link
sudo cat /sys/class/dmi/id/product_uuid

Container runtime

Kubernetes talks to the runtime through the Container Runtime Interface (CRI). Since Kubernetes 1.24 the built-in Docker integration (dockershim) is gone, so the runtime must be a CRI-compatible one, most commonly containerd or CRI-O. The runtime exposes a socket; kubeadm detects it automatically when exactly one is present:

Two settings matter. First, the cgroup driver: on systemd-based distributions the kubelet and the runtime must both use the systemd cgroup driver. In containerd's /etc/containerd/config.toml that means SystemdCgroup = true in the runc options. A mismatch makes the kubelet or Pods restart in loops. Recent kubeadm versions default the kubelet to the systemd driver, so the runtime configuration is the part you usually fix. Second, if more than one runtime socket exists, pass --cri-socket to kubeadm init and kubeadm join.

Swap

By default the kubelet refuses to start when swap is enabled. The exam-safe preparation is:

sudo swapoff -a
# make it permanent: comment the swap line in /etc/fstab
sudo sed -i '/ swap / s/^/#/' /etc/fstab

Swap support exists in recent Kubernetes versions (the NodeSwap feature, with the kubelet setting failSwapOn: false and a swapBehavior), but it is an opt-in, advanced configuration. For the CKA, assume swap must be off unless a task explicitly says otherwise.

Kernel modules and sysctl settings

Pod networking through bridges and iptables requires two kernel modules and three sysctl values:

cat <<EOT | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOT
sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOT | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOT
sudo sysctl --system

overlay supports the overlay filesystem used by container image layers; br_netfilter makes bridged traffic visible to iptables so Services and network policies work; ip_forward lets the node route packets between Pods and the outside. A kubeadm preflight failure such as /proc/sys/net/ipv4/ip_forward contents are not set to 1 is solved by exactly this step.

Ports

Make sure the firewall and cloud security groups allow the required ports. These come straight from the official documentation and are worth memorising:

Installing the packages

Install kubeadm, kubelet and kubectl from the Kubernetes package repository for your minor version (pkgs.k8s.io), then hold the packages so that an unattended upgrade does not move the cluster to a different version:

sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl enable --now kubelet

After this, the kubelet restarts every few seconds in a crash loop. That is normal: it is waiting for kubeadm init or kubeadm join to give it configuration.

Worked example: diagnosing a failed preflight

Scenario. Contoso is building a three-node lab cluster on Ubuntu VMs. On the first VM, sudo kubeadm init stops with:

[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
  [ERROR Swap]: running with swap on is not supported. Please disable swap
  [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1

Reasoning, step by step.

1. Read the error names. Swap and FileContent--proc-sys-net-ipv4-ip_forward are two independent failures; kubeadm lists all fatal ones at once.
2. Fix swap: sudo swapoff -a, then comment the entry in /etc/fstab so the fix survives a reboot.
3. Fix forwarding: create /etc/sysctl.d/k8s.conf with net.ipv4.ip_forward = 1 and run sudo sysctl --system.
4. Re-run kubeadm init. Do not use --ignore-preflight-errors=all to silence the problem: it hides real misconfigurations and the node would fail later in a harder-to-diagnose way.

Why the alternatives are worse. Passing --ignore-preflight-errors=Swap would let the init proceed, but the kubelet would still refuse to run with swap on, so the control plane Pods would never start. Rebooting without editing /etc/fstab or the sysctl file would reintroduce both errors.

Bootstrapping the control plane with kubeadm init

The simplest form

On the first control plane node:

sudo kubeadm init \
  --pod-network-cidr=10.244.0.0/16 \
  --apiserver-advertise-address=192.168.10.11

• --pod-network-cidr reserves the range from which Pods get IP addresses. Some CNI plugins need a specific range; others read this value or have their own default. It must not overlap with node or Service networks.
• --apiserver-advertise-address is the IP the API server advertises. If the machine has several interfaces, set it explicitly.
• --service-cidr (default 10.96.0.0/12) is the virtual IP range for Services.
• --kubernetes-version pins the control plane version (defaults to the latest stable matching the kubeadm binary).
• --control-plane-endpoint sets a stable DNS name or load balancer address for the API server. Set it from day one if you may ever add control plane nodes; it cannot easily be added later and is required for high availability (covered in the next lesson).
• --upload-certs uploads control plane certificates (encrypted with a key kubeadm prints) so that additional control plane nodes can fetch them when they join.

A configuration file is better for repeatability

Flags and a file cannot be mixed freely; for anything non-trivial use a ClusterConfiguration and InitConfiguration:

apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.10.11
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: v1.35.0
controlPlaneEndpoint: k8s-api.contoso.example:6443
networking:
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12

Run it with sudo kubeadm init --config kubeadm-config.yaml. If you do not remember the field names during the exam, kubeadm config print init-defaults prints a complete template, and the official documentation is open in the allowed browser tab.

What happens during init (phases)

kubeadm init runs ordered phases, and you can run them individually with kubeadm init phase <name>:

1. preflight: checks swap, ports, runtime, CPU and memory.
2. certs: creates the CA and certificates in /etc/kubernetes/pki (API server, kubelet client, front-proxy, etcd, service account keys).
3. kubeconfig: writes admin.conf, kubelet.conf, controller-manager.conf, scheduler.conf.
4. etcd and control-plane: write static Pod manifests in /etc/kubernetes/manifests.
5. kubelet-start: writes the kubelet configuration and restarts it; the kubelet notices the manifests and starts the control plane Pods.
6. upload-config, mark-control-plane: store configuration in ConfigMaps in kube-system and taint the node node-role.kubernetes.io/control-plane:NoSchedule, so ordinary workloads are not scheduled there.
7. bootstrap-token and addon: create the join token and deploy CoreDNS and kube-proxy.

After init: kubeconfig and the CNI plugin

The output tells you to configure kubectl for a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Then install a CNI plugin by applying its manifest, for example kubectl apply -f <cni-manifest>.yaml, and watch the nodes become Ready. Confirm with:

kubectl get nodes
kubectl get pods -n kube-system

Joining nodes

At the end of kubeadm init, a ready-made command is printed:

sudo kubeadm join 192.168.10.11:6443 --token abcdef.0123456789abcdef \
  --discovery-token-ca-cert-hash sha256:3f4a...e91

Two values authenticate the exchange in both directions: the token proves to the control plane that the joiner is allowed in, and the CA certificate hash lets the joiner verify that it is talking to the genuine API server (preventing man-in-the-middle attacks). Tokens expire after 24 hours by default. If yours has expired:

kubeadm token list
kubeadm token create --print-join-command      # new token and the full command

If you only lost the hash, compute it from the CA:

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt \
  | openssl rsa -pubin -outform der 2>/dev/null \
  | openssl dgst -sha256 -hex | sed 's/^.* //'

To add a control plane node, add --control-plane and --certificate-key <key> (the key printed by kubeadm init --upload-certs, valid for two hours; regenerate with kubeadm init phase upload-certs --upload-certs). To remove a node properly: kubectl drain <node> --ignore-daemonsets --delete-emptydir-data, kubectl delete node <node>, then run sudo kubeadm reset on the node itself and clean up CNI configuration under /etc/cni/net.d.

Worked example: a join that fails with an x509 error

Scenario. The platform team of Fabrikam rebuilt the control plane node and re-ran kubeadm init. A worker that was prepared earlier now fails with x509: certificate signed by unknown authority when it runs the old kubeadm join command.

Reasoning.

1. The error is about trust, not connectivity: the TCP connection to port 6443 works.
2. The hash in the old command matches the previous CA. After re-initialising, the cluster has a brand new CA, so the old hash and old token are both invalid.
3. On the new control plane run kubeadm token create --print-join-command to get a matching token and hash.
4. On the worker, wipe the stale state first: sudo kubeadm reset -f, then run the new join command.
5. Verify with kubectl get nodes on the control plane.

Why not the alternatives. Using --discovery-token-unsafe-skip-ca-verification would make the error disappear but turns off the protection against impersonation; it is acceptable only in throw-away labs. Copying the old CA file to the new control plane is possible in advanced recovery scenarios, but is not a join fix.

The control plane components

Because the control plane runs as static Pods, you manage it by editing files, not by kubectl edit. The kubelet watches the directory defined by staticPodPath in /var/lib/kubelet/config.yaml (by default /etc/kubernetes/manifests). Saving a changed manifest makes the kubelet recreate the Pod; deleting the file stops it. Mirror Pods named kube-apiserver-<node> appear in the API for visibility, but deleting them with kubectl has no lasting effect. This matters on the exam: to change an API server flag, edit /etc/kubernetes/manifests/kube-apiserver.yaml and wait; to diagnose a control plane failure, inspect those same files.

Worked example: adding an API server flag

Task. Enable audit logging by passing --audit-log-path=/var/log/audit.log to the API server.

Steps.

1. sudo cp /etc/kubernetes/manifests/kube-apiserver.yaml /root/kube-apiserver.yaml.bak (always keep a copy outside the manifests directory; a backup left inside would be started as a second Pod).
2. Edit the manifest and add the flag under spec.containers[0].command; if the path is outside the container, also add a volumeMounts entry and matching hostPath volume.
3. Save. The kubelet recreates the Pod; the API is unavailable for a short time, so kubectl may fail for 30-60 seconds.
4. Check with sudo crictl ps | grep kube-apiserver. If it keeps restarting, read sudo crictl logs <id> and restore the backup.

Why this approach. Static Pods are the only supported way to change a kubeadm control plane component; a flag set through kubectl on a mirror Pod is ignored.

kubeconfig files and contexts

A kubeconfig is a YAML file with three lists and one pointer:

• clusters: the API server URL and the CA data (certificate-authority-data);
• users: credentials (client certificate and key, token, or an exec plugin);
• contexts: a named triple of cluster + user + namespace;
• current-context: the context kubectl uses.

kubectl locates configuration in this order: the --kubeconfig flag, then the KUBECONFIG environment variable (a colon-separated list that is merged), then ~/.kube/config.

kubectl config get-contexts
kubectl config current-context
kubectl config use-context prod-admin
kubectl config set-context --current --namespace=payments
kubectl config view --minify            # only the active context
kubectl config view --raw               # reveals embedded certificate data

On the exam, every task states the context to use (for example kubectl config use-context k8s-c2). Forgetting to switch is the most expensive mistake you can make: a perfect solution applied to the wrong cluster scores nothing. Copy the provided command at the beginning of each task.

You can build a kubeconfig by hand with kubectl config set-cluster, set-credentials and set-context:

kubectl config set-cluster staging --server=https://10.0.0.5:6443 \
  --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=true --kubeconfig=dev.conf
kubectl config set-credentials dev-user --client-certificate=dev.crt \
  --client-key=dev.key --embed-certs=true --kubeconfig=dev.conf
kubectl config set-context dev --cluster=staging --user=dev-user \
  --namespace=dev --kubeconfig=dev.conf

admin.conf contains a client certificate in the system:masters group, which bypasses RBAC. Treat it like a root password; give people their own identities and RBAC bindings instead (see the RBAC lesson). Recent kubeadm versions also generate a separate super-admin.conf and make admin.conf a less privileged kubeadm:cluster-admins identity, so check which file a task refers to.

Common pitfalls and exam tips

• Nodes NotReady right after init is expected until a CNI plugin is applied. Do not reinstall; apply the CNI.
• Swap and kernel settings are lost on reboot if you only run swapoff -a or modprobe without persisting them.
• cgroup driver mismatch between kubelet and runtime causes crash loops; check SystemdCgroup = true.
• Expired token: tokens last 24 hours; regenerate with kubeadm token create --print-join-command.
• Pod CIDR overlap with the node network silently breaks routing.
• Backups inside /etc/kubernetes/manifests become live Pods or crash the kubelet's parsing; keep them elsewhere.
• Wrong context. Always run kubectl config use-context as the first command of each task.
• Exam tip: use the allowed documentation at kubernetes.io; search for "kubeadm" pages, and learn where kubeadm join, kubeadm token and the ports table live. Set alias k=kubectl and export do="--dry-run=client -o yaml" to save time, and enable shell completion.
• Exam tip: the exam is time-boxed (120 minutes) with a 66% passing score. Skip a slow task and come back; weighted tasks are shown with their value.

Mini case study: bootstrapping for Tailwind Traders

Situation. Tailwind Traders wants a cluster in a datacentre: one control plane node cp1 (192.168.20.10) and two workers. They plan to add two more control plane nodes next quarter. The first attempt used kubeadm init --pod-network-cidr=192.168.0.0/16, but the datacentre network is 192.168.20.0/24.

Questions.

1. What is wrong with the Pod CIDR?
2. What must change now so that adding control plane nodes later is possible?
3. Nodes show NotReady and CoreDNS is Pending after init. Is the init broken?

Solution, commented.

1. The Pod range 192.168.0.0/16 contains the node subnet, so Pod and node addresses overlap and routing becomes ambiguous. Choose a range such as 10.244.0.0/16 that does not overlap with node or Service ranges.
2. Initialise with --control-plane-endpoint=k8s-api.tailwind.example:6443 (a DNS name or load balancer that can later point at three API servers) and --upload-certs. Because the endpoint is baked into certificates and kubeconfigs, it is far easier to set it now than to retrofit it.
3. No. kubeadm installs CoreDNS but not networking. Apply the chosen CNI plugin and the nodes become Ready, then CoreDNS starts.

Key terms

• kubeadm: official bootstrap tool for Kubernetes clusters.
• CRI: Container Runtime Interface, the API between kubelet and runtime.
• CNI: Container Network Interface, plugin API for Pod networking.
• Static Pod: Pod managed directly by the kubelet from a manifest file.
• Bootstrap token: short-lived credential used by kubeadm join.
• Discovery token CA cert hash: SHA-256 of the CA public key, used to verify the API server.
• kubeconfig / context: client configuration and a named cluster+user+namespace selection.

Summary checklist

• Swap off, overlay and br_netfilter loaded, ip_forward enabled, all persisted.
• containerd (or CRI-O) installed with the systemd cgroup driver.
• Ports 6443, 2379-2380, 10250, 10257, 10259 (control plane) and 10250, 30000-32767 (workers) allowed.
• kubeadm init run with a non-overlapping --pod-network-cidr and, for future HA, --control-plane-endpoint.
• kubeconfig copied to ~/.kube/config; CNI plugin applied.
• Workers joined; expired token replaced with kubeadm token create --print-join-command.
• Control plane changes made in /etc/kubernetes/manifests, backups stored elsewhere.
• The right context selected before every task.

Official sources

• Installing kubeadm: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ (opens in a new tab)
• Creating a cluster with kubeadm: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/ (opens in a new tab)
• Ports and protocols: https://kubernetes.io/docs/reference/networking/ports-and-protocols/ (opens in a new tab)
• Organizing cluster access with kubeconfig: https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/ (opens in a new tab)
• CKA curriculum (v1.35): https://github.com/cncf/curriculum (opens in a new tab)
• Exam FAQ: https://docs.linuxfoundation.org/tc-docs/certification/faq-cka-ckad-cks (opens in a new tab)