In my last blog post, I discussed the basic key elements of Kubernetes, which can be found here. In this blog, I’ll walk you through the process of creating a Kubernetes cluster on your own hardware using the below steps. The actions need to perform in master and all worker node which I mentioned inside the brackets.
Step 1: VM readiness and install kubernetes tools like kubelet, kubeadm and kubectl (for all node)
Step 2: Install container runtime (for all node)
Step 3: Initialised the master node (for master node)
Step 4: Install Network Plugin (for master node)
Step 5: Join worker node in the cluster (for all worker node)
Step 6: Cluster Verification (for master node)
Step 7: Deploy Application (for master node)
Lab Setup
We’ll setup a cluster with four node. One master node and three worker node. We’ll use Ubuntu 22.04 LTS server for this lab.
Step 1: VM readiness and install kubelet, kubeadm and kubectl (for all node)
To achieve consistent performance and effective resource management, Kubernetes needs that swap be turned off. If swap memory is detected on the host, Kubelet will not start. So we’ll have to disable swap. To disable swap memory, perform the following steps.
sudo swapoff -a sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
Kubernetes requires overlay and br_netfilter kernel modules for efficient networking and communication between containers. Enable both parameter in kernel.
sudo tee /etc/modules-load.d/containerd.conf <<EOF overlay br_netfilter EOF
You will see the output like below
overlay br_netfilter
enable the parameter
sudo modprobe overlay sudo modprobe br_netfilter
Run the below command to install required packages
sudo apt-get install -y apt-transport-https ca-certificates curl gpg net-tools
Add the kubernetes repository. we will use version 1.32
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
Run the below command to add the kubernetes apt repository
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
Finally install the kubelet, kubeadm and kubectl package.
sudo apt update sudo apt upgrade -y sudo apt install -y kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl
But what are these packages? according to the kubernetes documentation
kubeadm: the command to bootstrap the cluster.
kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers.
kubectl: the command line util to talk to your cluster.
you can find more here.
Step 2: Container runtime (for all node)
Kubernetes requires a container runtime to manage, create, and run containers. Some common container runtime environments that can be used with Kubernetes are CRI-O, Docker, and containerd. In this lab, we’ll use containerd. To install containerd, enter the following command.
sudo apt install -y containerd
In a Kubernetes environment, the cgroup driver controls how resource limits (CPU, memory, etc.) are enforced for containers. Kubernetes supports systemd and cgroupfs drivers, with systemd being preferred for better integration and stability on modern Linux systems. To find out which driver is used in the system type the below command.
ps -p 1 PID TTY TIME CMD 1 ? 00:00:04 systemd
as you can see the system is using systemd driver. Let set systemd as the driver in containerd configuration file. Create a directory called containerd in /etc and generate the configuration called config.toml.
sudo mkdir -p /etc/containerd/ containerd config default | sed 's/SystemdCgroup = false/SystemdCgroup = true/' | sudo tee /etc/containerd/config.toml
To check the configuration is using the systemd, type the below command. At the end of the command output you will see a line similar to below which indicate containerd is using the systemd driver.
cat /etc/containerd/config.toml | grep -i SystemdCgroup -B 50 ...... ...... ...... SystemdCgroup = true
To take the configuration in effect restart the containerd like below.
sudo systemctl restart containerd
additionally add the below karnal parameter to kubernetes.conf file
sudo tee /etc/sysctl.d/kubernetes.conf <<EOT net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOT
set the kernel parameter.
sudo sysctl --system
Step 3: Initialised the master node (for master node)
Its time to initialise the master node. Run the below command in the master node.
sudo kubeadm init --apiserver-advertise-address 192.168.8.1 --pod-network-cidr "10.235.0.0/16" --upload-certs
–apiserver-advertise-address 192.168.8.1 (address of the master node, in our case its 192.168.8.1)
–pod-network-cidr “10.235.0.0/16” (POD network, in our case its 10.135.0.0/16)
After some time you will see an output like below. Save the output start with kubeadm join 192.168.8.1:6443 ……… till end in a safe place. we’ll need that command while adding the worker node into the cluster.
...... ...... Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ...... ...... kubeadm join 192.168.8.1:6443 --token 4poi1n.xxxxxxxxxxxxxscthp8x \ --discovery-token-ca-cert-hash sha256:1ed25xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx5a5d42761897bf
There are two task to follow now.
One: create the .kube directory and copy the config file.
Two: join the worker nodes in the cluster using the command provided above,
Type the below command to complete the task.
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
after creating the directory lets type the command kubectl get nodes.
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master NotReady control-plane 5m36s v1.32.2
You will see the master node named k8s-master but STATUS is Not-Ready that because we haven’t install any network plugin yet. In the next stage we’ll install the network plugin.
Step 4: Install Network Plugin (for master node)
A Kubernetes network plugin enables communication between pods, services, and external networks by implementing the Container Network Interface (CNI). It ensures seamless networking, policy enforcement, and connectivity across cluster nodes. Popular plugins include Calico, Flannel, Cilium, and Weave Net. In this lab we will use Flannel.
Download the flannel configuration file.
wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
Open the kube-flannel.yml file and find the section called net-conf.json. Under that section we will edit the Network value and set it to 10.235.0.0/16. Save and exit from the file and run the below command.
kubectl apply -f kube-flannel.yml
You can check the pod running by flannel by using the below command:
kubectl get pod -n kube-flannel NAME READY STATUS RESTARTS AGE kube-flannel-ds-4744m 1/1 Running 0 64s
Lets run the kubectl get nodes command again and see the change.
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready control-plane 11m v1.32.2
After installing the network plugin the master node become in Ready status.
Step 5: Join worker node in the cluster (for all worker node)
Our cluster is ready, now we can start adding worker nodes in that cluster. Run the below command in each node other than the master node which we save earlier.
sudo kubeadm join 192.168.8.1:6443 --token 4poi1n.xxxxxxxxxxxxxscthp8x \ --discovery-token-ca-cert-hash sha256:1ed25xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx5a5d42761897bf
You will see a confirmation that the node has joined in the cluster
This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
Step 6: Cluster Verification (for master node)
To check the status of the cluster run the below command in the master node. If things goes well you will see all your nodes are joined in the cluster and are in the Ready status.
kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready control-plane 23m v1.32.2 k8s-worker1 Ready <none> 8m31s v1.32.2 k8s-worker2 Ready <none> 7m5s v1.32.2 k8s-worker3 Ready <none> 6m56s v1.32.2
Step 7: Deploy Application (for master node)
Our cluster is ready for deploying application. Lets run a simple pod with nginx image. Run the below command in master node.
kubectl run web --image=nginx
To check the pod status run the below command.
kubectl get pods NAME READY STATUS RESTARTS AGE web 1/1 Running 0 36s
As you can see the pod is Running. Lets do some more testing. This time lets run a deployment with 5 replicas. Run the below command to do so:
kubectl create deployment web --image=nginx --replicas=5
run the command kubectl get pods -o wide to check the deployment.
kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES web-65d846d465-88h6v 1/1 Running 0 54s 10.235.3.3 k8s-worker3 <none> <none> web-65d846d465-r6b6d 1/1 Running 0 54s 10.235.1.2 k8s-worker1 <none> <none> web-65d846d465-s9pss 1/1 Running 0 54s 10.235.3.2 k8s-worker3 <none> <none> web-65d846d465-wksxd 1/1 Running 0 54s 10.235.1.3 k8s-worker1 <none> <none> web-65d846d465-x2t8g 1/1 Running 0 54s 10.235.2.3 k8s-worker2 <none> <none>
As you can see all the pods are running and they are running in different worker nodes.
Deploying a Kubernetes cluster on-premises provides complete control over your infrastructure. This guide explains how to set up master and worker nodes, install Kubernetes components, configure networking, and deploy a test application.
Thanks for reading the post. If you enjoyed the post, please share it with your network and let me know your thoughts in the comments.
About the Author: Imtiaz is working in a financial organization in Bangladesh and having experience in system, network and security administration. Feel free to contact with him on LinkedIn or Twitter.