Setup On Premise Kubernetes Cluster

Posted on Edited on In

Overview

I'll setup a k8s cluster with 3 nodes (1 control, 2 workers).

  • k8s-control
  • k8s-worker01
  • k8s-worker02

Setup

1. Install Packages on all nodes

  • Create configuration file for containerd

    cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
    overlay
    br_netfilter
    EOF

  • Load modules

    sudo modprobe overlay
    sudo modprobe br_netfilter

  • Set system configuration for k8s networking

    cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
    net.bridge.bridge-nf-call-iptables = 1
    net.ipv4.ip_forward = 1
    net.bridge.bridge-nf-call-ip6tables = 1
    EOF

  • Applying new settings

    sudo sysctl --system

  • Install containerd

    sudo apt-get update && sudo apt-get install -y containerd

  • Create default configuration file for containerd

    sudo mkdir -p /etc/containerd

  • Generate default containerd configuration and save to the newly created default file

    sudo containerd config default | sudo tee /etc/containerd/config.toml

  • Restart containerd to ensure new configuration file usage

    sudo systemctl restart containerd

  • Verify that containerd is running

    sudo systemctl status containerd

  • Disable swap

    sudo swapoff -a

  • Install dependencies

    sudo apt-get update && sudo apt-get install -y apt-transport-https curl

  • Download and add GPG key

    curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

  • Add Kubernetes to repository list

    cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
    deb https://apt.kubernetes.io/ kubernetes-xenial main
    EOF

  • Update package listings

    sudo apt-get update

  • Install Kubernetes packages

    sudo apt-get install -y kubelet=1.24.0-00 kubeadm=1.24.0-00 kubectl=1.24.0-00

  • Turn off automatic updates

    sudo apt-mark hold kubelet kubeadm kubectl

2. Initialize the Cluster

In the k8s-control host:

  • Initialize the Kubernetes cluster

    sudo kubeadm init --pod-network-cidr 192.168.0.0/16 --kubernetes-version 1.24.0

  • Set kubectl access

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config

  • Test access to cluster

    kubectl get nodes

3. Install the Calico Network Add-On

In the k8s-control host:

  • Install Calico Networking 

    kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

  • Check status of the control plane node 

    kubectl get nodes

4. Join the Worker Nodes to the Cluster

In the k8s-control:

  • Print the join command 
    kubeadm token create --print-join-command

In the k8s-worker01 and k8s-worker02 host:

  • Use the join command printed above 
    sudo kubeadm join ...