Deploy Kubernetes Using Kubeadm – CentOS7

I’ve been wanting to have a playground to mess around with Kubernetes (k8s) deployments for a while and didn’t want to spend the money on a cloud solution like AWS Elastic Container Service for Kubernetes or Google Kubernetes Engine . While these hosted solutions provide additional features such as the ability to spin up a load balancer, they also cost money every hour they’re available and I’m planning on leaving my cluster running. Also, from a learning perspective, there is no greater way to learn the underpinnings of a solution than having to deploy and manage it on your own. Therefore, I set out to deploy k8s in my vSphere home lab on some CentOS 7 virtual machines using Kubeadm. I found several articles on how to do this but somehow I got off track a few times and thought another blog post with step by step instructions and screenshots would help others. Hopefully it helps you. Let’s begin.

Prerequisites

This post will walk through the deployment of Kubernetes version 1.13.2 (latest as of this posting) on three CentOS 7 virtual machines. One virtual machine will be the Kubernetes master server where the control plane components will be run and two additional nodes where the containers themselves will be scheduled. To begin the setup, deploy three CentOS 7 servers into your environment and be sure that the master node has 2 CPUs and all three servers have at least 2 GB of RAM on them. Remember the nodes will run your containers so the amount of RAM you need on those is dependent upon your workloads.

Also, I’m logging in as root on each of these nodes to perform the commands. Change this as you need if you need to secure your environment more than I have in my home lab.

Prepare Each of the Servers for K8s

On all three (or more if you chose to do more nodes) of the servers we’ll need to get the OS setup to be ready to handle our kubernetes deployment via kubeadm. Lets start with stopping and disabling firewalld by running the commands on each of the servers:

systemctl stop firewalld
systemctl disable firewalld

Next, let’s disable swap. Kubeadm will check to make sure that swap is disabled when we run it, so lets turn swap off and disable it for future reboots.

swapoff -a
sed -i.bak -r 's/(.+ swap .+)/#\1/' /etc/fstabCode language: JavaScript (javascript)

Now we’ll need to disable SELinux if we have that enabled on our servers. I’m leaving it on, but setting it to Permissive mode.

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/configCode language: JavaScript (javascript)

Next, we’ll add the kubernetes repository to yum so that we can use our package manager to install the latest version of kubernetes. To do this we’ll create a file in the /etc/yum.repos.d directory. The code below will do that for you.

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOFCode language: JavaScript (javascript)

Now that we’ve updated our repos, lets install some of the tools we’ll need on our servers including kubeadm, kubectl, kubelet, and docker.

yum install -y docker kubelet kubeadm kubectl --disableexcludes=kubernetes

After installing docker and our kubernetes tools, we’ll need to enable the services so that they persist across reboots, and start the services so we can use them right away.

systemctl enable docker && systemctl start docker
systemctl enable kubelet && systemctl start kubelet

Before we run our kubeadm setup we’ll need to enable iptables filtering so proxying works correctly. To do this run the following to ensure filtering is enabled and persists across reboots.

cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --systemCode language: JavaScript (javascript)

Setup Kubernetes Cluster on Master Node

We’re about ready to initialize our kubernetes cluster but I wanted to take a second to mention that we’ll be using Flannel as the network plugin to enable our pods to communicate with one another. You’re free to use other network plugins such as Calico or Cillium but this post focuses on the Flannel plugin specifically.

Let’s run kubeadm init on our master node with the –pod-network switch needed for Flannel to work properly.

kubeadm init --pod-network-cidr=10.244.0.0/16

When you run the init command several pre-flight checks will take place including making sure swap is off and iptables filtering is enabled.

After the initialization is complete you should have a working kubernetes master node setup. The last few lines output from the kubeadm init command are important to take note of since it shows the commands to run on the worker nodes to join the cluster.

In my case, it showed that I should use the following command to join this cluster which we’ll use in the following section.

kubeadm join 10.10.50.50:6443 --token zzpnsn.jaf69dl0fb0z8yop --discovery-token-ca-cert-hash sha256:8f98d7e7fc701b2d686da84d7708376f4b59d83c0173b68a459a86cbe4922562Code language: CSS (css)

Before we start working on the other nodes, lets make sure we can run the kubectl commands from the master node, in case we want to use it later on.

export KUBECONFIG=/etc/kubernetes/admin.confCode language: JavaScript (javascript)

After setting our KUBECONFIG variable, we can run kubectl get nodes just to verify that the cluster is serving requests to the API. You can see that I have 1 node of type master setup now.

Configure Member Nodes

Now that the master is setup, lets focus on joining our two member nodes to the cluster and setup the networking. To begin, lets join the nodes to the cluster using the command that was output from the kubeadm init command we ran on the master. This is the command that I used, but your cluster will have a different value. Please use your own and not the one shown below as an example.

kubeadm join 10.10.50.50:6443 --token zzpnsn.jaf69dl0fb0z8yop --discovery-token-ca-cert-hash sha256:8f98d7e7fc701b2d686da84d7708376f4b59d83c0173b68a459a86cbe4922562Code language: CSS (css)

Once you run the command on all of your nodes you should get a success message that lets you know that you’ve joined the cluster successfully.

At this point I’m going to copy the admin.conf file over to my worker nodes and set KUBECONFIG to use it for authentication so that I can run kubectl commands on the worker nodes as well. This is step is optional if you are going to run kubectl commands elsewhere, like from your desktop computer.

scp root@[MASTERNODEIP]:/etc/kubernetes/admin.conf /etc/kubernetes/admin.conf 

export KUBECONFIG=/etc/kubernetes/admin.confCode language: JavaScript (javascript)

Now if you run the kubectl get nodes command again, we should see that there are three nodes in our cluster.

If you’re closely paying attention you’ll notice that the nodes show a status of “NotReady” which is a concern. This is because we haven’t finished deploying our network components for the Flannel plugin. To do that we need to deploy the flannel containers in our cluster by running:

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.ymlCode language: JavaScript (javascript)

Give your cluster a second and re-run the kubectl get nodes command. You should see the nodes are all in a Ready status now.

Configure a Mac Client

You’ve got your cluster up and running and are ready to go now. The last step might be to setup your laptop to run the kubectl commands against your k8s cluster. To do this, be sure to install the kubectl binaries on your mac and then copy the admin.conf file to your laptop by running this in a terminal window. Be sure to replace the IP with the master node IP address.

scp root@[MasterNodeIP]:/etc/kubernetes/admin.conf ~/.kube/admin.conf
export KUBECONFIG=~/.kube/admin.confCode language: JavaScript (javascript)

Once done, you should be able to run kubectl get nodes from your laptop and start deploying your configurations.