In this post, I’ll build a Kubernetes cluster with one master and two nodes using CRI-O and Cilium. I have DNS in my environment but you can use the hosts files if needed.
The OS will be Rocky Linux 9 running on ESXi 8.x but this should work on any other bare metal server too. The config for the nodes is:

– Master: 2 CPUs, 4GB RAM
– Workers: 2 CPUs, 8GB RAM.

First, we’ll have to disable swap and put SELinux in permissive mode.
Do this on all nodes (master + workers).

sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

Now, we have to open some firewall ports for Kubernetes..
Do this on the master node only.

sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=2379-2380/tcp
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=10259/tcp
sudo firewall-cmd --permanent --add-port=10257/tcp
sudo firewall-cmd --reload

Do this on the workers only.

sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=30000-32767/tcp   
sudo firewall-cmd --reload

In addition, we’ll have to open the firewall ports for Cilium.
Do this on all nodes.

sudo firewall-cmd --permanent --add-port=4240/tcp 
sudo firewall-cmd --permanent --add-port=8472/udp 
sudo firewall-cmd --reload

Now, we have to make some changes that are also required. Some necessary modules and IPtables change.
Do this on all nodes.

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

We have to install CRI-O container runtime.
Do this on all nodes.

VERSION=1.22
sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/CentOS_8/devel:kubic:libcontainers:stable.repo
sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:${VERSION}.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:${VERSION}/CentOS_8/devel:kubic:libcontainers:stable:cri-o:${VERSION}.repo
sudo dnf -y install cri-o cri-tools
sudo systemctl enable --now crio
sudo systemctl status crio

Then, we have to install Kubernetes.
Do this on all nodes.

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF
sudo dnf install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet

On the master node only, initialize the cluster. Replace master.homelab.com with your master IP if you don’t use DNS.

sudo kubeadm init --control-plane-endpoint master.homelab.local:6443

You’ll get an output that says how to join the nodes, something like kubeadm join master… + a token
On the master node, add these lines.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Join the nodes to the cluster.
Do this on workers only.

sudo kubeadm join master.homelab.local:6443 --token t2yir0.7n49rn6r57msy4j4 \
        --discovery-token-ca-cert-hash sha256:563360fcf60c49be91cdcf6486a4954c579a80c54503127ee0682ab8f86ec840

Check the nodes.
Do this on the master node.

kubectl get nodes
NAME                   STATUS   ROLES           AGE    VERSION
master.homelab.local   Ready    control-plane   116s   v1.28.2
node1.homelab.local    Ready    <none>          89s    v1.28.2
node2.homelab.local    Ready    <none>          85s    v1.28.2

…then check all the pods.
Do this on the master node.

kubectl get pods --all-namespaces
NAMESPACE     NAME                                           READY   STATUS    RESTARTS   AGE
kube-system   coredns-5dd5756b68-5ljgg                       1/1     Running   0          2m
kube-system   coredns-5dd5756b68-jp8tq                       1/1     Running   0          2m
kube-system   etcd-master.homelab.local                      1/1     Running   0          2m6s
kube-system   kube-apiserver-master.homelab.local            1/1     Running   0          2m6s
kube-system   kube-controller-manager-master.homelab.local   1/1     Running   0          2m6s
kube-system   kube-proxy-2z64r                               1/1     Running   0          2m
kube-system   kube-proxy-mrwjv                               1/1     Running   0          96s
kube-system   kube-proxy-nj5q4                               1/1     Running   0          100s
kube-system   kube-scheduler-master.homelab.local            1/1     Running   0          2m6s

Install Cilium on the master node only.

CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
cilium install --version 1.14.2

Check the status.

cilium status --wait

You should see something like this.

    /¯¯\
 /¯¯\__/¯¯\    Cilium:             OK
 \__/¯¯\__/    Operator:           OK
 /¯¯\__/¯¯\    Envoy DaemonSet:    disabled (using embedded mode)
 \__/¯¯\__/    Hubble Relay:       disabled
    \__/       ClusterMesh:        disabled

DaemonSet              cilium             Desired: 3, Ready: 3/3, Available: 3/3
Deployment             cilium-operator    Desired: 1, Ready: 1/1, Available: 1/1
Containers:            cilium             Running: 3
                       cilium-operator    Running: 1
Cluster Pods:          2/2 managed by Cilium
Helm chart version:    1.14.2
Image versions         cilium             quay.io/cilium/cilium:v1.14.2@sha256:6263f3a3d5d63b267b538298dbeb5ae87da3efacf09a2c620446c873ba807d35: 3
                       cilium-operator    quay.io/cilium/operator-generic:v1.14.2@sha256:52f70250dea22e506959439a7c4ea31b10fe8375db62f5c27ab746e3a2af866d: 1

Then you can run the connectivity test. Some of the egress test will fail because we don’t have a public IP on the cluster.

cilium connectivity test

You should be able to run your deployments now. If you need to access the pods using URL use something like NodePort or port-forwarding. It’s very cumbersome and inefficient, but you can use a load balancer with MetalLB. Read this post to see how to do that. It’s separated from this post, because it’s a separate topic than just k8s, container runtime and CNI.

Rocky Linux was finally release today (5/1/2021) and I’ve decided to try it in my lab. I’ve downloaded the minimal ISO (Rocky-8.3-x86_64-minimal) and installed it under my vCenter version 6.x.
I chose CentOS 8 as a base OS and after I mounted the ISO, I got an error saying ERROR Verification failed: (0x1A) Security Violation. If this is the case, shut down the guest, go to Settings | VM Options and if you expand Boot Options, change the Firmware from EFI to BIOS. This should do the trick.

What you’ll see next is pretty much the same as any other CentOS 7/8 installation.

NOTE: The current version is not considered for production.
Do the same as you do with any other CentOS install. Partition the hard drive, change the root password, configure the network and add a user.

Once the server is up, you’ll see the login prompt.

Check the kernel version and the release.

uname -a
Linux rocky.andreev.local 4.18.0-240.22.1.el8.x86_64 #1 SMP Mon Apr 12 04:29:16 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
cat /etc/redhat-release
Rocky Linux release 8.3

By default, both the firewall and SELinux are enabled.

systemctl status firewalld
sestatus

You can use dnf, yum or rpm to manage the packages. If you need to change the network settings, use nmtui.
Let’s install WordPress now. As root or using sudo, install Apache, PHP and MariaDB.

dnf install httpd php php-mysqlnd mariadb-server wget php-json

Start the Apache and MariaDB and enable them on both to start on reboot.

systemctl enable httpd --now
systemctl enable mariadb --now

Open port 80 on the firewall for the Apache.

firewall-cmd --permanent --zone=public --add-service=http --permanent
firewall-cmd --reload

Do the initian MariaDB configuration and specify the new root password for the SQL server.

mysql_secure_installation

Log as the SQL root user to MariaDB.

mysql -u root -p

Create the database and the WordPress user.

CREATE DATABASE wordpress;
GRANT ALL ON wordpress.* TO 'wpadmin'@'localhost' IDENTIFIED BY 'supersecret';
FLUSH PRIVILEGES;

Get the latest WordPress release under /tmp, unzip it and copy it where it belongs (root Apache data directory).

cd /tmp
wget https://www.wordpress.org/latest.zip
unzip latest.zip
mv wordpress/* /var/www/html/
chown -R apache:apache /var/www/html

Change the SELinux context for the root Apache data directory.

chcon -t httpd_sys_rw_content_t /var/www/html -R

At this point, you can access your WordPress install at http://your_IP_or_hostname.