Location>code7788 >text

[kubernetes] Deploying kubernetes with kubeadm and containerd

Popularity：329 ℃/2024-08-30 20:18:49

preamble

Due to host kernel version limitations and the need for vertical scaling features, the k8s version installed is 1.25, runtime is containerd and cni is calico.

containerd, kubeadm, and kubelet can also be installed using a package manager, and since I don't want to match the repo or check what differences there are in the repo versions, these are all installed in the native binary way.

Environmental information

IP Hostname OS Version Kernel Version Comment
192.168.0.11 node1 centos 7.9 3.10 control panel
192.168.0.12 node2 centos 7.9 3.10 worker node
192.168.0.13 node3 centos 7.9 3.10 worker node

Component version information

  • containerd: 1.7.21
  • k8s: 1.25.16

system initialization

The system initialization step requires each node to operate, and some hostname and other information needs to be modified according to the actual situation.

  1. Modify the hostname, k8s requires a different hostname for each node
hostnamectl set-hostname node1
hostnamectl set-hostname node2
hostnamectl set-hostname node3
  1. (Optional) If no dns is available to allow direct access between hostnames, you need to configure the/etc/hosts 
192.168.0.11 node1
192.168.0.12 node2
192.168.0.13 node3
  1. (Optional) If you are going to use it for a long time, it is better to configure the time synchronization.
  2. Turn off swap. by default, k8s detects swap and exits abnormally, causing k8s on the node to fail to start.
# Temporary shutdown. Permanent shutdown requires modifying /etc/fstab
swapoff -a
  1. Load the kernel module. If you do not load thebr_netfilter, the next step to configure the system parameters will report an error.
# Add configuration
cat << EOF > /etc//
overlay
br_netfilter
EOF

# Load now
modprobe overlay
modprobe br_netfilter

# Check for loading. If there is no output, it is not loaded.
lsmod | grep br_netfilter
  1. Configure system parameters. Edit/etc/Documents or/etc//directory, add or modify the following configuration. After editing, execute thesysctl -pto make the configuration take effect. (If the modification is/etc/directory.sysctl -p(You need to specify a filename for this to work)
net.ipv4.ip_forward=1
-nf-call-ip6tables = 1
-nf-call-iptables = 1
 = 0
  1. (Optional) If the kernel version is higher than 4.1, consider using ipvs to enhance network communication performance.
  2. Install the dependencies, otherwise when installing containerd, theruncComponents can be problematic
# The version of conntrack-tools is 1.4.4-4.el7, if it is lower, it may cause runc exception.
# If the installation prompts for missing dependencies that are not in the centos 7 repo source, download the rpm package from a higher version of centos, alma, etc. and install it again.
yum install -y conntrack-tools

Installing containerd

  1. through (a gap)/containerd/containerd/releasesDownload the binary package
  2. Extract the archive to the root directory. The paths of the files inside the zip archive have been organized according to the root directory, so just extract them directly to the root path.
tar xf cri-containerd-cni-1.7. -C /
  1. Generate the containerd configuration file
mkdir /etc/containerd
containerd config default > /etc/containerd/
  1. Edit the containerd configuration file/etc/containerd/The main changes are to the container's data directory and to enable systemd's cgroup.
# Modify the data storage directory
root = "/home/apps/containerd"

# For linux distributions that use systemd as the init system, it is officially recommended to use systemd as the container cgroup driver.
# Change false to true
SystemdCgroup = true
  1. Reload systemd configuration, start containerd
systemctl daemon-reload
systemctl start containerd
systemctl enable containerd
  1. Simply verify that the containerd is working.
# Check systemd status
systemctl status containerd
# View image, normally there is no image yet
crictl images
# Verify that runc is working properly, if the output reports an error, refer to step 8 of "System Initialization", "Install Dependencies".
runc --version

Install kubelet and kubeadm

The steps in this section operate on all k8s nodes. k8s binaries installation package can be downloaded from github:/kubernetes/kubernetes/releases

Find the download link for the binary package in changelog and download the server binary, which contains the master and node binaries.

  1. Unzip the downloaded zip file and put the binaries into the environment variablePATHdirectory
tar xf 
cd kubernetes/server/bin/
find . -type f -perm /u+x -exec cp {} /usr/local/bin/ \;
  1. New or edited service file for kubelet/usr/lib/systemd/system/ 
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=/docs/
Wants=
After=

[Service]
ExecStart=/usr/local/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=
  1. Create a catalog
mkdir -p  /usr/lib/systemd/system//
  1. Create or edit a new file/usr/lib/systemd/system// 
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/ --kubeconfig=/etc/kubernetes/"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the . object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/local/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
  1. Starting a kubelet
systemctl enable --now kubelet

Creating a Cluster

The steps to create a cluster are available on the control panel node.

  1. Initialize the cluster, here mainly specify the k8s version, according to the demand, refer to thekubeadm init --helpPrompts to configure initialization parameters such as pause mirror address, pod ip range, etc.
kubeadm init --kubernetes-version v1.25.16

in the event thatinitFailed to report an error. Check it out.containerdcap (a poem)kubeletThe log of the When it fails, you can reset thekubeadm reset

systemctl status containerd
journalctl -xeu containerd

systemctl status kubelet
journalctl -xeu kubelet

in the event thatinitSuccess, the console will output the command for the worker node to join the cluster, just paste this command into the worker node and execute it, for example:

kubeadm join 192.168.0.11:6443 --token 123456 \
        --discovery-token-ca-cert-hash sha256:123456

initAfter success, the output will also prompt to create kubeconfig, follow the prompts to operate can be

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/ $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
  1. Configure the network plugin. k8s requires the cni plugin to allow proper communication between pods, here the calico plugin is used
# Download the configuration file. After downloading, you can modify the image address in it according to your needs
wget /manifests/

# Deploy calico
kubectl apply -f
  1. Check the effect of network plugin deployment. Successful only if all ready are 1/1
kubectl get nodes
kubectl get pod -n kube-system

Test Cluster

After the deployment is complete, you can start a pod to try to see if it can be scheduled properly.

# Create apod。nginxadvance noticedocker pull
kubectl create deployment nginx --image=nginx
# expose a port
kubectl expose deployment nginx --port=80 --type=NodePort
# View Status
kubectl get pods,svc
<Previous Next>