in the previous chapterLocalized Cintron Open Source Cloud Native PlatformsIn the article, we introduce Rainbond, as probably the first open source localized SINC platform in China, in terms of its ability to support localization and SINC, and briefly introduce how to deploy Kubernetes and Rainbond online in a localized SINC environment.
However, the need for offline deployment is more prevalent for most localized SINC environments, such as banks, governments, and other organizations. It is worth noting that the official Rainbond documentation currently only provides guidelines for deploying Rainbond offline in existing Kubernetes environments. So why don't we provide documentation for deploying Kubernetes offline?The Rainbond open source community is different from other open source communities in that Rainbond takes responsibility for every open source user's issue and actively helps to resolve it. However, this creates an additional workload for the community support team, especially when dealing with issues that are outside of Rainbond's scope.
Therefore, this article will introduce in detail how to deploy Kubernetes and Rainbond in a localized Xintronics environment, in the hope of providing users with practical guidance and reducing troubles in the deployment process.
Prepare offline image and installer
Prepare the following images and installation packages in a networked Arm environment.
Docker Offline Package
Download the Docker offline installer and offline installation script.
wget /offline/docker/docker-arm-20.10.
wget /install_docker_offline.sh
Kubernetes Offline Package
This deployment of the K8s version isv1.23.10The Rancher Kubernetes Engine, or RKE, is a CNCF-certified Kubernetes installer.
Get the following offline packages in your Arm environment.
# Kubectl and Helm binaries
wget /pkg/kubectl/v1.23.10/kubectl-arm -O kubectl
wget /pkg/helm/v3.10.1/helm-arm64 -O helm
# RKE install binary
wget /pkg/rke/v1.3.15/rke-arm -O rke
#!/bin/bash
# RKE Dockermirroring
image_list="/goodrain/mirrored-coreos-etcd:v3.5.3
/goodrain/rke-tools:v0.1.87
/goodrain/rke-tools:v0.1.87
/goodrain/rke-tools:v0.1.87
/goodrain/rke-tools:v0.1.87
/goodrain/mirrored-k8s-dns-kube-dns:1.21.1
/goodrain/mirrored-k8s-dns-dnsmasq-nanny:1.21.1
/goodrain/mirrored-k8s-dns-sidecar:1.21.1
/goodrain/cluster-proportional-autoscaler:1.8.1
/goodrain/mirrored-coredns-coredns:1.10.1
/goodrain/cluster-proportional-autoscaler:1.8.1
/goodrain/mirrored-k8s-dns-node-cache:1.21.1
/goodrain/hyperkube:v1.23.10-rancher1
/goodrain/mirrored-coreos-flannel:v0.15.1
/goodrain/flannel-cni:v0.3.0-rancher6
/goodrain/mirrored-pause:3.6
/goodrain/mirrored-metrics-server:v0.6.1"
for image in ${image_list}; do
docker pull "${image}"
done
docker save -o ${image_list}
Rainbond Offline Bag
Prepare the images needed for Rainbond in advance in an environment with internet access.
#!/bin/bash
VERSION=${VERSION:-'v5.17.3-release'}
image_list="/goodrain/kubernetes-dashboard:v2.6.1
/goodrain/registry:2.6.2
/goodrain/metrics-server:v0.4.1
/goodrain/etcd:v3.3.18
/goodrain/metrics-scraper:v1.0.4
/goodrain/rainbond:${VERSION}-allinone
/goodrain/rbd-mesh-data-panel:${VERSION}
/goodrain/rbd-webcli:${VERSION}
/goodrain/rbd-eventlog:${VERSION}
/goodrain/rbd-init-probe:${VERSION}
/goodrain/rbd-chaos:${VERSION}
/goodrain/rbd-mq:${VERSION}
/goodrain/rainbond-operator:${VERSION}
/goodrain/rbd-worker:${VERSION}
/goodrain/rbd-node:${VERSION}
/goodrain/rbd-monitor:${VERSION}
/goodrain/rbd-gateway:${VERSION}
/goodrain/rbd-api:${VERSION}
/goodrain/rbd-db:8.0.19
/goodrain/mysqld-exporter:latest
/goodrain/nfs-provisioner:latest"
for image in ${image_list}; do
docker pull "${image}"
done
docker save -o rainbond-"${VERSION}".tar ${image_list}
Get the Rainbond Helm Chart Installation Package
git clone --depth=1 /goodrain/rainbond-chart
Starting a Kubernetes Deployment
prerequisite
Before you start installing K8s, please read theRKE Installation RequirementsThis document describes the operating system, software, port, and SSH configuration requirements for RKE. Before installing, check that your node meets these requirements.
Deploying Docker
Import the Docker offline package to all nodes and execute the script to install Docker.
$ ls
docker-arm-20.10. install_docker_offline.sh
$ bash ./install_docker_offline.sh
Import K8s-related offline packages and Docker images to all nodes.
Configuring Docker Users
RKE requires an unencrypted user for subsequent cluster installations who has permission to execute Docker.
# Create a user and join the root group
adduser -g root docker && echo "docker:password" | chpasswd
# Generate the ssh key, just keep entering all defaults
ssh-keygen
# Configure a passwordless login
ssh-copy-id docker@xxxx
Check if you have Docker execution privileges by logging in as a Docker user.
$ ssh docker@xxxx
$ docker ps
Edit file
Installing a K8s cluster with RKE requires the use of an RKE-generated configuration file, here is my example, for more seeRKE Configuration File Description。
The only thing we need to change is thenodes field, if the address of the mirror repository where the image is imported remains unchanged, then the yml does not need to be modified.system_images All mirror addresses under the field.
nodes:
- address: 192.168.0.138
port: "22"
internal_address: 192.168.0.138
role:
- etcd
- controlplane
- worker
hostname_override: ""
user: docker
docker_socket: ""
ssh_key: ""
ssh_key_path: ~/.ssh/id_rsa
ssh_cert: ""
ssh_cert_path: ""
labels: {}
taints: []
services:
etcd:
image: ""
extra_args: {}
extra_args_array: {}
extra_binds: []
extra_env:
- ETCD_AUTO_COMPACTION_RETENTION=1
win_extra_args: {}
win_extra_args_array: {}
win_extra_binds: []
win_extra_env: []
external_urls: []
ca_cert: ""
cert: ""
key: ""
path: ""
uid: 0
gid: 0
snapshot: null
retention: ""
creation: ""
backup_config: null
kube-api:
image: ""
extra_args: {}
extra_args_array: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_args_array: {}
win_extra_binds: []
win_extra_env: []
service_cluster_ip_range: 10.43.0.0/16
service_node_port_range: ""
pod_security_policy: false
always_pull_images: false
secrets_encryption_config: null
audit_log: null
admission_configuration: null
event_rate_limit: null
kube-controller:
image: ""
extra_args: {}
extra_args_array: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_args_array: {}
win_extra_binds: []
win_extra_env: []
cluster_cidr: 10.42.0.0/16
service_cluster_ip_range: 10.43.0.0/16
scheduler:
image: ""
extra_args: {}
extra_args_array: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_args_array: {}
win_extra_binds: []
win_extra_env: []
kubelet:
image: ""
extra_args: {}
extra_args_array: {}
extra_binds:
- /grlocaldata:/grlocaldata:rw,z
- /cache:/cache:rw,z
extra_env: []
win_extra_args: {}
win_extra_args_array: {}
win_extra_binds: []
win_extra_env: []
cluster_domain:
infra_container_image: ""
cluster_dns_server: 10.43.0.10
fail_swap_on: false
generate_serving_certificate: false
kubeproxy:
image: ""
extra_args: {}
extra_args_array: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_args_array: {}
win_extra_binds: []
win_extra_env: []
network:
plugin: flannel # calico
options: {}
mtu: 0
node_selector: {}
update_strategy: null
tolerations: []
authentication:
strategy: x509
sans: []
webhook: null
addons: ""
addons_include: []
system_images:
etcd: /goodrain/mirrored-coreos-etcd:v3.5.3
alpine: /goodrain/rke-tools:v0.1.87
nginx_proxy: /goodrain/rke-tools:v0.1.87
cert_downloader: /goodrain/rke-tools:v0.1.87
kubernetes_services_sidecar: /goodrain/rke-tools:v0.1.87
kubedns: /goodrain/mirrored-k8s-dns-kube-dns:1.21.1
dnsmasq: /goodrain/mirrored-k8s-dns-dnsmasq-nanny:1.21.1
kubedns_sidecar: /goodrain/mirrored-k8s-dns-sidecar:1.21.1
kubedns_autoscaler: /goodrain/cluster-proportional-autoscaler:1.8.1
coredns: /goodrain/mirrored-coredns-coredns:1.10.1
coredns_autoscaler: /goodrain/cluster-proportional-autoscaler:1.8.1
nodelocal: /goodrain/mirrored-k8s-dns-node-cache:1.21.1
kubernetes: /goodrain/hyperkube:v1.23.10-rancher1
flannel: /goodrain/mirrored-coreos-flannel:v0.15.1
flannel_cni: /goodrain/flannel-cni:v0.3.0-rancher6
calico_node: /goodrain/mirrored-calico-node:v3.22.0
calico_cni: /goodrain/calico-cni:v3.22.0-rancher1
calico_controllers: /goodrain/mirrored-calico-kube-controllers:v3.22.0
calico_ctl: /goodrain/mirrored-calico-ctl:v3.22.0
calico_flexvol: /goodrain/mirrored-calico-pod2daemon-flexvol:v3.22.0
canal_node: /goodrain/mirrored-calico-node:v3.22.0
canal_cni: ""
canal_controllers: ""
canal_flannel: ""
canal_flexvol: ""
weave_node: ""
weave_cni: ""
pod_infra_container: /goodrain/mirrored-pause:3.6
ingress: ""
ingress_backend: ""
ingress_webhook: ""
metrics_server: /goodrain/mirrored-metrics-server:v0.6.1
windows_pod_infra_container: ""
aci_cni_deploy_container: ""
aci_host_container: ""
aci_opflex_container: ""
aci_mcast_container: ""
aci_ovs_container: ""
aci_controller_container: ""
aci_gbp_server_container: ""
aci_opflex_server_container: ""
ssh_key_path: ""
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
mode: rbac
options: {}
ignore_docker_version: null
enable_cri_dockerd: null
kubernetes_version: ""
private_registries: []
ingress:
provider: none
options: {}
node_selector: {}
extra_args: {}
dns_policy: ""
extra_envs: []
extra_volumes: []
extra_volume_mounts: []
update_strategy: null
http_port: 0
https_port: 0
network_mode: ""
tolerations: []
default_backend: null
default_http_backend_priority_class_name: ""
nginx_ingress_controller_priority_class_name: ""
default_ingress_class: null
cluster_name: ""
cloud_provider:
name: ""
prefix_path: ""
win_prefix_path: ""
addon_job_timeout: 300
bastion_host:
address: ""
port: ""
user: ""
ssh_key: ""
ssh_key_path: ""
ssh_cert: ""
ssh_cert_path: ""
ignore_proxy_env_vars: false
monitoring:
provider: none
options: {}
node_selector: {}
update_strategy: null
replicas: null
tolerations: []
metrics_server_priority_class_name: ""
restore:
restore: false
snapshot_name: ""
rotate_encryption_key: false
dns: null
Perform the installation
Execute the following commands to start installing K8s. It is verified that Kirin V10 must be configured with SSHAllowTcpForwarding yesor it will report an error, seeRKE SSH Configuration。
./rke up
If you encounter errors during installation and need to clean up the cluster you can use the following script to do so.
curl -sfL /clean-rke | bash
You need to copy the kubeconfig file to the default path after the cluster has been successfully installed.
mkdir /root/.kube && cp kube_config_cluster.yml /root/.kube/config
Execute the following command to confirm the installation result
kubectl get node
Starting Rainbond Deployment
prerequisite
Each node needs to have thenfs-utils package, I will not go into detail here, there are many online tutorials, probably just mount the DVD image, and then make a local image source, directly yum install can be.
Importing an image package
docker load -i rainbond-v5.17.
Install Rainbond
Copy the Helm Chart that prepares the node Git clone.
Install Rainbond using Helm Chart.
- Creating a Namespace
kubectl create namespace rbd-system
- To write a Helm, see Chart Parameters for more information.Chart Installation Options。
operator:
image:
name: /goodrain/rainbond-operator
tag: v5.17.3-release
Cluster:
enableEnvCheck: false
gatewayIngressIPs: 192.168.0.138
nodesForChaos:
- name: 192.168.0.138
nodesForGateway:
- externalIP: 192.168.0.138
internalIP: 192.168.0.138
name: 192.168.0.138
rainbondImageRepository: /goodrain
installVersion: v5.17.3-release
Component:
rbd_app_ui:
enable: true
env:
DISABLE_DEFAULT_APP_MARKET: true
- Execute the Helm installation command
helm install rainbond ./rainbond-chart -n rbd-system -f
Installation Progress Inquiry
After executing the install command, execute the following command in the cluster to check the status of the installation.
watch kubectl get po -n rbd-system
When the name containsrbd-app-ui The installation is successful when the Pod is in the Running state.
Access platforms
Copy the following command and execute it in the cluster to get the platform access address. If there is more than one gateway node, any one of the addresses can access the console.
kubectl get rainbondcluster rainbondcluster -n rbd-system -o go-template --template='{{}}{{.}}:7070{{printf "\n"}}{{end}}'
Offline environment source code build (optional)
If you need to build source code in an offline environment, see theRainbond Offline Source Build DocumentationPerform the configuration.
ultimate
With the guidance in this article, we hope you can successfully complete the offline deployment of Kubernetes and Rainbond in Kunpeng ARM and Kirin V10 environments. The need for offline deployment is becoming more and more prevalent in localized Xintronics environments, and we provide detailed steps and examples to help you reduce the uncertainties and challenges in the deployment process. In the future, we will continue to update more related tutorials and documents to better serve the needs of the localized Cybernetics space.