A few days ago, KubeSphere v4 was released, and compared to the previous version, the new version has a disruptive change in architecture. In order to make the transition from the old version to the new version silky smooth for all the community members, we are releasing this installation guide article for reference.
For an introduction to KubeSphere v4, read this article:KubeSphere v4 Open Source and Releases New Pluggable Architecture LuBan。
Note that upgrading from KubeSphere 3.4.1 to v4 is not supported at this time. You need to uninstall the original version before installing the v4 version.
Uninstalling KubeSphere 3.4.1
Attention:
- This article only applies to the test environment, please do not operate directly in the production environment.
- If you need to operate in the production environment, please verify it in the test environment first before proceeding.
- Uninstallation is a high-risk operation, so make sure you know what you are about to do before you perform it.
- This action renders the KubeSphere platform itself unusable, but does not affect workloads running outside of KubeSphere, i.e., in a K8s cluster.
- This action deletes all KubeSphere components and related data, which you can back up before doing so.
- You can choose the data migration tool on your own, or wait for the community's data migration program, which is planned to help you backup your platform accounts, permissions and related data through scripts, which can be imported after the new version is installed.
- If you are expecting a full data migration and upgrade, we suggest you considerKubeSphere Enterprise Edition。
- The community solemnly reminds you to proceed with caution.
Unbundle Cluster
If multiple clusters are enabled, be sure to unbundle the clusters before uninstalling. Before uninstalling a Host cluster, make sure that no Member clusters are currently managed by the current cluster, and that roles and accounts are deleted.
Uninstalling KubeSphere 3.4.1
Execute this script for the cluster to be offloaded.
Attention:
- Make sure the current cluster is unbound from the Host cluster before executing this script.
- Make sure that the cluster you are about to operate on is the one that will be unmounted.
- Executing this script removes Prometheus and Elasticsearch from the cluster; inventory monitoring and logging data is not retained.
- After executing this script, the project gateway can still be used in the cluster, but you need to switch the gateway to the new version of the gateway after nanopiping to KubeSphere 4.1.2, and you need to delete the old version of the gateway and switch to the new version of the gateway during the switching process.
Uninstalling DevOps Components
helm del -n argocd devops
helm del -n kubesphere-devops-system devops
Uninstalling ServiceMesh Components
kubectl -n istio-system delete jaeger jaeger
kubectl -n istio-system delete kiali kiali
helm del -n istio-system kiali-operator
helm del -n istio-system jaeger-operator
# downloading istioctl
wget /istio/istio/releases/download/1.15.6/istioctl-1.15.
tar -zxvf istioctl-1.15.
./istioctl uninstall --purge
Uninstall ks-core, monitoring and logging related components
#！/bin/bash
############################################################################################################
# This script is used to unload the cluster's KubeSphere v3.4.1
#
# take note of： If it is a multi-cluster environment，Before executing this script, make sure the current cluster has been removed from the host Unbundling in clusters
############################################################################################################
set -x
# Clear the cluster of all namespace hit the nail on the head workspace tab (of a window) (computing)
kubectl get ns -l /workspace -o name | xargs -I {} bash -c "kubectl label {} /workspace- && kubectl patch {} -p '{\"metadata\":{\"ownerReferences\":[]}}' --type=merge"
# # Clear the cluster of all namespace hit the nail on the head kubefed tab (of a window) (computing)
kubectl get ns -l /managed -o name | xargs -I {} bash -c "kubectl label {} /managed- && kubectl patch {} -p '{\"metadata\":{\"ownerReferences\":[]}}' --type=merge"
# 清除集群hit the nail on the head workspace as well as workspacetemplate resource (such as manpower or tourism)
kubectl get workspacetemplate -A -o name | xargs -I {} kubectl patch {} -p '{"metadata":{"ownerReferences":[]}}' --type=merge
kubectl get workspace -A -o name | xargs -I {} kubectl patch {} -p '{"metadata":{"ownerReferences":[]}}' --type=merge
kubectl get workspacetemplate -A -o name | xargs -I {} kubectl delete {}
kubectl get workspace -A -o name | xargs -I {} kubectl delete {}
# removing clusterroles
delete_cluster_roles() {
for role in `kubectl get clusterrole -l /role-template -o jsonpath="{.items[*].}"`
do
kubectl delete clusterrole $role 2>/dev/null
done
}
delete_cluster_roles
# removing clusterrolebindings
delete_cluster_role_bindings() {
for rolebinding in `kubectl get clusterrolebindings -l /role-template -o jsonpath="{.items[*].}"`
do
kubectl delete clusterrolebindings $rolebinding 2>/dev/null
done
}
delete_cluster_role_bindings
# removing validatingwebhookconfigurations
for webhook in ks-events-admission-validate validating-webhook-configuration
do
kubectl delete . $webhook 2>/dev/null
done
# removing mutatingwebhookconfigurations
for webhook in ks-events-admission-mutate logsidecar-injector-admission-mutate mutating-webhook-configuration
do
kubectl delete . $webhook 2>/dev/null
done
# removing users
for user in `kubectl get users -o jsonpath="{.items[*].}"`
do
kubectl patch user $user -p '{"metadata":{"finalizers":null}}' --type=merge
done
kubectl delete users --all 2>/dev/null
# removing iam resource (such as manpower or tourism)
for resource_type in `echo globalrolebinding loginrecord rolebase workspacerole globalrole workspacerolebinding`; do
for resource_name in `kubectl get ${resource_type}. -o jsonpath="{.items[*].}"`; do
kubectl patch ${resource_type}. ${resource_name} -p '{"metadata":{"finalizers":null}}' --type=merge
done
kubectl delete ${resource_type}. --all 2>/dev/null
done
# uninstallation ks-core
helm del -n kubesphere-system ks-core
helm del -n kubesphere-system ks-redis &> /dev/null || true
kubectl delete pvc -n kubesphere-system -l app=redis-ha --ignore-not-found || true
kubectl delete deploy -n kubesphere-system -l /managed-by!=Helm --field-selector =redis --ignore-not-found || true
kubectl delete svc -n kubesphere-system -l /managed-by!=Helm --field-selector =redis --ignore-not-found || true
kubectl delete secret -n kubesphere-system -l /managed-by!=Helm --field-selector =redis-secret --ignore-not-found || true
kubectl delete cm -n kubesphere-system -l /managed-by!=Helm --field-selector =redis-configmap --ignore-not-found || true
kubectl delete pvc -n kubesphere-system -l /managed-by!=Helm --field-selector =redis-pvc --ignore-not-found || true
kubectl delete deploy -n kubesphere-system --all --ignore-not-found
kubectl delete svc -n kubesphere-system --all --ignore-not-found
kubectl delete cm -n kubesphere-system --all --ignore-not-found
kubectl delete secret -n kubesphere-system --all --ignore-not-found
kubectl delete sa -n kubesphere-system --all --ignore-not-found
# removing Gateway resource (such as manpower or tourism)
for gateway in `kubectl -n kubesphere-controls-system get -o jsonpath="{.items[*].}"`
do
kubectl -n kubesphere-controls-system patch $gateway -p '{"metadata":{"finalizers":null}}' --type=merge
done
kubectl -n kubesphere-controls-system delete --all 2>/dev/null
# removingcrd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
## uninstallation Monitoring Components
# removing Prometheus/ALertmanager/ThanosRuler
kubectl -n kubesphere-monitoring-system delete Prometheus k8s --ignore-not-found
kubectl -n kubesphere-monitoring-system delete secret additional-scrape-configs --ignore-not-found
kubectl -n kubesphere-monitoring-system delete serviceaccount prometheus-k8s --ignore-not-found
kubectl -n kubesphere-monitoring-system delete service prometheus-k8s --ignore-not-found
kubectl -n kubesphere-monitoring-system delete role prometheus-k8s-config --ignore-not-found
kubectl -n kubesphere-monitoring-system delete rolebinging prometheus-k8s-config --ignore-not-found
kubectl -n kubesphere-monitoring-system delete Alertmanager main --ignore-not-found
kubectl -n kubesphere-monitoring-system delete secret alertmanager-main --ignore-not-found
kubectl -n kubesphere-monitoring-system delete service alertmanager-main --ignore-not-found
kubectl -n kubesphere-monitoring-system delete ThanosRuler kubesphere --ignore-not-found
# removing ServiceMonitor/PrometheusRules
kubectl -n kubesphere-monitoring-system delete ServiceMonitor alertmanager coredns etcd ks-apiserver kube-apiserver kube-controller-manager kube-proxy kube-scheduler kube-state-metrics kubelet node-exporter prometheus prometheus-operator s2i-operator thanosruler --ignore-not-found
kubectl -n kubesphere-monitoring-system delete PrometheusRule kubesphere-rules prometheus-k8s-coredns-rules prometheus-k8s-etcd-rules prometheus-k8s-rules --ignore-not-found
# removing prometheus-operator
kubectl -n kubesphere-monitoring-system delete deployment prometheus-operator --ignore-not-found
kubectl -n kubesphere-monitoring-system delete service prometheus-operator --ignore-not-found
kubectl -n kubesphere-monitoring-system delete serviceaccount prometheus-operator --ignore-not-found
# removing kube-state-metrics/node-exporter
kubectl -n kubesphere-monitoring-system delete deployment kube-state-metrics --ignore-not-found
kubectl -n kubesphere-monitoring-system delete service kube-state-metrics --ignore-not-found
kubectl -n kubesphere-monitoring-system delete serviceaccount kube-state-metrics --ignore-not-found
kubectl -n kubesphere-monitoring-system delete daemonset node-exporter --ignore-not-found
kubectl -n kubesphere-monitoring-system delete service node-exporter --ignore-not-found
kubectl -n kubesphere-monitoring-system delete serviceaccount node-exporter --ignore-not-found
# removing Clusterrole/ClusterRoleBinding
kubectl delete clusterrole kubesphere-prometheus-k8s kubesphere-kube-state-metrics kubesphere-node-exporter kubesphere-prometheus-operator
kubectl delete clusterrolebinding kubesphere-prometheus-k8s kubesphere-kube-state-metrics kubesphere-node-exporter kubesphere-prometheus-operator
# removing notification-manager
helm delete notification-manager -n kubesphere-monitoring-system
# clear up kubesphere-monitoring-system
kubectl delete deploy -n kubesphere-monitoring-system --all --ignore-not-found
# removing监控 crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
kubectl delete crd
# removing metrics-server
kubectl delete apiservice .
kubectl -n kube-system delete deploy metrics-server
kubectl -n kube-system delete service metrics-server
kubectl delete ClusterRoleBinding system:metrics-server
kubectl delete ClusterRoleBinding metrics-server:system:auth-delegator
kubectl -n kube-system delete RoleBinding metrics-server-auth-reader
kubectl delete ClusterRole system:metrics-server
kubectl delete ClusterRole system:aggregated-metrics-reader
kubectl -n kube-system delete ServiceAccount ServiceAccount
## uninstallation log component
# removing fluent-bit
kubectl -n kubesphere-logging-system delete fluentbitconfigs fluent-bit-config --ignore-not-found
kubectl -n kubesphere-logging-system patch fluentbit fluent-bit -p '{"metadata":{"finalizers":null}}' --type=merge
kubectl -n kubesphere-logging-system delete fluentbit fluent-bit --ignore-not-found
# removing ks-logging
helm del -n kubesphere-logging-system logsidecar-injector &> /dev/null || true
# removing ks-events
helm del -n kubesphere-logging-system ks-events &> /dev/null || true
# removing kube-auditing
helm del -n kubesphere-logging-system kube-auditing &> /dev/null || true
# removing es
helm del -n kubesphere-logging-system elasticsearch-logging &> /dev/null || true
helm del -n kubesphere-logging-system elasticsearch-logging-curator &> /dev/null || true
# removing opensearch
helm del -n kubesphere-logging-system opensearch-master &> /dev/null || true
helm del -n kubesphere-logging-system opensearch-data &> /dev/null || true
helm del -n kubesphere-logging-system opensearch-logging-curator &> /dev/null || true
# clear up kubesphere-logging-system
kubectl delete deploy -n kubesphere-logging-system --all --ignore-not-found
Checking Namespace Tags
Make sure that all Namespaces do not contain/workspace Tags.
kubectl get ns --show-labels
Uninstalling Kubefed (Host cluster)
helm del -n kube-federation-system kubefed
Installing KubeSphere 4.1.2
Upgrade Crd
# downloading ks-core chart contract (to or for)
helm fetch /main/ks-core-1.1. --untar
# update crds
kubectl apply -f ks-core/charts/ks-crds/crds/
Host cluster installation of ks-core
Starting with KubeSphere v4.1, ks-core can be deployed only in Host clusters, and Member clusters are added via a page (do not deploy ks-core again in Member clusters).
# This command needs to be executed only on the host cluster
# If access to dockerhub is limited, add the following command to
# --set =/ks
# --set =/ks
helm upgrade --install -n kubesphere-system --create-namespace ks-core /main/ks-core-1.1. --debug --wait
Adding a Member Cluster
After the Host cluster is deployed, go to the Workbench and click theCluster Management：
strike (on the keyboard)Add Cluster：
Fill in the cluster information and clickthe next step：
Fill out the Member cluster Kubeconfig:
Fill in the cluster configuration:
# Fill in here if access to dockerhub is limited, or skip this step if you have normal access to dockerhub.
global.
imageRegistry: /ks
Click Create and wait for the ks-agent on the Member cluster to be created successfully.
Gateway Migration
Installing Gateway Components
Install the KubeSphere Gateway Extension Component and install the Gateway Extension Component Agent for the clusters that need to use the gateway.
Add projects that use the Project Gateway to the target enterprise space
When you execute the uninstall script, the enterprise space association of the project is deleted, and you need to rejoin the project that uses the project gateway to the new target enterprise space.
Migrating the project gateway to a new version
Note: This step will result in a disruption to the project gateway and business impacts need to be considered in advance.
Delete the project gateway to be migrated
To ensure that the access ports remain unchanged, be sure to record the Nodeport or LoadBalancer information for the corresponding gateway before deleting it, so that the original configuration is preserved when a new gateway is created.
New Project Gateway
Go to Gateway Settings in the Project Settings of the project through Enterprise Space.
Take Nodeport, which retains its original gateway, as an example:
Click OK.
Associative Application Routing to New Gateway
Once the new project gateway has been created successfully, click Admin->Edit in the gateway page and copy the IngressClassName in it.
For the application route under the current project, click Edit YAML.
Spec add ingressClassName: xxx, save.
summarize
These are the complete steps for KubeSphere v4 installation for your reference. If you have problems during the installation, you can go to the forum to search if there is any answer:/forum/。
KubeSphere v4 Call for Papers: We invite you to experience and share your best practices! Those whose submissions are accepted will receive community gifts such as T-shirts and canvas bags, and outstanding authors will have a chance to win CKA coupons. For submissions, please contact our assistant, KK, by searching for "kubesphere" on WeChat.
About KubeSphere
KubeSphere （The company is an open source container platform built on top of Kubernetes, providing full-stack IT automation capabilities and simplifying DevOps workflows for the enterprise.
KubeSphere has been adopted by Aqara Smart Home, BenLife, Orient Communications, Microhome, Neusoft, Huayun, Sina, Sany Heavy Industry, Huaxia Bank, Sichuan Airlines, Sinopharm, Microcrowd Bank, Zijin Insurance, GoWhere.com, Zhongtong, People's Bank of China, Bank of China, PICC Life Insurance, China Taiping Insurance, China Mobile, China Unicom, China Telecom, Tianyi Cloud, China Mobile, China Unicom, China Telecom, Tianyi Cloud, China Mobile Gold, Radore, ZaloPay, and tens of thousands of domestic and international enterprises.
KubeSphere provides a developer-friendly wizard-like interface and a rich set of enterprise-class features, including Kubernetes multi-cloud and multi-cluster management, DevOps (CI/CD), application lifecycle management, edge computing, Service Mesh, multi-tenant management, observability, storage and network management, GPU support, and more. support and other features to help enterprises quickly build a powerful and feature-rich container cloud platform.
✨ GitHub：/kubesphere
💻 Official website (China site):/zh
🙋 Forum:/forum/
👨💻 WeChat group: please search and add group assistant micro signal kubesphere