Nacos is an acronym for Dynamic Naming and Configuration Service, a dynamic service discovery, configuration management and service management platform that makes it easier to build cloud-native applications.
Nacos is built onserviceservice infrastructure for modern application-centric architectures (e.g., microservices paradigm, cloud-native paradigm). More features can be found atNacos Overview。
In this article, I'll provide you with a comprehensive hands-on guide that walks you through in detail how to deploy Nacos services running in cluster mode in a Kubernetes cluster.
Practical server configuration (architecture 1:1 replica of small-scale production environment, configuration is slightly different)
| hostname (of a networked computer) | IP | CPU | random access memory (RAM) | system disk | data disk | use |
|---|---|---|---|---|---|---|
| ksp-registry | 192.168.9.90 | 4 | 8 | 40 | 200 | Harbor mirror repository |
| ksp-control-1 | 192.168.9.91 | 4 | 8 | 40 | 100 | KubeSphere/k8s-control-plane |
| ksp-control-2 | 192.168.9.92 | 4 | 8 | 40 | 100 | KubeSphere/k8s-control-plane |
| ksp-control-3 | 192.168.9.93 | 4 | 8 | 40 | 100 | KubeSphere/k8s-control-plane |
| ksp-worker-1 | 192.168.9.94 | 8 | 16 | 40 | 100 | k8s-worker/CI |
| ksp-worker-2 | 192.168.9.95 | 8 | 16 | 40 | 100 | k8s-worker |
| ksp-worker-3 | 192.168.9.96 | 8 | 16 | 40 | 100 | k8s-worker |
| ksp-storage-1 | 192.168.9.97 | 4 | 8 | 40 | 400+ | ElasticSearch/Longhorn/Ceph/NFS |
| ksp-storage-2 | 192.168.9.98 | 4 | 8 | 40 | 300+ | ElasticSearch/Longhorn/Ceph |
| ksp-storage-3 | 192.168.9.99 | 4 | 8 | 40 | 300+ | ElasticSearch/Longhorn/Ceph |
| ksp-gpu-worker-1 | 192.168.9.101 | 4 | 16 | 40 | 100 | k8s-worker(GPU NVIDIA Tesla M40 24G) |
| ksp-gpu-worker-2 | 192.168.9.102 | 4 | 16 | 40 | 100 | k8s-worker(GPU NVIDIA Tesla P100 16G) |
| ksp-gateway-1 | 192.168.9.103 | 2 | 4 | 40 | Self-built application service proxy gateway/VIP: 192.168.9.100 | |
| ksp-gateway-2 | 192.168.9.104 | 2 | 4 | 40 | Self-built application service proxy gateway/VIP: 192.168.9.100 | |
| ksp-mid | 192.168.9.105 | 4 | 8 | 40 | 100 | Service nodes deployed outside the k8s cluster (Gitlab, etc.) |
| add up the total | 15 | 68 | 152 | 600 | 2100+ |
Real-world environment involving software version information
- Operating System:openEuler 22.03 LTS SP3 x86_64
- KubeSphere：v3.4.1
- Kubernetes：v1.28.8
- KubeKey: v3.1.1
- MySQL：v5.7.44
- Nacos： v2.4.2.1
1. Deployment programming
1.1 Deployment architecture diagram
1.2 Preparing Nacos Deployment Resources
- Creating the Deployment Resource Root Directory
mkdir /srv/nacos
cd /srv/nacos
- gainList of official resourcing
# wget method (recommended)
wget /nacos-group/nacos-k8s/zip/refs/heads/master -O
# git
git clone /nacos-group/
- gainInitialize database files
wget /alibaba/nacos/refs/heads/master/distribution/conf/
1.3 Preparing MySQL
Nacos requires the use of MySQL, and this article uses a more production-oriented MySQL master-slave replication scheme to deploy MySQL, which can be found atDeploying a Master-Slave Replication of MySQL on a Kubernetes Cluster in One Article。
Tip: You can also use the official list of resources provided
deploy/mysql/The MySQL service can be deployed on a stand-alone computer.
Step 1: Import MySQL Initialization Data
- Go inside the MySQL master node container.
$ kubectl exec -it mysql-source-0 -- mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.44-log MySQL Community Server (GPL)
Copyright (c) 2000, 2023, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
- Create the database and Nacos users.
-- Create database
mysql> CREATE DATABASE IF NOT EXISTS `nacos` DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- Create the user
mysql> CREATE USER 'nacos'@'%' IDENTIFIED BY 'ChangeMe';
-- Give permissions
mysql> GRANT ALL PRIVILEGES ON `nacos`. * TO 'nacos'@'%'; -- GRANT ALL PRIVILEGES ON `nacos`.
-- Flush Privileges
mysql> FLUSH PRIVILEGES; -- Refresh permissions.
- Import data (without logging inside the container).
# Enter the database initialization sql file directory
$ cd /srv/nacos/
# Import data
kubectl exec -i mysql-source-0 -- mysql -S /var/lib/mysql/ -u nacos -pChangeMe nacos <
Step 2: View the imported data
- Login inside the MySQL master node container.
$ kubectl exec -it mysql-source-0 -- mysql -u nacos -p
Enter password:
- View the database.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| nacos |
+--------------------+
2 rows in set (0.01 sec)
- View Table.
# Switching databases
mysql> use nacos;
Database changed
# View tables
mysql> show tables;
+----------------------+
| Tables_in_nacos |
+----------------------+
| config_info |
| config_info_aggr | config_info_beta | config_info_aggr
| config_info_beta |
| config_info_tag |
| config_tags_relation |
| group_capacity |
| his_config_info |
| group_capacity | | his_config_info | | permissions |
| group_capacity | | his_config_info | permissions | roles |
| group_capacity | | group_info
| tenant_info |
| users |
+----------------------+
12 rows in set (0.00 sec)
From the results, you can see that executing After that, the relevant tables and data for the Nacos service are automatically created.
1.4 Preparing the Persistent Store
This environment uses NFS as the persistent storage for K8s clusters, and for new clusters you can refer to theThe Ultimate Hands-On Guide to Exploring Kubernetes Persistent Storage with NFS Deploy NFS storage.
Tip: You can also use the officialdeploy/nfs/ directory for a list of resources to deploy a standalone NFS service.
2. Cluster mode Nacos deployment
2.1 Modification of configuration files
- Unzip the deployment code.
$ cd /srv/nacos
$ unzip
$ cd nacos-k8s-master/deploy/nacos
- compiler, modify the database configuration.
data.
: "" # database address, this article uses the DNS domain name of the MySQL service within the k8s cluster
: "nacos"
: "3306"
: "nacos"
: "ChangeMe"
- Modify the StoreClass name (Optional, used when building your own NFS storage）。
The default profile uses a StoreClass name ofmanaged-nfs-storage, use the following command to change to the actual value.
$ sed -i 's/managed-nfs-storage/nfs-sc/g'
- Remove serviceAccountName(Optional, used when building your own NFS storage）。
sed -i '/serviceAccountName/d'
- Modify the mirror address (Optional, for limited image downloads or offline deployments）。
sed -i 's#nacos/nacos-peer-finder-plugin:1.1#:8443/nacos/nacos-peer-finder-plugin:1.1#g'
sed -i 's#nacos/nacos-server:latest#:8443/nacos/nacos-server:v2.4.2.1#g'
- Enable forensic configuration (suggestion）。
Nacos is not configured with authentication turned on by default.Recommended for production environments. In Add the following to the section:
- name: NACOS_AUTH_ENABLE
value: "true"
- name: NACOS_AUTH_TOKEN
value: "SecretKeyYzJlMTMxOTU5ZTljZTkxZGQ2MDcwZGIxMzU1YTFkMjg="
- name: NACOS_AUTH_IDENTITY_KEY
value: "serverIdentity"
- name: NACOS_AUTH_IDENTITY_VALUE
value: "ChangeMe"
Attention: customizableNACOS_AUTH_TOKEN It is recommended that you set the configuration item toBase64 encodingstring, andOriginal key length must not be less than 32 characters。
The following command can be executed to generate a TOKEN key:
echo -n $(openssl rand -hex 16) | base64 -w0
2.2 Deploying a Nacos Cluster
- Execute the following command to create Nacos.
$ kubectl create -f
When executed correctly, the output is as follows :
$ kubectl create -f
service/nacos-headless created
configmap/nacos-cm created
/nacos created
- Verify Nacos node status.
$ kubectl get pod -l app=nacos -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nacos-0 1/1 Running 0 25s 10.233.96.233 ksp-worker-3 <none> <none>
nacos-1 1/1 Running 0 25s 10.233.94.125 ksp-worker-1 <none> <none>
nacos-2 1/1 Running 0 25s 10.233.68.221 ksp-worker-2 <none> <none>
2.3 Configuring K8s Cluster External Access
We use NodePort to publish the Nacos service externally in the Kubernetes cluster so that administrators can access the graphical console and also serve applications outside the cluster, specifying a port of31848。
utilizationvi editor, create a new NodePort service resource manifest file, and enter the following:
kind: Service
apiVersion: v1
metadata:
name: nacos-external
labels:
app: nacos-external
spec:
ports:
- protocol: TCP
port: 8848
targetPort: 8848
nodePort: 31848
selector:
app: nacos
type: NodePort
2.4 Setting the administrator password
since2.4.0 Starting with this release, Nacos builds no longer provide an administrator usernacosThe default password for the administrator user needs to be enabled for the first time after authentication is turned on, either through the API or the Nacos console.nacosof the password initialization.
In this article, we have chosen the Nacos console method of initializing passwords, which verifies the administrator user when accessing the Nacos console after the Nacos cluster has turned on authentication.nacosIf it is found that the password has not been initialized, it jumps to the page for initializing the password to initialize it.
Just enter your custom password in the Password text box on that page and click Submit.
Attention: If you do not enter a customized password in the Password text box or if you enter a blank password, Nacos will generate a random password, so please save the generated random password.
After successful initialization, there will be a pop-up window indicating successful initialization and explicitly displaying the specified password or randomly generated password, please save this password.
After clicking "OK", you will be redirected to the login page with a pop-up box of permission authentication failure.
After clicking OK, enter the nacos username and corresponding password.
After you have successfully logged in, you will enter the "Configuration Management" page.
3. Verify that the test Nacos service is properly configured.
Verify that the Nacos service is working by calling the Nacos API interface on a machine outside the K8s cluster using the curl command on the NodePort port corresponding to the Nacos external service.
3.1 Getting a Token
First, log in to nacos with your username and password. If your username and password are correct, the Token information will be returned.
curl -X POST 'http://192.168.9.91:31848/nacos/v1/auth/login' -d 'username=nacos&password=ChangeMe'
When executed correctly, the result returned is as follows:
{"accessToken":"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTcyNzcwOTk0Mn0.Ki2kgZyh_dj_Zfb9HKPCkKr1cgWfi3szQS4hlZPIwkI","tokenTtl":18000,"globalAdmin":true,"username":"nacos"}
3.2 Registration of services
curl -X POST 'http://192.168.9.91:31848/nacos/v1/ns/instance?accessToken=eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTcyNzcwOTk0Mn0.Ki2kgZyh_dj_Zfb9HKPCkKr1cgWfi3szQS4hlZPIwkI&serviceName=&ip=192.168.9.81&port=8080'
3.3 Service discovery
curl -X GET 'http://192.168.9.91:31848/nacos/v1/ns/instance/list?accessToken=eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTcyNzcwOTk0Mn0.Ki2kgZyh_dj_Zfb9HKPCkKr1cgWfi3szQS4hlZPIwkI&serviceName='
When executed correctly, the result returned is as follows:
{"name":"DEFAULT_GROUP@@","groupName":"DEFAULT_GROUP","clusters":"","cacheMillis":10000,"hosts":[],"lastRefTime":1727692102280,"checksum":"","allIPs":false,"reachProtectionThreshold":false,"valid":true}[
3.4 Release Configuration
curl -X POST "http://192.168.9.91:31848/nacos/v1/cs/configs?accessToken=eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTcyNzcwOTk0Mn0.Ki2kgZyh_dj_Zfb9HKPCkKr1cgWfi3szQS4hlZPIwkI&dataId=&group=test&content=helloWorld"
3.5 Obtaining Configurations
curl -X GET "http://192.168.9.91:31848/nacos/v1/cs/configs?accessToken=eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJuYWNvcyIsImV4cCI6MTcyNzcwOTk0Mn0.Ki2kgZyh_dj_Zfb9HKPCkKr1cgWfi3szQS4hlZPIwkI&dataId=&group=test"
When executed correctly, the result returned is as follows:
helloWorld
3.6 Nacos Console View
- Configuration Management List
- Configuration Details
At this point, we have completed the entire process of manually deploying a Nacos cluster on a KubeSphere-managed Kubernetes cluster, and we are now ready to configure it according to the needs of the actual application.
Disclaimer:
- The author's level of competence is limited, although every effort has been made to ensure the accuracy of the content, after many verifications and checks.But there may still be omissionsI'd like to ask the experts in the industry to give me some advice. I'm looking forward to hearing from the experts in the industry.
- The content described in this article is only verified through the real-world environment testing, readers can learn, learn from, butDirect use in production environments is strictly prohibited。The author is not responsible for any problems arising from this!
The content of this article was first published: Ops has the art.
About KubeSphere
KubeSphere （The company is an open source container platform built on top of Kubernetes, providing full-stack IT automation capabilities and simplifying DevOps workflows for the enterprise.
KubeSphere has been adopted by Aqara Smart Home, BenLife, Orient Communications, Microhome, Neusoft, Huayun, Sina, Sany Heavy Industry, Huaxia Bank, Sichuan Airlines, Sinopharm, Microcrowd Bank, Zijin Insurance, GoWhere.com, Zhongtong, People's Bank of China, Bank of China, PICC Life Insurance, China Taiping Insurance, China Mobile, China Unicom, China Telecom, Tianyi Cloud, KubeSphere provides a developer-friendly wizard-like interface and rich enterprise-class features, including Kubernetes multi-cloud and multi-cluster management, DevOps (CI/CD), application lifecycle management, edge computing, Service Mesh, multi-tenancy, and multi-cluster management. Mesh, multi-tenant management, observability, storage and network management, GPU support, and more to help organizations quickly build a powerful and feature-rich container cloud platform.
✨ GitHub：/kubesphere
💻 Official website (China site):/zh
🙋 Forum:/forum/
👨💻 WeChat group: please search and add group assistant micro signal kubesphere