Location>code7788 >text

Detailed steps to enable kerberos authentication at kafka

Popularity：665 ℃/2025-03-11 11:11:05

1. Kerberos installation and deployment

The basic principles of kerberos are not introduced too much, you can check them yourself; this article mainly introduces the installation and use of kerberos; the software version used: System: Red Hat Enterprise Linux release 8.6 (Ootpa), krb5-server: 1.18.2

#The software version used
 [root@kafka01 data]# cat /etc/redhat-release
 Red Hat Enterprise Linux release 8.6 (Ootpa)
 #install via yum
 [root@kafka01 ~]# yum install krb5-server
 #View this version number
 [root@kafka01 ~]# rpm -qi krb5-server
 Name : krb5-server
 Version: 1.18.2
 Release: 30.el8_10
 Architecture: x86_64
 Install Date: Fri 07 Mar 2025 11:11:35 AM CST
 Group: System Environment/Daemons
 Size : 1481553
 License: MIT
 Signature : RSA/SHA256, Tue 22 Oct 2024 11:00:23 PM CST, Key ID 199e2f91fd431d51

2. Prepare the machine

Serial number IP Host Deployment Services
1 192.168.10.100 kafka01 Kerberos Server、Kerberos Client
2 192.168.10.101 kafka02 Kerberos Client
3 192.168.10.102 kafka03 Kerberos Client
Bind the host file
[root@kafka01 ~]# cat /etc/hosts
192.168.10.100    kafka01
192.168.10.101    kafka02
192.168.10.102    kafka03

The Kerberos Client is installed as needed. After installation, you can use the kadmin command; the corresponding command is used on the Kerberos Server.

3. Kerberos Server installation

[root@kafka01 ~]# yum install krb5-server

1. Configuration

#Edit configuration file
 [root@kafka01 ~]# vim /etc/
 # To opt out of the system crypto-policies configuration of krb5, remove the
 # symlink at /etc//crypto-policies which will not be recreated.
 includeir /etc//

 [logging]
     default = FILE:/var/log/
     kdc = FILE:/var/log/
     admin_server = FILE:/var/log/

 [libdefaults]
     dns_lookup_realm = false
     ticket_lifetime = 24h
     renew_lifetime = 7d
     forwardable = true
     rdns = false
     pkinit_anchors = FILE:/etc/pki/tls/certs/
 # spake_preauth_groups = edwards25519
     default_realm = #Domain
     default_ccache_name = KEYRING:persistent:%{uid}

 [realms]
   = {
      kdc = kafka01 #hostname
      admin_server = kafka01 #hostname
  }

 [domain_realm]
  #.kafka01 =
  #kafka01 =
The above configuration related parameters
[logging]: The location of the log
 [libdefaults]: The default configuration for each connection
 dns_lookup_realm: Whether to search for the release to be used through dns
 ticket_lifetime: The validity time limit of the voucher, generally 24 hours
 renew_lifetime: The maximum time limit for a voucher to be extended, usually one week.  When the credential expires, subsequent access to the secure authentication service will fail
 forwardable: Whether the ticket can be forwarded (if the user already has a TGT, when he logs into another remote system, the KDC will recreate a TGT for him without having to re-authenticate the user)
 rdns: If true, in addition to the forward search based on the hostname, the corresponding principal is also searched in reverse.  If dns_canonicalize_hostname is set to false, this flag does not work.  The default value is true.
 pkinit_anchors: The location of the trusted anchor (root) certificate; if the user specifies X509_anchors on the command line, this configuration is not used.
 default_realm: The default realm must be the same as the realm name to be configured.
 default_ccache_name: Specifies the name of the default credential cache.  The default value is DEFCCNAME
 [realms]: List the realm used
 kdc: The machine running on kdc
 admin_server: The machine running on the kdc database management service
 [domain_realm]: Configure the release corresponding to domain name or hostname

 For detailed instructions, please refer to the official website document: /kerberos/krb5-latest/doc/admin/conf_files/krb5_conf.html.

2. Configuration (/var/kerberos/krb5kdc/)

[root@kafka01 data]# vim /var/kerberos/krb5kdc/ 
[kdcdefaults]
    kdc_ports = 88
    kdc_tcp_ports = 88
    spake_preauth_kdc_challenge = edwards25519

[realms]
 = {
     #master_key_type = aes256-cts
     acl_file = /var/kerberos/krb5kdc/
     dict_file = /usr/share/dict/words
     admin_keytab = /var/kerberos/krb5kdc/
     supported_enctypes = aes256-cts:normal aes128-cts:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal
}
Detailed explanation of the above configuration related parameters
Related parameter description:
 [kdcdefaults]: KDC default configuration
 kdc_ports: UDP port number
 kdc_tcp_ports: TCP port number
 [realms]: realm database configuration
 master_key_type: The key type of the master key; the default value is aes256-cts-hmac-sha1-96.
 acl_file: The control file used to specify which users can access the kdc database; if current user access is not required, this value can be set to empty
 dict_file: dictionary file location, words in this file cannot be used for passwords; if the file is empty, or the user is not assigned a policy, a password dictionary check will not be performed.
 admin_keytab: KDC keytab for verification.
 supported_enctypes: The supported encryption method, default is aes256-cts-hmac-sha1-96:normal aes128-cts-hmac-sha1-96:normal.
 For detailed instructions, please refer to the official website document: /kerberos/krb5-latest/doc/admin/conf_files/kdc_conf.html.
3. Create a database
[root@kafka01 ~]# kdb5_util create -s -r

4. Start the service

1. Start the service

#Enable self-start
 [root@kafka01 ~]# systemctl enable
 Created symlink /etc/systemd/system// → /usr/lib/systemd/system/.
 #Enable Kerberos service
 [root@kafka01 ~]# systemctl start

 #Enable kadmin service
 [root@kafka01 ~]# systemctl enable
 Created symlink /etc/systemd/system// → /usr/lib/systemd/system/.
 [root@kafka01 ~]# systemctl start

2. Create an account

  • A variety of managed operations can be performed on the Kerberos service machine. Enter :

    Common operations:

    operate describe example
    add_principal, addprinc, ank Add principal add_principal -rnadkey test@
    delete_principal, delprinc Delete principal delete_principal test@
    modify_principal, modprinc Modify principal modify_principal test@
    rename_principal, renprinc Rename principal rename_principal test@ test2@
    get_principal, getprinc Get principal get_principal test@
    list_principals, listprincs, get_principals, getprincs Show all principals listprincs
    ktadd, xst Export entries to keytab xst -k /root/ test@ 
#Execute the command
 [root@kafka01 ~]#
 : add_principal admin/admin@
 : add_principal kafka-server/kafka01@
 : add_principal kafka-server/kafka02@
 : add_principal kafka-server/kafka03@
 : add_principal kafka-client@


 #Export account key
 : xst -norandkey -k /root/data/ kafka-server/kafka01@
 : xst -norandkey -k /root/data/ kafka-server/kafka02@
 : xst -norandkey -k /root/data/ kafka-server/kafka03@
 : xst -norandkey -k /root/data/ kafka-client@

5. Kerberos Client Installation

Install on other cluster machines

[root@kafka01 ~]#yum install krb5-workstation

1. Configuration

Copy /etc/ from 192.168.10.100 and overwrite the local /etc/.

#The client can use the kadmin command
 [root@kafka01 ~]# kadmin

 kinit (authenticate user on client side)
 [root@kafka02 ~]# kinit admin/admin@ #Enter password to authenticate

 #View the current authenticated user
 [root@kafka01 ~]# klist

 #kdestroy(delete the current authentication cache)
 [root@kafka01 ~]# kdestroy

6. kafka cluster enables kerberos authentication

1. Machine preparation
Serial number IP Host Deployment Services
1 192.168.10.100 kafka01 zookeeper、kafka
2 192.168.10.101 kafka02 zookeeper、kafka
3 192.168.10.102 kafka03 zookeeper、kafka
Bind the host file
[root@kafka01 ~]# cat /etc/hosts
192.168.10.100    kafka01
192.168.10.101    kafka02
192.168.10.102    kafka03
2. Create a keytab file

Enter kadmin on the machine where Kerberos is installed (used on the Kerberos server, kadmin can be used on the machine where Kerberos Client is installed), and then execute the following command to create the keytabs of the server and client respectively:

#Execute the command
 [root@kafka01 ~]#
 : add_principal admin/admin@
 : add_principal kafka-server/kafka01@
 : add_principal kafka-server/kafka02@
 : add_principal kafka-server/kafka03@
 : add_principal kafka-client@

 #Export account key
 : xst -norandkey -k /root/data/ kafka-server/kafka01@
 : xst -norandkey -k /root/data/ kafka-server/kafka02@
 : xst -norandkey -k /root/data/ kafka-server/kafka03@
 : xst -norandkey -k /root/data/ kafka-client@
3. Kerberos related configuration

Copy and keytab files to all machines where Kafka is installed, and put all the files in Kafka's config/kerveros directory (kerberos directory needs to be created new).

[root@kafka01 kerberos]# pwd
/opt/kafka_2.12-3.9.0/config/kerberos
[root@kafka01 kerberos]# ll
total 24
-rw-r--r-- 1 root root  95 Mar 10 15:53 
-rw-r--r-- 1 root root 246 Mar 10 16:11 
-rw------- 1 root root 379 Mar 10 16:03 
-rw-r--r-- 1 root root 256 Mar 10 16:10 
-rw------- 1 root root 424 Mar 10 16:01 
-rw-r--r-- 1 root root 786 Mar 10 16:10
4. Kafka server configuration ()
#Execute the command
 [root@kafka01 config]# vim
 #Configuration file authentication
 =SASL_PLAINTEXT
 =GSSAPI
 =GSSAPI
 =kafka-server
5. Create a new file, and the file is also placed in Kafka's config/kerveros directory.
[root@kafka01 kerberos]# cat
 KafkaServer {
    .Krb5LoginModule required
    useKeyTab=true
    keyTab="/opt/kafka_2.12-3.9.0/config/kerberos/" #This is the exported account keytab file. Different accounts have different files.
    storeKey=true
    useTicketCache=false
    principal="kafka-server/kafka01@"; #Different machines Different accounts,
 };
6. Modify the bin/ script, add the following configuration in the second to last line:
#Enter the startup script
 [root@kafka01 bin]# vim

 #-=false zk sets false without authentication enabled
 export KAFKA_OPTS="-=false -=zk-server -.=/opt/kafka_2.12-3.9.0/config/kerberos/ -=/opt/kafka_2.12-3.9.0/config/kerberos/"

Client configuration

7. Create a new file, and the file is also placed in Kafka's config/kerveros directory.
[root@kafka01 kerberos]# vim
 KafkaClient {
    .Krb5LoginModule required
    useKeyTab=true
    keyTab="/opt/kafka_2.12-3.9.0/config/kerberos/" #Client key
    storeKey=true
    useTicketCache=true
    principal="kafka-client@"; #Client Account The ';' here cannot be omitted
 };

This configuration is mainly used to use bin/, bin/, and other commands

#The second to last line of three files add the following content
 export KAFKA_OPTS="-.=/opt/kafka_2.12-3.9.0/config/kerberos/ -=/opt/kafka_2.12-3.9.0/config/kerberos/"

7. Start the test

#View topic
 [root@kafka01 ~]# sh /opt/kafka_2.12-3.9.0/bin/ --list --bootstrap-server kafka:9092 --command-config /opt/kafka_2.12-3.9.0/config/kerberos/

 #Create topic & test link
 [root@kafka01 ~]# sh /opt/kafka_2.12-3.9.0/bin/ --create --topic test --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092 --command-config /opt/kafka_2.12-3.9.0/config/kerberos/
 #Producer
 [root@kafka01 ~]# sh /opt/kafka_2.12-3.9.0/bin/ --topic test --bootstrap-server :9092 -- /opt/kafka_2.12-3.9.0/config/kerberos/

 #consumer
 [root@kafka01 ~]# sh /opt/kafka_2.12-3.9.0/bin/ --topic test --from-beginning --bootstrap-server :9092 -- /opt/kafka_2.12-3.9.0/config/kerberos/
<Previous Next>