How to Configure Apache Kafka® with Kerberos and Microsoft Active Directory for Authentication

The purpose of this document is to provide information on configuring Apache Kafka and Zookeeper on Linux to use Microsoft Active Directory (AD) for authentication. Active Directory will run on a Windows Server.

The documentation will provide a simple setup of the Simple Authentication and Security Layer (SASL) setup of Zookeeper/Kafka Broker. The configuration can then be expanded to support TLS and multiple brokers if desired (not documented).

The document will outline:

• Create an AD user for both Zookeeper and Kafka on the Windows Server
• Set a Service Principal Name (SPN) to be used with AD and Kerberos
• Create a Kerberos keytab file for Zookeeper and Kafka
• Secure the keytab file
• Configure Kerberos on Linux
• Configure a single Zookeeper and Kafka Broker to use the keytab file for Kerberos authentication
• Connect Zookeeper and Kafka using AD Authentication

Prerequistes:

• RedHat 8 (or equivalent) must be running and configured with Apache Kafka 2.8 installed. The TIBCO download of Apache Kafka was used, but the opensource version from Apache will also work.
• Kerberos 5 installed on the Linux server. This can be a full Kerberos installation of just the workstation version. ? sudo yum install krb5-workstation.
• Windows 2022 Server with Active Directory install/configured. There are a number of sites on the internet which can be used to install and configure Active Directory on the Windows 2022 server. Windows 2019 can also be used.
• ALL servers, Linux and Windows, must have a fully qualified DNS name (FQDN) that must be resolvable by all servers. This can be done using a DNS Server, or properly configuring /etc/hosts on both servers.

how_to_configure_kafka_with_kerberos_and_active_directory.pdf