TIBCO® Messaging - Apache Kafka Distribution Extended Monitoring

Table of Contents

• New Statistics
• New Apache Kafka Statistics
• New Apache ZooKeeper Statistics
• Installation
• Configuration

TIBCO® Messaging - Apache Kafka Distribution - FTL Bridge Release 2.0.0 added the capability to collect and display a wide range of monitoring statistics to keep an eye on your Apache Kafka clusters and clients.  With the information and extended monitoring template in this article, you'll be able to extend your monitoring to include more Apache Kafka statistics as well as adding monitoring of your Apahe ZooKeeper installation, all using the same InfluxDB database and Grafana-based dashboards you've already deployed.

New Statistics

The tables below cover the new statistics collected, their attributes, how they are stored within InfluxDB, and any extra information related to that statistic.

New Apache Kafka Statistics

New Apache ZooKeeper Statistics

Installation

To use these new capabilities, you will need to replace two files in your Apache Kafka Distribution - FTL Bridge installation; the first adds the new statistics to the JSON template file and the other adds additional configuration fields to the .csv file.  Both of these are used to generate the monitoring configuration file that drives the entire statistics collection and archiving process. 

Regardless of your platform, you'll need to know the directory where the Apache Kafka Distribution - FTL Bridge was installed.  By default, for Linux and Mac this directory will be /opt/tibco/akd/bridge/2.0 (where 2.0 is the version number and uld change as new versions are released), for Windows this directory will be C:\tibco\akd\bridge\2.0.  If you installed in a non-default directory, change any paths specified below to reflect your actual installation directory. 

To add the new statistics, copy the tibkmon_Template.json and tibkmon_config.csv files attached to this article into the monitoring/tibkmon_JSON_generator directory of your Apache Kafka Distribution - FTL Bridge installation.

Configuration

There are two steps needed to complete the configuration and get you ready to collect the new statistics.  First, edit the tibkmon_config.csv file and add the appropriate information for accessing your ZooKeeper installation - this covers the same settings that you've already configured for the Apache Kafka brokers such as JMX host, port, user name, and password (just with teh appropriate settings for your ZooKeeper installation which might or might not be the same depending on your overall system configuration).  When you are finished, save this file and continue to the next step.

If there are specific questions you would like to see answered or new topics covered, please add a comment because that will help us prioritize future content.