Table of Contents
TIBCO Hawk® provides the industry's best, most sophisticated tool for monitoring and managing distributed applications and systems throughout the enterprise. With Hawk, system administrators can monitor application parameters, behavior, and loading activities for all nodes in a local or wide-area network and take action when pre-defined conditions occur. In many cases, runtime failures or slowdowns can be repaired automatically within seconds of their discovery, reducing unscheduled outages and slowdowns of critical business systems.
Hawk 6.1.0 introduces the Universal Collector microagent which is an enhanced version of the Logfile microagent. Using it, you can add log collectors and log forwarders to send logs and rulebase data to LogLogic Log Management Intelligence (LMI) for further analysis.
Hawk is traditionally used for monitoring instantaneous data, without any data storage, historical views, data exploration, and analytics. This microagent is a step towards storing Hawk data and analyzing the data. Loglogic LMI provides an end-to-end machine and application log data management solution that helps organizations collect, store and analyze machine data to gain operational insights. You can learn more about LogLogic LMI here.
Problem statement: As an infrastructure operator/administrator, how do I use the rich monitoring data captured by Hawk agents and visualize it in LMI?
This article will walk you through how to configure collectors and forwarders to send Hawk rulebase data to LogLogic LMI. It assumes that you have a basic understanding of Hawk and LMI functionality and features.
Prerequisite: This article requires you to install and configure the Universal Collector microagent in the Hawk plug-ins directory. Refer this doc (page 62) for more details: https://docs.tibco.com/pub/hawk/6.1.0/doc/pdf/TIB_hawk_6.1_plugin_reference.pdf
The communication from Universal Collector Microagent to Loglogic LMI happens over Universal Lossless Data Protocol (ULDP), which is a proprietary protocol. ULDP supports logs forwarding via syslog over TCP or UDP. ULDP is also available as a client library, easily embedded in any application.
Once you have enabled the Universal Collector Microagent it will show up in your agent?s microagent tab as follows:
The various methods of this microagent are:
- addForwarder: Adds ULDP (LMI) or TCPsyslog based data forwarder configuration.
- addLogFileCollector: Configures the log file whose logs need to be forwarded
- addRulebaseDataCollector: Configured Hawk rulebases whose data needs to be forwarded
- getCollector: gets list of configured collectors
- getForwarder: gets list of configured forwarders
- removeCollector: removes a particular collector configuration at a time
- removeForwarder: removes a particular forwarder configuration at a time
- sendMessage: send custom alert messages through the forwarder
You can now follow the following steps to send monitoring data to LMI:
1. Use the addForwarder method to configure an LMI instance you want to send Hawk data to. (see screenshot below)
- Type - ULDP
- Host - IP of the LMI instance
- Port - you can leave this blank. The default port is 5516
- Use TLS - if you want secure data transfer then you can enable this and configure rest of the properties. This article does not use TLS.
The Forwarder configuration is successful if you see the following message. If the LMI host IP you entered is wrong ?failure? message will show up instead.
2. You can also see the list of configured Forwarders (LMI hosts or other syslog forwarders) using the getForwarder method:
3. The removeForwarder method lets you remove Forwarders. You can provide one of the keys from getForwarder method?s output and use it to remove the Forwarder.
Beware though - once you remove a forwarder which is being used by a collector, that collector will stop sending data to the configured LMI/other receivers.
4. Because we want to send the metrics/monitoring data that Hawk is capturing to the LMI, I?ll use the addRulebaseDataCollector method.
This requires that you know the rulebase name for which you want to send data to LMI.
On successful configuration of the Collector, you will see the following message:
This collector will send data to *all* forwarders that you have currently configured. The collection interval will depend on the rulebase interval that has been defined for the particular Rulebase.
5. You can also see the list of configured Collectors (LogFile or Rulebase) using the getCollector method:
This also gives the count of messages for the particular collector.
Once you remove the Collector, the Forwarder will stop sending the data to LMI/other receivers.
7. When the successful Forwarder and Collector configurations are done, you can now log into your LMI web UI and see all the monitoring data sent by Hawk.
8. Now you can do advanced data modeling and custom dashboarding on all the monitoring data to get operational insights.
As you can see one can monitor process stats like CPU, memory, threads etc from the monitoring data gathered by LMI.
For advanced data modeling in LMI please see: https://docs.tibco.com/pub/loglmi/6.2.1/doc/html/GUID-20A0508D-EC40-4946-BCD4-621E2C909E06.html