Introducing the all-new TIBCO Community site!

For current users, please click "Sign In" to reset your password and access the enhanced features. If you're a first-time visitor, we extend a warm welcome—click "Sign Up" to become a part of the TIBCO Community!

If you're seeking alternative community sites, explore ibi, Jaspersoft, and Spotfire.

Jump to content
  • Hawk RedTail Container Edition 7.0.1 Kubernetes Deployment - Quickest Deployment with minimal Security

    Manoj Chaurasia

    Table of Contents

    TIBCO Operational Intelligence Hawk® (Formerly TIBCO Hawk®) and TIBCO Operational Intelligence Hawk® RedTail provide the industry's best, most sophisticated tool for monitoring and managing distributed applications and systems throughout the enterprise. With Hawk, system administrators can monitor application parameters, behavior, and loading activities for all nodes in a local or wide-area network and take action when pre-defined conditions occur. In many cases, runtime failures or slowdowns can be repaired automatically within seconds of their discovery, reducing unscheduled outages and slowdowns of critical business systems.

    Overview of TIBCO Hawk RedTail Deployment 

    TIBCO OI Hawk RedTail - Container Edition helps you to monitor and manage distributed applications by providing distributed rules, alerts, and remediation actions. You can enable additional monitoring capabilities in TIBCO OI Hawk RedTail - Container Edition with the ?TIBCO Operational Intelligence Hawk® RedTail - Standard Edition License?, such as centralized storage and processing of metrics, customizable dashboards, Tag-Based Rulebases, Fine-grained Role-Based Access Control, Collect and Forward logs to TIBCO LogLogic LMI, etc.

    OI Hawk RedTail Container Edition deployment consists of a set of microservices where each component provides internal/ external services and interacts with other components over HTTP/REST/ JDBC.

    (Refer to TIBCO® Operational Intelligence Hawk® RedTail -Container Edition 7.0.1 Installation, Configuration, and Administration section for a description of the components.)

    The components Hawk Console, Webapp, and QueryNode expose REST APIs over HTTPS and so certificates and keys must be configured for these components including the client certificates and keys for other components which interact with these 3 components.

    Note that OI Hawk RedTail consists of multiple components. All these components talk to each other in a secure manner and for most of the components, security is not optional. The security is enforced using server and client certificate validation. The certificates, and keys can be configured for each component using environment variables. As a convenience, self-signed certificates and keys are auto-generated at the time of the creation of RedTail Container images and placed within the containers at a certain location. You can use these self-signed certificates and keys or choose to use more secured CA-signed certificates.

    Another configuration aspect for OI Hawk RedTail is the storage requirements for some of the components. You will need to create Persistent Volume Claims (PVCs) and mount the volumes for persisting data for components like Prometheus, Grafana, MySQL Database, Hawk Console, etc.

    With the above-mentioned considerations, Kubernetes deployment configuration for RedTail components can be complex and requires careful planning and execution.

    This document describes Kubernetes deployment of RedTail Container Edition components with the bare minimum secure communication.

    TIBCO OI Hawk RedTail Container Edition provides the ? script to build Docker images for each of the components. The script also generates the necessary certificates and keys at the time of building the images. These are self-signed certificates, you can change the certificate validity and password by configuring the details in cert.conf file.

    Refer to TIBCO® Operational Intelligence Hawk® RedTail - Container Edition Security Guidelines Certification Management section for more information.

    The certificates and keys are placed into each of the container images at the location: /loglogic/conf/certs/


    1. The steps mentioned in this document are only for reference purposes. Your actual production deployment may vary depending on your corporate security requirements.
    2. The sample YML files attached to this document are for reference only. You may compose your own YML/ Helm charts manifests for your production deployment.

    RedTail Kubernetes Deployment with Minimal Security

    We will use the default self-signed server and client certificates which are auto-generated and available at each of the container images.

    Note that Hawk Console, WebApp, and Query Node offer public-facing REST APIs, so access to these components must be secured. At a minimum, we need to configure these 3 components and other components which interact with these using certs and keys via environment variables in their respective YML files.

    Step 1: Download Hawk RedTail and build images

    Download TIBCO OI Hawk RedTail Container Edition 7.0.1 binaries ( Extract it (refer to it as <OIHRCE_HOME>) to find the following directory structure:

     drwx------  12 ntamhankar  staff  384 Jun  5 22:26 . drwxr-xr-x  10 ntamhankar  staff  320 Jun  6 11:10 .. drwxr-xr-x@  4 ntamhankar  staff  128 Jan 29 04:33 docker drwxr-xr-x@  5 ntamhankar  staff  160 Jun  8 16:36 hawkconsolenode drwxr-xr-x@  8 ntamhankar  staff  256 Jun  8 16:36 hkceagent drwxr-xr-x@  6 ntamhankar  staff  192 Jun  8 16:33 mysql-connector drwxr-xr-x@  5 ntamhankar  staff  160 Jun  8 16:35 querynode drwxr-xr-x@  5 ntamhankar  staff  160 Jun  8 16:36 redtail-grafana-plugin drwxr-xr-x@  3 ntamhankar  staff   96 Jan 29 04:33 redtail-prometheus-discoveryservice drwxr-xr-x@  4 ntamhankar  staff  128 Jan 29 04:33 samples drwxr-xr-x@  9 ntamhankar  staff  288 Jun  8 16:36 webapp drwxr-xr-x@  5 ntamhankar  staff  160 Jan 29 04:33 zookeeper


    Go to <OIHRCE_HOME>/mysql-connector and download the mysql connector from the publicly available MySQL portal:

     $ wget


    Now go to <OIHRCE_HOME>/docker/build-images and run:

     $ ./


    The script will prompt you if you want to proceed with the default TLS key password. The default password is ?changeit?. Type yes and proceed. You can change the default password and other configurations. However, if you change the password, then you need to change it in the sample YML files provided.

    You will notice the certificates and keys will be generated and copied into the images as they are being created.

    This will create all the images for various RedTail components.


     $ docker images | grep redtail redtail/zookeeper                                7.0.1     d8f445572e61   5 hours ago   254MB redtail/prometheus_discoveryservice              7.0.1     3f331a20aa7e   7 hours ago   18.6MB redtail/grafana                                  7.0.1     201289c3812b   7 hours ago   159MB redtail/hkceagent                                7.0.1     b6bf5cf65c1a   7 hours ago   454MB redtail/hawkconsolenode                          7.0.1     b771f39f86e1   7 hours ago   370MB redtail/webapp                                   7.0.1     15a86460c839   7 hours ago   239MB redtail/querynode                                7.0.1     002c10fd574a   7 hours ago   356MB redtail/base                                     7.0.1     b6c1a8ef3ff8   7 hours ago   267MB redtail/mysql                                    7.0.1     3717ea8b7df9   7 hours ago   447MB 



    1. build-all script does not build the Prometheus image.  This is because we refer to the official prom/prometheus:v2.22.0 image, we do not modify this image, however, we configure Prometheus to securely connect to Hawk Console for the targets using the certificates and keys configured using ConfigMap. More about this in Step 3.

    2. You may need to tag and push these images to the container registry of your preferred Kubernetes platform. The sample YML files attached here refer to Google Cloud Container Registry, so the image tag needs to be of the format:<GCP_Project>/<folder>/component_name:7.0.1


    Step 2: Identify the certificates and keys

    Though all the self-signed server and client certificates and keys are present in every RedTail component image, they need not be explicitly configured as environment variables for each of the components. The only exception is the Prometheus component. The Prometheus image is not customized, rather a standard docker image is used prom/prometheus:v2.22.0.

    Since Prometheus needs to connect to the Hawk Console component as a secure target for pulling the metrics, it needs to initialize the target with the Hawk Console server certificate, Prometheus client key, and certificate.

    This can be done by configuring the cacert, client certificate, and key as ?Secrets? and volume mounting them to the Prometheus container.

    In order to configure the secrets, we need to get the actual cacert, Prometheus client certificate, and key and use the Base64 encoded content of these files.

    Note that all the certificates and keys are available within <OIHRCE_HOME>.

     $ cd <OIHRCE_HOME>/docker/build-images/build-context/loglogic/conf/certs $ base64 -i cacert -o cacert-base64.txt $ base64 -i prometheus-client-certificate -o prometheus-client-cert-base64.txt $ base64 -i prometheus-client-key -o prometheus-client-key-base64.txt 


    These base64 encoded content will be useful in configuring redtail_prometheus.yaml file in the step below.

    Step 3: Deploy RedTail components

    We are all set to start deploying each of the YAML files one by one. Download all the YAML files attached. You will have to change the image name as per your Kubernetes requirements.

    Make sure your kubectl context points to the right cluster you want to deploy RedTail.

    1. Deploy Zookeeper component:

     $kubectl apply -f redtail_zk.yaml


    This will create a couple of services and a stateful set for Zookeeper. This will also create a Persistent Volume Claim and a persistent volume of 50 MB for persisting the zookeeper data directory.

    The startup scripts in the container will create necessary nodes on the Zookeeper which is required for RedTail functioning.

    Note: Access to zookeeper is unsecured. And you cannot configure secured access to Zookeeper.



    2. Deploy MySQL component:

     $kubectl apply -f redtail_db.yaml


    This will create the necessary tables and populate them.


    Note: Access to MySQL is not secured. You can configure TLS for MySQL access and configure client certificates for other components which communicate with MySQL.



    3. Deploy Grafana component:

     $kubectl apply -f redtail_grafana.yaml


    This will deploy the customized grafana component which provides the dashboarding features. Grafana also connects to MySQL DB to store its configuration.


    Note: By default, Grafana is unsecured. You can protect the access to Grafana by using a side-car reverse proxy such as nginx and configuring secure communication for nginx.

    4. Deploy Hawk Console component:

     $kubectl apply -f redtail_console.yaml


    Hawk Console is the pivotal component of RedTail. It provides Authentication, connectivity to Hawk Agent, RBAC, and other features such as public-facing REST APIs. These REST APIs are secured.

    5. Deploy Hawk QueryNode component:

     $kubectl apply -f redtail_querynode.yaml


    QueryNode component is responsible for providing query capability to Hawk microagent methods as well as Time-series metrics stored in Prometheus. QueryNode provides public-facing REST APIs. These REST APIs are secured.

    6. Deploy Hawk Agent component:

     $kubectl apply -f redtail_hkceagent.yaml


    A single Hawk Agent is required for monitoring all the Applications and infrastructure components in the Kubernetes cluster. Hawk Agent connects to the Hawk Console using Console API via TCP transport and also connects to the microagents (Applications) using the TCP AMI API. Note that both the Console API and AMI (Microagent) API can be configured with secured communication using TLS. The sample YML files do not configure the secured transport communication.

    In addition, the Hawk Console also scrapes from various Prometheus targets and facilitates to collects metrics. It also exposes metrics for the Hawk Microagent methods to be collected as time series data.

    7. Deploy Prometheus component:

    Note: This needs an edit in redtail_prometheus.yaml file. Copy the base64 encoded content for cacert, Prometheus client certificate, and Prometheus client key from Step 2 above.

    Replace the values at the three places in the secret section of redtail_prometheus.yaml file:

     apiVersion: v1 kind: Secret metadata:   name: prometheus-hkc-certs type: Opaque data:   hkc-cacert: <base64_encoded_value_of_cacert>   prom-certificate: <base64_encoded_value_of_prometheus_client_certificate>   prom-key: <base64_encoded_value_of_prometheus_client_key>


    Save redtail_prometheus file.

     $kubectl apply -f redtail_prometheus.yaml


    The Prometheus server with help of the side-car container for service discovery will start collecting metrics from the targets added by the Hawk Agent.

    Note: By default, Prometheus is unsecured. You can protect the access to Prometheus by using a side-car reverse proxy such as nginx and configuring secure communication for nginx.



    8. Deploy Webapp component:

     $kubectl apply -f redtail_webapp.yaml


    Webapp is responsible for providing the UI for the entire RedTail.

    9. Finally, create Load Balancers:

    Create Load Balancers for Webapp, Hawk Console, and Query node. This is required so that the Ui and REST API Swagger pages can be accessed from outside:

     $ kubectl apply -f redtail_webapp_lb.yaml  $ kubectl apply -f redtail_console_lb.yaml  $ kubectl apply -f redtail_querynode_lb.yaml


    Step 4: Check the Kubernetes services running and access the UI

     $ kubectl get svc NAME                  TYPE         CLUSTER-IP     EXTERNAL-IP      PORT(S)                 AGE redtail-agent         ClusterIP    None           <none>           2571/TCP                143m redtail-console-cmgr     ClusterIP    None           <none>           2561/TCP             143m redtail-console-connect ClusterIP      None         <none>           9687/TCP              143m redtail-console-rest    LoadBalancer   9687:32303/TCP        5h23m redtail-db              ClusterIP    None         <none>           3306/TCP                143m redtail-grafana         ClusterIP <none>           3000/TCP                143m redtail-prometheus      NodePort   <none>           9090:30244/TCP          143m redtail-querynode       LoadBalancer    9681:31695/TCP          5h23m redtail-webapp          LoadBalancer     9680:31773/TCP        5h23m redtail-zk              ClusterIP    None           <none>           <none>                143m redtail-zk-client       ClusterIP   <none>           9600/TCP              143m 


    Access the RedTail UI:


    (Username/ Password: admin/ admin)


    When you log in for the first time, the UI will show the classic Hawk. Go to Administration ? About and change the license to RedTail standard. On login, you will see the full view.

    Go to Dashboards ? Grafana settings and configure the RedTail plugin with the correct Webapp URL. For e.g. if you are using the sample YAML files, then the same webapp service is ?redtail-webapp?. So the RedTail data source URL should be ("https://redtail-webapp:9680"). Save and test the configuration



    Access the Swagger page for Hawk Console:


    Access the Swagger page for Query Node:


    User Feedback

    Recommended Comments

    There are no comments to display.

  • Create New...