Anomaly Detection and Root cause analysis using TIBCO Analytics and Microsoft Cognitive Services

This offering consists of custom mod for TIBCO Data Science - Team Studio and python data functions for TIBCO Spotfire to detect anomalies using Microsoft Azure services and containerized microsoft cognitive services to perform root cause analysis

Compatible Products

TIBCO Spotfire® TIBCO® Data Science Team Studio

Provider

TIBCO Software

Compatible Versions

TIBCO Data Science - Team Studio 6.5
TIBCO Spotfire 10.7 and above

License

TIBCO Component Exchange License

Overview

Based on TIBCO Spotfire and TIBCO Data Science, along with Azure Cognitive Services, the TIBCO Anomaly Detection solution detects and analyzes anomalies—sudden changes in data patterns, discovers  root causes and provides suggested actions. Manufacturing, energy, mining and power plant customers use TIBCO Data Science to detect anomalies on historical data from various facilities at scale.

TIBCO Spotfire calls the Microsoft Containerized Cognitive services using a Spotfire data function via a python API. As the maintenance engineer detects anomalies onsite, Spotfire’s brush-linked visual analytics and data science tooling runs root cause anomaly analysis. These analyses are combined with results from the Text Analytics containerized service using key phrase extraction to determine the recommended actions to be taken
 

Release(s)

Release 1.0.0

Published: May 2020

Initial Release

Includes:

  • data function for using in TIBCO Spotfire
  • custom operator for usage in TIBCO Data Science - Team Studio
  • documentation
  • license file

There are currently no reviews for this content.

Why not be the first to review it - click here to login

Anomaly Detection and Root cause analysis using TIBCO Analytics and Microsoft Cognitive Services

The TIBCO anomaly detection solution includes Microsoft Cognitive Services container deployment with anomaly detection, text mining and root cause analysis. Anomaly detection and analysis provides value across nearly every industry including energy, financial fraud and risk, algorithmic insurance, connected vehicles, healthcare and insurance claims, and manufacturing fault detection and yield optimization.  

This article focuses on energy, identifying anomalies for asset management. This specific example uses machine learning techniques to detect anomalies, understand root cause from related text data, and alert case managers when sensor readings are deviating from expected patterns by recommending a suggested maintenance action. This enables operators to implement predictive-based maintenance actions before equipment failure, and to prevent costly manufacturing process shutdown. 

 The solution is built out in 3 phases; 

Phase 1 - Anomaly Detection on historical data

TIBCO Data Science platform is used for detecting anomalies across all the sites of a power plant. The input file is power plant data consisting of a timestamp column and several other numeric sensor readings. The response variable in this case is ‘Prodperminute’(sensor reading tracking production of power per minute). The workflow consists of multiple steps including data pre-processing, transformation of time-series data, filtering and ultimately calling the MS Azure services from a TIBCO Data Science custom mod designed to invoke the services. The result is an sbdf file(inferred data) which the maintenance engineer uses to carry along to the remote site.

 

Phase 2 – Detecting Anomalies at remote site

Once the maintenance engineer is at the remote site that may not be connected to the internet, the first step is to perform anomaly detection analysis using the Spotfire equipment maintenance dashboard. 

The user selects a response variable(in the demo: ‘Prodperminute)’ anda time granularity. Anomalies are then detected using historical data collected at the particular site. This produces two visualizations, the original data readings along with the expected values provided from the containerized service, and the difference between the original and expected values over time. The red markers indicate the anomalies detected by the Anomaly Detection container.

 Phase 3 – Performing root cause analysis

The site engineer investigates anomalies from a certain time window to perform a root cause analysis. As a part of root cause analysis, key driving factors indicate what factors are contributing to the anomalies in production per minute for the time window selected by an engineer using the TIBCO Spotfire’s brush linking capabilities.

At this point, the site engineer digs deeper to understand the maintenance action items performed prior to these anomalies occurring by getting insights from log data using key phrase extraction from the Text Analytics container. 

From here, the equipment maintenance dashboard generates  a recommendation for next steps by further pre-processing the key phrases extracted from the logs. In the example shown in the demo, the maintenance logs from the selected time window indicate ‘BaroPressure’ sensor was restarted 5 times vs. being recommended for replacement just once. As this sensor was one of the top key driving factors, recommendation to replace the ‘BaroPressure’ sensor is made.

To learn more about the solution visit;

Microsoft Build’20 on demand recording/live demo: AI Show: Bringing AI to Edge

Partnership between TIBCO and Microsoft: Customer Case Study

View the Wiki Page