Statistical Process Control Template for TIBCO Spotfire® using TIBCO® Data Science - Workbench

This TIBCO Spotfire template enables the user to build a wide range of Shewhart quality control charts. The chart specifications can be defined interactively or retrieved from a table. This comprehensive template is constructed with a Statistica data function that leverages a wide range of functionality and settings already implemented in TIBCO Data Science - Statistica. It is an example of a no-code data function.

Compatible Products

TIBCO Spotfire® TIBCO® Data Science – Workbench TIBCO® Data Science

Provider

TIBCO Software

Compatible Versions

TIBCO Spotfire 10.10, 11.0, 11.1, 11.4 (not supported in 11.2 and 11.3)

Together with extensions:

  • TIBCO Data Science for TIBCO Spotfire Analyst Software (version 13.6 and 14.0), for Spotfire 11.4, hotfix V140HF02-Spotfire is needed
  • TIBCO Data Science Service for TIBCO Spotfire (version 13.6 and 14.0), for Spotfire 11.4, hotfix V140HF02-Spotfire is needed - needed in case of web browser usage

License

TIBCO Component Exchange License

Overview

The basic building block of this template is the quality control chart. A control chart is used to monitor a process variable over time. The objective in monitoring is to determine if the process is in control or out of control. This helps to ensure that the process is stable over time and to minimize it’s output variability.  It can be applied in different fields like manufacturing, telco, energy, marketing, sales, healthcare, etc.

This template provides various quality control charts and tests relevant for process control:  you can construct charts for monitoring the process mean as well as variance, you can create charts for individuals or X-bar charts. Here is the list of charts: X type charts for individuals, X-bar, EWMA, Moving average, Cusum, R/S charts and histograms.  They can include central lines, control, warning as well as specification limits.

Points outside of limits are not the only way to identify atypical process behavior. Other tests implemented in this template are Runs Tests which identify a set of consecutive points that are not behaving randomly (e.g. 6 consecutive points with increasing or decreasing trend). All Rules Tests are configurable from inside the dashboard.

Implementation

This component consists of two applications.

The first one - Application 1 - is an interactive dashboard application with extensive manual options to configure charts.  The user can interactively change almost every setting using document properties and parameters. Additional capabilities implemented in this application are Brushing (recalculating charts with excluded samples) and assigning causes, actions and comments. This application is a good place to start to define and experiment with various chart types, rules and configurations.

If the user wants to investigate in an automated way a larger number of parameters with predefined configurations for quality control chart types and alarms, Application 2 can be used. All the definitions for each parameter are driven by one configuration line inside a configuration file. To run quality control charts for your data, all you need is to upload your configuration file and the measurement data. 

This template is working at the moment for English locale.

Information on a comprehensive Process Control Monitoring and Alerting Solution, utilizing and extending capabilities of this template, can be found here

Release(s)

Release v2.1.0

Published: September 2021

This release adds some enhancements to previous release 2.0.0 of the template. Changes to previous version:

  • Selection condition parameter enabled for Application 2.
  • Fixing the not covered situation: in case there is only one sided limit defined from the variable (means not calculated) the alarm is now triggered for this situation. 
  • Data type for sigma multipliers in the configuration file  (Application 2) changed to real numbers instead of integers. 
  • Extended description inside the configuration file documentation (Application 2).
  • Redundant IronPython scripts has been removed from the dxp.

Release includes:

  • Two Spotfire applications for the quality control chart calculation and visualization 
  • Documentation
  • License information

Release v2.0.0

Release v1.0.0

Process Control Monitoring and Alerting Solution Overview

Introduction 

We’d like to introduce a new solution - one that easily scales to monitor large numbers of processes and parameters and produces automated alerts when anomalies are detected.  It employs univariate Statistical Process Control methods and is an important Quality Control tool.  This method is used to monitor, control and improve any measured process.  The focus is on reducing variability to improve quality.

View a demo of the solution:

 

A number of versions of this solution are available and can be downloaded from the TIBCO Exchange here.  

Control Charts:  Brief History and Importance in Industry

Control Charts have played a vital role in the evolution and success of the global manufacturing industry.  The control chart was first developed in 1924 by Walter Shewhart at Western Electric.  It was initially used to improve the reliability and audio quality of the candlestick telephone.  It was then used during WWII by the US Army, but was not adopted by US industry after the war.  However, it was introduced to Japanese industry and became an important quality tool that helped Toyota, Sony and others capture the global auto and electronics industries.  This contributed to the 1980s US Quality crisis, resulting in the re-introduction of statistical process control methods in the US.  It was first championed by Ford, with their Quality is Job 1 initiative, and Motorola, with 6 Sigma. 

Today, it is widely used across the globe and considered an essential tool in the mass production of manufactured goods.  It’s usage is growing in many other sectors, as well, such as Energy, Healthcare, and Telco.             

Solution Overview

This solution is quite comprehensive – many types of standard process control charts, alerting rules and charting options are supported.  While rich in features, it is designed to be easy to configure for large numbers of parameters. 

The solution consists of two components.  The end-user visualizes and interacts with all results in Spotfire dashboards.  TIBCO Data Science is the underlying backend computation engine, where all the QC charts are computed on a scheduled basis. Together with computation, in the backend, there is also automated batch monitoring and alerting.  Hundreds to thousands of parameters can be monitored.  Once an alert is triggered an e-mail can be sent.  Alerts can also be investigated interactively in the frontend Spotfire dashboard application.

Click on the image to view it fullscreen.  Use browser back button to return to this page.

Spotfire dashboard summary page example

Here we are monitoring multiple parameters separately for product A and product B

In the upper left corner, we see the current unacknowledged alarms.   This tells us where there are problems, that triggered alarm notifications, but have not yet been addressed.  When we have many parameters, we need some indication about which charts currently show out-of-control or rules violations.   On the bottom left, the heatmaps show all recent alarms including those that have already been acknowledged.  In the application we are looking at two types of violations:

  • out of control (OOC) = sample point is outside the control limits on the QC chart
  • runs rules violations indicate systemic problems even if samples are not yet out of control.  For example several samples in a row have increasing values or several samples are on the same side of the center line

On the right, there is a summary of the monitored parameters.  Spark lines show when the alarms took place.  The Application is interactive, so if we want to look at a current alarm, we can mark a parameter and see it marked in all other visualisations on the page.

The purpose of this page is to see the overall situation and to help pick the parameter we want to investigate further. When we mark it here, the backend engine makes all calculations required to display that chart.

Click on the image to view it fullscreen.  Use browser back button to return to this page.

Control Charts

On the Chart pages, QC charts for the selected parameter can be seen. This is an X-bar chart and S (variance) chart. Points that contributed to runs rules violations are highlighted in yellow.  On other pages we can see more detailed info for the X-chart, variance charts or runs violations. 

Click on the image to view it fullscreen.  Use browser back button to return to this page.

On On the Runs Test page, the tests and violations are shown.  

Click on the image to view it fullscreen.  Use browser back button to return to this page.

The configuration file

In this solution every parameter can have its own chart configuration.  The configuration includes:

  • Chart type: X-bar R/S, MA (moving average), Exponentially Weighted Moving Average (EWMA) or Cusum
  • Control or warning limits display can be based on calculations from the data or predefined values you provide
  • Spec limits display
  • Runs rules to be used
  • other configurations are defined in the solution documentation  

A configuration table defines how the charts and alarms are configured. A few rows and columns of the configuration table are shown at the top of the screenshot below.  Under the hood there is a parameterized data function calling TIBCO Data Science –Workbench (the backend computation engine) calculating charts and out of control points based on values in the highlighted row in the configuration file.

With this approach, you only need to maintain the configuration file and everything else is handled automatically in the dashboards and calculation engine.

Click on the image to view it fullscreen.  Use browser back button to return to this page.

The Computation Engine

Computations are done using visual workflows, leveraging the same configuration file for computing limits, out of control and rules violations that is used by the Spotfire data function. There is a loop implemented, shown below, that will run QC charts for all the parameters in the configuration file. This simplifies the task of implementing hundreds of QC charts. Inside the loop the results are saved into a database, so they can be picked up by the Dashboard application at any time.

Click on this image to view full screen.  Click your browser back arrow to return to this page.

Click on the image to view it fullscreen.  Use browser back button to return to this page.

In addition, the monitoring and alerting server is leveraged. It runs at scheduled intervals to determine if there are any alerts generated. It will also send email notifications in case of alerts.

This solution enables the user to review the state of alerts, investigate any parameter interactively and acknowledge the alarms. The essential component for simple implementation is one configuration file for all QC charts. TIBCO Data Science workbench has a data entry functionality that can be used for governed insertion and modification of current entries in the data configuration file, if needed.

Summary

This process control monitoring and alerting solution features hierarchical monitoring and alerting based on quality control charts.  The Configuration Table makes it easy to set up for large numbers of parameters.  Many types of charts and rules are supported.  Because the Dashboards are in Spotfire … and TIBCO Data Science is the underlying computation engine, the solution is easily modified, customized and extended. 

A number of versions of this solution are available and can be downloaded from the TIBCO Exchange here.  

If you are interested in implementing this solution, including capabilities shown here but not yet available on the Exchange, please contact us at askdatascience@tibco.com.

Advanced Solutions

This solution involves batch processing of univariate statistical process control charts.  While this is an essential first step in any process control program, it is really only a starting point.  It is the most fundamantal form of anomaly detection. 

For many applications, more advanced anomaly detection methods are required. 

  • For complex processes and products, multivariate methods such as multivariate statistical process control or AI-based anomaly detection may be required to detect more subtle anomalies that degrade product quality and reliability. 
  • If you need to detect problems as soon as they occur, in order to take immediate corrective action, you may need 'real-time' processing on streaming data or at the edge
  • Advanced Process Control solutions involve predicting the output of a process, while it is still 'in-flight', and adjusting the process recipe to hit the desired target. This may be necessary to reduce the variability of processes that are not capable (have low CpKs).
  • Detecting anomalous time-series traces coming from connected sensors requires specialized data connection and pattern detection algorithms. 

TIBCO works with customers to develop and deploy advanced solutions in all of these areas.  For more information about implementing one of these solutions, contact us at askdatascience@tibco.com.

View the Wiki Page