Machine Learning with TIBCO Connected Intelligence
Last updated:
3:10am Jun 04, 2018

                            


Overview

Machine learning is a method of data analysis that automates analytical model building, enabled by recent increases in compute power available to the average user.  Powered by algorithms that iteratively learn from data, machine learning allows computers to find hidden insights without being explicitly programmed where to look.

Machine Learning algorithms are an evolution over traditional algorithms. They can utilize Big data to identify complex patterns, make highly accurate predictions and find optimal solutions. They are well suited to use cases such as micro-segmentation, personalization, root cause analysis of complex processes, fraud detection, resource optimization, etc.

  • Machine Learning 101 overview presentation - Topics:  ♦ What is Machine Learning?  ♦ Decision Trees  ♦ Customer Analytics, Manufacturing and Fraud examples

Machine Learning Algorithms

Machine learning methods are available in TIBCO Statistica, TIBCO Spotfire Data Science and TIBCO Spotfire® and TIBCO® Enterprise Runtime for R as workflow and point and click data functions and analysis templates. Data scientists have access to the underlying R code and can extend the data function collection. These functions can be deployed to Streambase for real-time processing.  Machine learning functions can be shared with the user community for easy reuse.

We focus on 3 classes of machine learning algorithms

  • Supervised Learning – Solving defined problems, predicting some response based on predictor variables

    • Examples of Questions answered: What factors are driving manufacturing defects or fraud or customer behavior?  Can we build a model to predict future occurences based on historical data?
    • Algorithms:  Decision Trees, Random Forest, Gradient Boosting Machine (gbm), Linear and logistic regression, Generalized Additive Models
  • Unsupervised Learning – Identifying new patterns, Detecting anomalies
    • Examples of Questions answered: Are there new failure modes or crime clusters or patterns of customer behavior emerging?  Can we identify outliers in this data?
    • Algorithms:  Clustering, Principle Components, Neural Networks, Support Vector Machines
  • Optimization – Supporting and automating Decision-making
    • Examples of Questions answered:   What is the optimum scheduling and utilization of people or equipment?
    • Algorithms:  Genetic Algorithm

Edge Machine Learning Use Cases with TIBCO Flogo

Here is a overview and IoT related Machine Learning demonstration of TIBCO's Project Flogo: an open source framework for building microservices and functions that run on serverless cloud, IoT, or edge computing infrastructure. For more information see the Flogo TIBCO Community section and http://www.flogo.io/

Streaming Analytics & Machine Learning

TIBCO StreamBase® has multiple ways to integrate with machine learning to utilize models discovered during the real-time processing of streaming information. The TIBCO Accelerator for Spark is a set of reusable components that show you how to use TIBCO StreamBase with Apache Spark, TIBCO Spotfire, and machine learning technologies such as SparkML and H20. Over 40 new connectivity points were developed for the Spark Accelerator that connect the TIBCO platform to Spark for machine learning, model monitoring, model retraining, streaming analtyics, and automated action. The presentation on the Spark Accelerator, which also featured customer Scotia Bank, who uses Hadoop in this configuration to discover new algorithms, is a great example that we shared recently. As the use of Spotfire and Streaming Analytics for managing IoT data increases, this pattern shown by this Accelerator will become, we believe, increasingly important.

The other benefit of this Accelerator project was that we created built-in components ("operators") in TIBCO StreamBase for SparkML and H20, and it shows an example of how TIBCO Spotfire and TIBCO StreamBase can work together to apply machine learning to IoT data.

How to Apply Big Data Analytics and Machine Learning to Real Time Processing: Extensive article at RTInsights explaining how to combine historical analysis and real time even processing.

TIBCO Now 2016 session on How to Apply Big Data Analytics and Machine Learning to Real-time Processing. This slide deck focuses on the concepts behind event processing and its relation to Apache Hadoop, Apache Spark, and other big data platforms. It shows a flexible solution architecture that combines the speed of Fast Data decisioning using TIBCO Event Processing with the intelligence obtained from big data analysis using TIBCO Spotfire®, TIBCO® Enterprise Runtime for R (TERR) and other analytic frameworks such as Apache Spark or H20. See how this concept is put into successful practice in manufacturing and airlines respectively for predictive maintenance, customer experience, and cross-selling.

The following video shows a conference talk from Voxxed Days in Zurich (March 2016) about this topic (including use cases, architecture, TIBCO products and live demo):

Machine Learning Use Cases

Machine learning allows you to go beyond traditional analytics to identify complex patterns in data to solve business-critical problems. Here are some examples of how it is being used in a variety of industries with TIBCO software:

Consumer Product Goods & Retail

Financial Services

  • Fighting financial crime -  Improved handling of financial crimes such as antimoney laundering (AML), credit card fraud, trade surveillance, or medical fraud.  Supervised learning algorithms guarantee accuracy, while unsupervised learning algorithms adapt to rapidly changing realities.  Read Whitepaper,    Watch video of demos,    Download Solution template,    Learn more
  • Algorithmic trading and algorithm discovery. Learn more.
  • Risk assessment

Energy

  • Production planning and value estimation of oil fields. Learn More
  • Predictive and preventive equipment maintenance.whitepaper

Manufacturing

  • Optimization of manufacturing equipment, processes and product yield using decision trees.  Covers historical analysis to build predictive models and real-time deployment with streaming data.  Read Whitepaper, View webinar Video, See Slideshare
  • Predictive and preventive equipment maintenance.whitepaper

Transportation and logistics

  • Streaming Analytics and the Internet of Things in Transportation and Logistics.  Cover use cases such as routing optimization, fuel efficiency, predictive maintenance, warehouse space optimization and missing luggage models.  Read the Whitepaper

Components on the Exchange

Gradient Boosting Machine Regression Data Function: Gradient Boosting Machine is a machine learning technique that automatically identifies patterns in data in order to build highly accurate predictive models.

The Gradient Boosting Machine Analysis Template is used to create a GBM machine learning model to understand the effects of predictor variables on a single response.  Examples of business problems that can be addressed include understanding causes of financial fraud, product quality problems, equipment failures, customer behavior, fuel efficiency, missing luggage and many others.  You can import your own data into this template, configure and perform the analysis, evaluate the model quality and visualize the results.  It is designed for use by a business analyst or citizen data scientist.  No specialized knowlege of statistics, data science or programming is assumed.

TIBCO Accelerator for Spark. Reusable components for TIBCO Spofire and TIBCO StreamBase to score models via SparkML and H20, and a general architecture for integrating visual and streaming analytics with Spark and machine learning.

Clustering with Variable Importance Data Function:  This data function clusters data rows based on multiple numeric input columns and ranks inputs columns by importance in determining clusters

Financial Crime Buster Analysis Template:  This Spotfire template guides the user through the tasks of adhoc data discovery, supervised model creation and unsupervised model creation to build a strategy for combating financial crime.

Customer Analytics Segmentation Analysis Template:  This template lets you identify customer segments for targeted marketing based on their past purchasing behavior in selected product categories.

View all Machine Learning components on the Exchange.

Learn More

A visual introduction to machine learning: cool, non-technical introduction to this topic.

10min live demo with TIBCO Spofire, TIBCO StreamBase and TIBCO Live Datamart of how business user, data scientist and developer work together to deploy an analytic model to real time streaming analytics.

How to Apply Machine Learning to Real Time Processing: Extensive introductory article with real world examples.

Avoiding the Anti-Pattern in Analytics: Three Keys for Machine Learning Success: Best practices for leveraging machine learning in a complete project including data discovery, model building and application to real time processing.

Advanced Analytics Wiki page

 

Back to Main TIBCO Spotfire Wiki page

Attachments

Feedback (1)

This is a very important area for TIBCO customers and we look forward to hearing community feedback on these techniques, and other that you're preferring and using, with the TIBCO stack. The Spark Accelerator is an important open-source toolkit that we've provided that implementes a template for using Machine Learning with Hadoop / Spark and our real-time infrastructure. Please do check out the Machine Learning Accelerator and give us your feedback, and / or share your extensions that you've made in the Component Exchange area!

Mark Palmer 11:41am Aug. 16, 2016