Data Cleansing

Table of Contents

• Planning 
• Event Configuration
• Purging
• Final Archiving Activity
• Cassandra Archiving

Data Cleansing is a very Important Topic for Business Process Management, to keep the System clean, quick, easy to handle and restore.

Planning 

The Main Area for Housekeeping is for the the Audit Log Tables of the Event Collector, used for many Reporting Activities in most Projects.

Every Project should contain a Activity to check the needs for keeping Event Entries for Business Transparency, e.g.

• how long is the Audit Data needed ?
• What Events are really needed for Reporting ?
• What Event Attributes need to be stored ?
• Is a final Archiving of Messages needed ?

Note: Many times a Timeframe of keeping Data for around ~ 1 Month - 3 Months is fine.

Event Configuration

By Default a lot of Events automatically tracked and audited by the Event Collector, also a lot Attributes are automatically kept.

Also any times Projects just configure the System to just store all Workitem Attributes.

This Audit Configuration about what is really needed and what is not needed should be done carefully and as Part of every Project.

Purging

TIBCO ActiveMatrix BPM comes with a number of Options to perform this Activities in a standard Way.

• Stored Procedures for various Database Types, incl
  • Clearing Audit and Statistical Data for Process Instances
  • Clearing Audit and Statistical Data for Process Instances Across Multiple Sessions
  • Clearing Audit and Statistical Data Not Related to a Process Instance
• Command Line Interfaces to execute purging
• API's start purging and track their Status via REST/SOAP/JAVA

Final Archiving Activity

Some Projects using also a Archiving Activity at the final End of a Procedure to 

• create a final Case Instance Data Extract as PDF
• create a final Audit Data Extract as PDF
• create a final Case Notes Extract as PDF

With this kind of Process Activity in place, most times Event Collector Data can be purged earlier.

Cassandra Archiving

Finally Apache Cassandra can be used to Archiving automatically all Event Collector Messages, this has the Benefit that still Reporting can be used based on the Cassandra Data e.g. by using Spotfire.

• Archiving Messages
• Cassandra Configuration Details