What's New in TIBCO Spotfire® 10.1

Last updated:
8:50pm Aug 26, 2019

 

Spotfire® 10.1 contains improvements in visual analytics, data wrangling, data access, and developer capabilities. In visual analytics, map controls in Spotfire have been redesigned and functionalities are now grouped more logically. Other key enhancements include ease in sorting by multiple columns within your tables, and the ability to more easily manage and keep in view multiple markings in your visualizations. To augment today's session-based delete rows feature, we have added a new row filter transformation to Spotfire. Now you can remove rows with one click inline from filters, when importing data or later from the data canvas. You can reload both linked and stored data sources in one step in Spotfire Analyst. Also, it’s now possible to trigger a reload of multiple data tables from the Data Table Properties dialog. In addition to a range of small but frequently requested enhancements, our APIs have been updated to support all the new features of the 10.1 release. 

Note that Spotfire® 10.1 is a mainstream version. Fixes to critical issues discovered after the release will only be made to the most current version and to any long term supported versions. For more information on the difference between mainstream versions and long term supported versions see the documentation.

Visual Analytics

Easier to sort by multiple columns in tables

It is now easier to sort a table by multiple columns, by using the contextual header popover. The popover is opened by clicking the table header.

Above: new sorting options for multiple columns in the contextual header popover.

Scroll new marking into view

When adding a new marking in the data section of the visualization properties dialog, the list of markings is now automatically scrolled so that the new marking will be in view. This makes it easier to manage multiple markings when there are many markings in the analysis.

New Map Chart controls

The map controls have been redesigned and are now visually cleaner and keep the functionalities grouped more logically.

Data Wrangling

Remove rows inline in filters or with the new data transformation

A new row filter data transformation has been added which enables you to remove rows based on what you have selected in filters. If you know that the values you have selected in a filter are the only values of interest going forward, you can remove all other corresponding values (rows) with one click. This automatically creates a new 'Filter rows' data transformation. The data transformation is automatically documented in the data canvas, and can be edited as any other data transformation type.

In the image below, we have created a filter and selected values to filter on. Once done, right-click anywhere on the filter and select Create Filter Transformation.

The filter transformation is automatically created and you can continue with your analysis. The transformation will be applied and your rows will be removed also when you reload data or when the analysis file is opened the next time.

If you would like to review, edit or remove the filter transformation, just navigate to the data canvas and click on the transformation step in the source view.

The expression defines what values to include. For a value column, the expression could look something like the example below.

(([fare_amount]>=20.0) AND ([fare_amount]<=150.0)) OR ([fare_amount] IS NULL)

You can also remove rows based on dates, as seen below.

The expression editor is reached through the data canvas, as for all data transformations.

This is an example of a filter expression on an Integer column:

([payment_type] NOT IN (1, 2)) OR ([payment_type] IS NULL)

You can also insert a filter rows transformation manually from the summary view when adding imported data, or later, from the data canvas.

Export data as comma-separated files (.csv)

You can now export data to .csv files with values separated by commas or semicolons, in addition to today's tabs. This makes it easier to import data that has been wrangled in Spotfire into other programs that sometimes only support these delimiters.

Use Export > Data to file like before.

 

Four new options are available in the file type selector:

The result for a semicolon-separated file looks like the example below.

VendorID;tpep_pickup_datetime;tpep_dropoff_datetime;passenger_count;trip_distance;RatecodeID;store_and_fwd_flag;PULocationID;DOLocationID;payment_type;fare_amount;extra;mta_tax;tip_amount;tolls_amount;improvement_surcharge;total_amount 1;2017-01-09 11:13:28 AM;2017-01-09 11:25:45 AM;1;3.3;1;N;263;161;1;12.5;0;0.5;2;0;0.3;15.3 1;2017-01-09 11:32:27 AM;2017-01-09 11:36:01 AM;1;0.9;1;N;186;234;1;5;0;0.5;1.45;0;0.3;7.25 1;2017-01-09 11:38:20 AM;2017-01-09 11:42:05 AM;1;1.1;1;N;164;161;1;5.5;0;0.5;1;0;0.3;7.3

 

Option to reload both linked and stored data sources in one step in Spotfire Analyst

It's now possible to reload not only all linked data sources as before, but also stored data sources, from the Data menu of Spotfire Analyst. You can also reload linked and stored data sources via the Spotfire API. The API is useful, for example, when you want to add custom reload buttons in a text area to be used by Spotfire Consumer web client users.

Data sources in Spotfire can have different loading methods. In the example below we have both linked (Always new data) data sources and stored (Stored data) data sources.

With this update, you can select whether to reload linked data sources only, or all data sources (that is, all data sources that are possible to reload). This is useful when you use stored data sources that are large and only need to be refreshed once in a while.

This feature is also available through the Spotfire API. For more information, see the Developer section further down.

Multi-select tables to reload in data table properties.

It is now possible to trigger a reload of multiple data tables at once from the Data Table Properties dialog.

Overview of selected prompt values in the data canvas

You can now review the values selected in prompts by looking at the information for the data source node in the data canvas. Previously, you had to reload your data to be able to review the selected prompt values.

Today() in-memory data engine function

This function is similar to DateTimeNow() and returns the current date. It is available in Spotfire Analyst and Business Author. You can use it in both calculated columns and custom expressions.

Edit transformation dialog now scrolls added or modified transformations into view

When a transformation is added or modified, the scroll position is now kept so that the added or modified transformation is still visible. When a transformation is deleted, the scroll position is retained so that you get a confirmation of the operation.

The transformation 'Calculate and Replace Column' now always sets the default column name

When adding a calculate and replace column transformation, the default column name is now automatically set to the selected column name.

Data Access

New database versions supported by self service connectors

Spotfire's native connectors now support the following new database versions:

  • Microsoft SQL Server 2017
  • Oracle 18c
  • Oracle MySQL 8.0
  • TIBCO Data Virtualization 8
  • Vertica 9.x

Please use the corresponding connector to connect to the above data sources.

Developer

API to reload linked and/or stored data sources

You can now reload individual data sources, both linked and stored, via the Spotfire C# API. This is useful, for example, when you want to add custom reload buttons in a text area to be used by Spotfire Consumer web client users.

Edit custom transformations

It is now possible to edit custom transformations from the Data Canvas.

Allow for empty values in custom row methods

The Spotfire C# API now allows for empty values to be propagated to custom row method implementations. Empty values may, in many cases, have special meaning and with this API addition those cases can now be handled in a custom row method.

API to configure the AutoCreateFilters property on a data table 

It is now possible, via the Spotfire C# API, to configure the AutoCreateFilters property on a data table. By setting the property to false (default is true) users get to manage filters manually and they can decide whether they want to create filters for certain columns or not. 

API to rename, copy and move items in the Library

The Spotfire C# API has been extended with additional Spotfire library management operations that make it possible to rename, copy, move and set metadata (description, keywords and properties) on library items. The operations can be called from custom extensions or from IronPython scripts.

Look up Automation Services jobs via the API

It is now possible to use the Spotfire C# API to browse for Automation Services jobs stored in the Spotfire library. This can be done directly from the LibraryManager class or using the LibraryBrowserDialog UI.

New runtime parameters for custom tasks in Automation Services

The Automation Services API now provides the following runtime parameters for custom tasks:

  • Information about which context a job is executed in, on the server or locally in the client. This enables custom tasks to execute differently depending on the context.
  • File path to running job.

Updated Java User Directory API

A new version of the Java User Directory API is released, which is easier to use and has extended functionality. The following capabilities have been added to the API, giving it the same options as the User Directory web service:

  • Set display name and email address when creating users and groups.
  • Update display name and email address for a user or group.
  • Improved search for users and groups.
  • Enable/disable a user.
  • Lock/unlock a user.

New Filter Rows Transformation

The Spotfire C# APi now exposes FilterRowsTransformation, a transformation type that is used to filter out rows that do not match the given expression. In addition, the more generic ExpressionTransformation now includes a where clause property. This API is useful when you want to add transformations to a data table using IronPython scripts or custom extensions.

Platform and system requirements

  • Dropped support for Oracle 11gR2 as Spotfire Server db.
  • The default behavior for importing CSV files has changed so that "Allow newline characters in quoted fields" is enabled by default.

Deprecated functionality

  • jQueryUI.
  • Exporting to PowerPoint as editable objects.
  • SAP BW Connector - ODBO.