Displaying a Sankey chart in TIBCO Spotfire using TIBCO Enterprise Runtime for R and JSViz

By:
Last updated:
9:50am Sep 27, 2020

JSViz is a framework for creating JavaScript-enabled visualizations in TIBCO Spotfire®. This article walks through using the framework with TIBCO® Enterprise Runtime for R (TERR), TIBCO's open-source R-compatible statistical engine available in Spotfire, to create visualizations not currently natively available in Spotfire.

You can see detailed information on JSViz and review its documentation in a downloadable zip, which you can find elsewhere on the TIBCO Community Site. The documentation contains examples, tutorials, and an extensive User Guide.

This article focuses on creating JavaScript graphics for display in Spotfire. If you want to create a JavaScript graphic directly from TERR for use outside of Spotfire, see the section Graphics in TIBCO Enterprise Runtime for R in the TIBCO® Enterprise Runtime for R Technical Documentation.

Note   As a matter of good development and debugging practice, we recommend exporting the data set from Spotfire as an SBDF file, and then running the example either from the TERR console or from RStudio. See the article Testing TIBCO® Enterprise Runtime for R (TERR) data functions inside and outside of TIBCO Spotfire for more information.

In this article, we create a Sankey chart using sample data that is installed by default in Spotfire.


Prerequisites

Make sure you have all of the following components to run this example in Spotfire.

Required component Description

TIBCO Spotfire Analyst version 7.5 or later, with example DXPs installed on the server.

We tested this example with version 7.10 of Spotfire Analyst, which includes TERR 4.3.1.

TERR version 4.3.1 (included with Spotfire Analyst 7.10) or later.

This release enables the graphics support required to run the functions in the networkD3 and htmlwidgets packages.

The Pandoc document converter.

The files resulting from building the JavaScript visualization must be converted to one HTML document that you can display in Spotfire. The function htmlwidgets::saveWidget, which we use in this example, relies on help from Pandoc to convert the JavaScript to a single HTML document.

A third-party document converter tool, Pandoc, is free, and available to learn about and to download at https://github.com/jgm/pandoc/releases/tag/1.19.2. We used version 1.19.2.1, released in February of 2017. (Later versions do not work as of this posting in November 2017. 

Note   You must put Pandoc in your path, and then restart Spotfire Analyst to use it.

The networkD3 and htmlwidgets packages installed in TERR.

The R packages networkD3 and htmlwidgets are available on CRAN.

  • The package networkD3 provides a variety of JavaScript-enabled charts, including the Sankey chart.
  • The package htmlwidgets provides the function to create a single HTML file from a collection of JavaScript files so you can display the single page in Spotfire.

Other packages, such as rCharts, are not based on htmlwidgets. They have a different way of generating the HTML file.

From within Spotfire, you can use the Tools > TERR Tools > Package Management system to download and install these two packages and their dependencies.

To install the packages from the TERR console, you can use the function install.packages().

TIBCO Spotfire Statistics Services running TERR 4.3.1 or later.

This service is required if you want to run the solution in Spotfire Business Author or Consumer.

JSViz extension v3.4.0.10 or newer.

You can download the required version of the JSViz extension from the TIBCO Community site. It is important to use the correct version, because this example requires new, recently-added features.  Users who are upgrading to, or installing, Spotfire 7.12 must download and install JSViz 3.4.0.12.

To install JSViz, give the extension's .SDN package to your Spotfire Server administrator to install on the Spotfire Server that you access with your installation of Spotfire Analyst. After the server administrator has added the package, you must restart Spotfire Analyst and accept the updates when you are prompted.

Note  Custom extensions are not supported in Spotfire Desktop. To run this example, you must run Spotfire Analyst.

When the extension is successfully installed, the JSViz icon is displayed on the Spotfire toolbar.

If you believe that JSViz is installed on Spotfire Server, but you do not see the JSViz icon, check with your Spotfire administrator to make sure the Custom Extensions license is enabled for your group.

JSViz icon on the Spotfire toolbar.

It is also useful to have a basic understanding of how to construct Spotfire data functions. For readers new to the data function feature, we explain the steps as simply as possible in this example. For more information about data functions, see the Spotfire Analyst help, included with your installation.

Overview

In this walkthrough, you can perform the following tasks to create a JavaScript visualization that uses TERR inside of Spotfire.

Note   You can perform this task in the TERR console or in RStudio running TERR or open-source R by exporting the data from Spotfire and opening the resulting file in an HTML browser. Pandoc is not required in that case.

  1. Open the sample DXP file and create the document property.
  2. Create a Spotfire data function to call the TERR code.
  3. Return the output as a document property with all the required HTML code embedded.
  4. Create a JavaScript visualization that contains the embedded HTML.

Spotfire setup

To get started:

  1. Open a session of Spotfire Analyst that is connected to a Spotfire Server with the JSViz extension installed.
  2. Open the Spotfire sample Expense Analyzer Dashboard.dxp. This sample DXP is installed by default and is displayed under the Samples tab on the Spotfire start screen. If you do not see the Samples DXPs, ask your Spotfire Server administrator to install the samples.

Expense Analyzer Dashboard example on the Spotfire start page.

Installing the required packages

If you have not yet installed the required packages, then follow these steps.

  1. In Spotfire Analyst, from the menu, click Tools > TERR Tools.
  2. In the TERR Tools dialog box, click the Package Management tab.
  3. In the CRAN Package Repository drop-down list, select your favorite repository, and then click Load to populate the Available Packages list.
  4. From the list, select networkD3 and htmlwidgets, and then click Install.
  5. Close the dialog box.

Creating the document property

Create the document property to hold the HTML data returned by the data function. Be sure to use the specified name and data type. 

  1. In Spotfire Analyst, from the menu, click Edit > Document Properties.
  2. In the Document Properties dialog box, click New.
  3. In the New Properties dialog box, provide the following property values, and then click OK to save the document property.
Property Value
Property name HTMLData
Data type String
Description Provide a description that is useful to future users. For example, "This document property contains the HTML string returned from running a data function that creates an HTML page."

You are now ready to create the data function that uses these packages.

Creating the data function script

The script contained in step 6 of this section uses functions in the installed packages plus TERR code to analyze the data specified by the input parameter. The script returns the results in an HTML string, which is then stored in the output parameter.

  1. From the Spotfire Analyst menu, click Tools > Register Data Functions.

    The Register Data Functions dialog box is displayed.
  2. Provide a Name.

    We used Create Sankey chart.
  3. For Packages, specify networkD3;htmlwidgets.
  4. Optionally, provide a Description.
  5. Clear the Allow caching checkbox.
  6. Copy and paste the following R script into the Script edit box.
# Input Parameter: 'data' 
# A Table with the character string columns 'Department' and
# 'Type', and a numeric column, 'Line Amount'. Each row represents
# an expense of the given value of the given type by the given 
# department. 

# Output Parameter: 'htmlData' 
# A character string containing the contents of 
# an HTML file that displays a Sankey chart. 

# make sure you are using the correct columns. 

stopifnot("Line Amount" %in% colnames(data), 
          "Department"  %in% colnames(data), 
          "Type"        %in% colnames(data)) 

# aggregate the expenditures by type and department, showing the  
# amounts flowing from each department to the expense type.

summedData <- aggregate(data["Line Amount"], 
                        by=data[c("Department", "Type")], 
                        FUN=sum
                        )

categories <- unique(c(summedData$Department, summedData$Type)) 

# create the chart information for the nodes and links 
# to match the dept, type, and amount. 

chartInfo <- list( nodes = data.frame(stringsAsFactors = FALSE, 
                                name = categories
                                ), 
             links = data.frame(dept = match(summedData$Department, 
                                             categories
                                             ) - 1L, 
                                type = match(summedData$Type, 
                                             categories
                                             ) - 1L, 
                                amt  = summedData$'Line Amount'
                                )
            ) 

# create the chart by calling the networkD3 package function , 
# sankeyNetwork passing in the information for the nodes and links,    
# and setting the display properties for fonts and width. 

chart <- networkD3::sankeyNetwork( Links     = chartInfo$links, 
                                   Source    = "dept", 
                                   Target    = "type", 
                                   Value     = "amt", 
                                   Nodes     = chartInfo$nodes, 
                                   NodeID    = "name", 
                                   units     = "USD", 
                                   fontSize  = 12, 
                                   nodeWidth = 30
                                  ) 

# From the resulting files created by the function, 
# create the single HTML file to display in Spotfire. 

dir.create(htmlDir <- tempfile()) 
htmlFilename       <- file.path(htmlDir, "chart.html") 

# create a single HTML file of the graphic for any graphic object 
# that inherits from the class "htmlwidgets" (defined in the 
# htmlwidgets package).

htmlwidgets::saveWidget(chart, 
                        file = htmlFilename, 
                        selfcontained = TRUE
                        ) 
htmlData <- readChar(htmlFilename, file.size(htmlFilename)) 

# clean up. 

unlink(recursive=TRUE, htmlDir)
The Register Data Functions dialog box now looks like the following.

Register Data Functions dialog box populated with the script.

Adding the input and output parameters

The input and output parameters that you add to the Spotfire data function must match the inputs and outputs specified in the TERR script (described in the code comments and used in the script). In addition, Spotfire expects the data types given below, so you must specify them as described.

  1. Click the Input Parameters tab.
  2. Click Add, and in the resulting Input Parameter dialog box, set the following properties.
    Property Value
    Input parameter name data
    Display name String
    Type Table
    Allowed data types Integer, Real, SingleReal, Currency, String
    Description Optionally, copy the description from the code. This is useful to someone running the data function who does not have access to the script.
    Required parameter Selected
  3. Click the Output Parameters tab.
  4. Click Add, and in the resulting Output Parameter dialog box, set the following properties.
    Property Value
    Result parameter name htmlData
    Display name htmlData
    Type Value
    Description Optionally, copy the description from the code. This is useful to someone running the data function who does not have access to the script.
  5. On the dialog box toolbar, click Run to run the data function.

When you run the data function, the TERR engine prompts Spotfire to match the input and output objects specified in the script to the input and output parameters named in Spotfire. Spotfire displays the Edit Parameters dialog box to prompt you to assign the match for the parameters.

Assigning the data function inputs and outputs

In these steps, first you specify which of the data sample's available columns the data function expects to operate. Then you match the HTML output to the document property that can render the long HTML string (called htmlData in the R script, and matched to HTMLData in the document property) created by the TERR code.

  1. In the Edit Parameters dialog box, select Refresh function automatically.

    This step is needed if you expect the data passed to the function to change due to streaming or filtering.
  2. In the Input tab, for Input handler, select Columns.
  3. Click Select Columns.

    The Select Columns dialog box is displayed.
  4. From the Available columns list, select Type, Department, and Line Amount, and add them to the Selected columns list, and then click OK to close this dialog box and return to the Edit Parameters dialog box.

    Remember from the script that these are the only columns we specified for the analysis. (Type and Department are type String, and Line Amount is type Currency.)

    Input columns for the data functions in JSViz.
  5. Scroll down to display the Limit by option, and from the Filtered rows drop-down, select (Active filtering scheme).

    This step is needed if you plan to apply a filter, which causes the data function to rerun and the visualization to refresh.
  6. Click the Output tab, and then assign the output to the document property you created.

    Output property for JSViz
  7. Click OK to close the Edit parameters dialog box.
  8. Click Refresh to run the data function and write to the document property the HTML for the visualization.

    (The HTML string created by the data function is too large to display in the Document Properties dialog box; however, if you want to examine the string, you can export it from the TERR console, available from Spotfire, by typing htmlData at the console prompt.)
  9. Close the Data Function Properties dialog box.
  10. Optionally, save the data function to the Spotfire Server library, or you can embed the data function in the analysis.

Next, set up JSViz to render the visualization.

JSViz Setup

Adding a JSViz Visualization

  1. Create a new visualization page.
  2. On the toolbar, click the JSViz icon to add an empty JSViz visualization.

Next, add the HTML content from the Document Property HTMLData.

Adding HTML Linked Content

Property Value Description
Name chart.html The HTML file that contains the content.
URL %DocProp.HTMLData% Special syntax that assigns the named document property for the URL.
Type HTML The format of data the content contains.

Create the linked content.

  1. Open the properties for the JSViz visualization, and in the left panel, click Library.
  2. Add a Linked Content item with the following properties.
  3. Save the setting to close the dialog box and create the Linked Content item.
  4. In the left pane, click Contents, and then click the arrow to add chart.html to the Used Content list.

The visualization is displayed in the JSViz window.

JSViz visualization

Summary

We created a simple example of using the networkD3 package in TERR. We have shown how to integrate JSViz into Spotfire, and how to pass in parameters. By using these steps as a starting point, you should be able to create your own interactive visualization inside Spotfire using TERR, JSViz, and your own data.

Feedback (2)

@Regi, on your Windows computer, go to  Settings, and then search for environment variables. From the results, select Edit the system environment variables.  In the System Properties dialog box, click Environment Variables, and then add the path to your Pandoc installation in the system variables list. (Click New, provide the name, and for the value, provide the path.)  Hope this helps!

kroan 3:41pm Feb. 19, 2019

Dear Kroan,

First, super thanks for having created such a detailed tutorial.
I had your dxp adjusted to an analysis in my personal computer and everything worked fine.
Now I am trying to run it at the office and I am having trouble with the pandoc installation.
How exactly am I to " put Pandoc in your path, and then restart Spotfire Analyst to use it." ?

I appreciate the extra help.

Kind regards,
Regi

fhb160157 12:00pm Feb. 17, 2019