TIBCO Spotfire Survey Data Chart using JSViz
Last updated:
4:07pm Aug 03, 2017

 

A customer recently asked for some help creating the following chart in Spotfire using JSViz:

http://bl.ocks.org/wpoely86/e285b8e4c7b84710e463

 

It took a few hours and the final chart looks pretty good.  So I thought it would be good to walk through the process of creating the chart in JSViz.

Getting Started

The first thing we need is some data, so I took the sample data from the linked site, raw_data.csv, and converted it into a Spotfire stdf file.  You can download the data file from here.

With the data loaded, I added a JSViz visualization to my DXP.  JSViz picks up the table and creates a default Data Configuration, I just need to tweak it to suit my needs.  As my chart will use all the rows of data, I added all the columns to both parts of the Data Configuration page and set the column order and names to match the example as shown here:

Next I need to add some JavaScript files.  As a starting point I always use the Template files that ship with JSViz and are used in the tutorials.  So I added the following files:

Adding these to the Contents page in JSViz, keeping the above order, gives me my starting visualization:

Adding the Chart Code

Adding the css is just a matter of overwriting the contents of PollChart.css with the css code on the web site.

For the JS code, as with most visualizations I create, I separated out the code into two parts:

  • A section that converts the incoming Spotfire data into the objects expected by the chart code.  This goes in the renderCore() method and is called by Spotfire every time the data set changes.
     
  • A section that takes those objects and draws them on the screen.  This is pulled out into a function drawchart() which is called from renderCore() but is also called from the resizing routine whenever anyone resizes the visualization.  This will replace our current displayWelcomeMessage() call.

This approach ensures that when we need to resize the visualization, the chart data is available to redraw the visualization.

Here is the code that gets inserted into renderCore():

var polldata = [];

var color = d3.scale.ordinal()
            .range(["#c7001e", "#f6a580", "#cccccc", "#92c6db", "#086fad"])
            .domain(["Strongly disagree", "Disagree", "Neither agree nor disagree", "Agree", "Strongly agree"]);

var svg;

//
// Main Drawing Method
//

function renderCore(sfdata)
{
    if (resizing) {
        return;
    }

    // Log entering renderCore
    log ( "Entering renderCore" );

    // Extract the columns
    var columns = sfdata.columns;

    // Extract the data array section
    var chartdata = sfdata.data;

    // count the marked rows in the data set, needed later for marking rendering logic
    var markedRows = 0;
    for (var i = 0; i < chartdata.length; i++) {
        if (chartdata[i].hints.marked) {
            markedRows = markedRows + 1;
        }
    }

    polldata = [];

    for ( var nIndex = 0 ; nIndex < chartdata.length ; nIndex++ )
    {
        var items = chartdata[nIndex].items;

        var pollrow = items;

        pollrow.Question = items[0];

        pollrow["Strongly disagree"] = +items[1]*100/+items[6];

        pollrow["Disagree"] = +items[2]*100/+items[6];

        pollrow["Neither agree nor disagree"] = +items[3]*100/+items[6];

        pollrow["Agree"] = +items[4]*100/+items[6];

        pollrow["Strongly agree"] = +items[5]*100/items[6];

        var x0 = -1*(pollrow["Neither agree nor disagree"]/2+pollrow["Disagree"]+pollrow["Strongly disagree"]);
        var idx = 0;

        pollrow.boxes = color.domain().map(function(name) { return {name: name, x0: x0, x1: x0 += +pollrow[name], N: +pollrow[6], n: +pollrow[idx += 1] }; });
		
        polldata.push ( pollrow );
    }

    drawchart ();

    wait ( sfdata.wait, sfdata.static );
}

 

and the drawchart () routine looks like this:

 

function drawchart ()
{
    var width = window.innerWidth * 0.95;
    var height = window.innerHeight * 0.95;

    var margin = {top: 50, right: 20, bottom: 10, left: 85},
        width = width - margin.left - margin.right,
        height = height - margin.top - margin.bottom;

    var y = d3.scale.ordinal()
              .rangeRoundBands([0, height], .3);

    var x = d3.scale.linear()
              .rangeRound([0, width]);

    var xAxis = d3.svg.axis()
                  .scale(x)
                  .orient("top");

    var yAxis = d3.svg.axis()
                  .scale(y)
                  .orient("left")

    d3.select("#d3-plot").remove ();

    svg = d3.select("#js_chart").append("svg")
            .attr("width", width + margin.left + margin.right)
            .attr("height", height + margin.top + margin.bottom)
            .attr("id", "d3-plot")
            .append("g")
            .attr("transform", "translate(" + margin.left + "," + margin.top + ")");

    var min_val = d3.min(polldata, function(d) {
        return d.boxes["0"].x0;
    });

    var max_val = d3.max(polldata, function(d) {
        return d.boxes["4"].x1;
    });

    x.domain([min_val, max_val]).nice();
    y.domain(polldata.map(function(d) { return d.Question; }));

    svg.append("g")
       .attr("class", "x axis")
       .call(xAxis);

    svg.append("g")
       .attr("class", "y axis")
       .call(yAxis)

    var vakken = svg.selectAll(".question")
                    .data(polldata)
                    .enter().append("g")
                    .attr("class", "bar")
                    .attr("transform", function(d) { return "translate(0," + y(d.Question) + ")"; });

    var bars = vakken.selectAll("rect")
               .data(function(d) { return d.boxes; })
               .enter().append("g").attr("class", "subbar");

    bars.append("rect")
        .attr("height", y.rangeBand())
        .attr("x", function(d) { return x(d.x0); })
        .attr("width", function(d) { return x(d.x1) - x(d.x0); })
        .style("fill", function(d) { return color(d.name); });

    bars.append("text")
        .attr("x", function(d) { return x(d.x0); })
        .attr("y", y.rangeBand()/2)
        .attr("dy", "0.5em")
        .attr("dx", "0.5em")
        .style("font" ,"10px sans-serif")
        .style("text-anchor", "begin")
        .text(function(d) { return d.n !== 0 && (d.x1-d.x0)>3 ? d.n : "" });

    vakken.insert("rect",":first-child")
          .attr("height", y.rangeBand())
          .attr("x", "1")
          .attr("width", width)
          .attr("fill-opacity", "0.5")
          .style("fill", "#F5F5F5")
          .attr("class", function(d,index) { return index%2==0 ? "even" : "uneven"; });

    svg.append("g")
       .attr("class", "y axis")
       .append("line")
       .attr("x1", x(0))
       .attr("x2", x(0))
       .attr("y2", height);

    var startp = svg.append("g").attr("class", "legendbox").attr("id", "mylegendbox");

    // this is not nice, we should calculate the bounding box and use that
    var legend_tabs = [0, 120, 200, 375, 450];

    var legend = startp.selectAll(".legend")
                       .data(color.domain().slice())
                       .enter().append("g")
                       .attr("class", "legend")
                       .attr("transform", function(d, i) { return "translate(" + legend_tabs[i] + ",-45)"; });

    legend.append("rect")
          .attr("x", 0)
          .attr("width", 18)
          .attr("height", 18)
          .style("fill", color);

    legend.append("text")
          .attr("x", 22)
          .attr("y", 9)
          .attr("dy", ".35em")
          .style("text-anchor", "begin")
          .style("font" ,"10px sans-serif")
          .text(function(d) { return d; });

    d3.selectAll(".axis path")
      .style("fill", "none")
      .style("stroke", "#000")
      .style("shape-rendering", "crispEdges")

    d3.selectAll(".axis line")
      .style("fill", "none")
      .style("stroke", "#000")
      .style("shape-rendering", "crispEdges")

    var movesize = width/2 - startp.node().getBBox().width/2;

    d3.selectAll(".legendbox").attr("transform", "translate(" + movesize  + ",0)");
}

 

The main changes here are:

  • Changing the code to use the polldata object instead of the data read from the CSV file
  • Changing the DIV target to be "#js_chart"
  • Using window.innerWidth and window.innerHeight to determine the size of the drawing area
  • Remembering to clear the existing chart before adding a new one

With this in place we get a basic chart:

Adding Resizing Logic

This step is fairly simple because we have already separated the chart drawing logic into it's own function.  So the resizing routine looks like this:

var resizing = false;

window.onresize = function (event)
{
    resizing = true;
    if ($("#js_chart"))
    {
        drawchart ();
    }
    resizing = false;
}

 

Marking Logic

There are two aspects to adding Marking logic:

  1. Having the visualization render marked rows and un-marked rows differently.  In our case we will set the opacity of un-marked rows to 0.3 so they appear dimmed, similar to how Spotfire does it.  Spotfire provides the information on whether a row is marked in the data sent to renderCore() so we need to store this information in the polldata objects and use it in the drawchart() routine.
     
  2. Allowing the user to select a row, or rows, on the visualization and tell Spotfire to mark these items.  Spotfire provides a unique marking identifier in the data sent to renderCore() so we need to store this information in the polldata objects and use it in the markModel() routine.

Adding Marking Rendering Logic

Spotfire passes a flag indicating whether an item is marked in the "hints" section of the data passed to renderCore().  If we mark the first row of data in a Table visualization, and look at the first 2 rows of JSON data passed to JSViz, we can see the first row has the "marked" flag set and both rows have a marking id:

...
  "data": [
    {
      "items": [
        "Question 1",
        24,
        294,
        594,
        1927,
        376,
        3215
      ],
      "hints": {
        "marked": true,
        "index": 0
      }
    },
    {
      "items": [
        "Question 2",
        2,
        2,
        0,
        7,
        0,
        11
      ],
      "hints": {
        "index": 1
      }
    },
...

 

So we add code to our renderCore() method to store these values as properties of each polldata object as follows:

    for ( var nIndex = 0 ; nIndex < chartdata.length ; nIndex++ )
    {
		var items = chartdata[nIndex].items;
		//
		// Marking Index and Marked Flag 
		//
		var markid = chartdata[nIndex].hints.index;
		var marked = chartdata[nIndex].hints.marked ? true : false;

        ...

        pollrow.boxes = color.domain().map(function(name) { return {name: name, x0: x0, x1: x0 += +pollrow[name], N: +pollrow[6], n: +pollrow[idx += 1], markid: markid, marked: marked}; });
		
        polldata.push ( pollrow );
    }

 

Now that we have that data, we can use it in our drawchart() function to change the opacity of each rows as follows:

 

    ...

    bars.append("rect")
        .attr("height", y.rangeBand())
        .attr("x", function(d) { return x(d.x0); })
        .attr("width", function(d) { return x(d.x1) - x(d.x0); })
        .style("fill", function(d) { return color(d.name); })
        .attr("opacity", function (d, i) //Spotfire style faded marking coloring
        {
            if ( markedRows != 0 && !d.marked )
            {
                return (0.3);
            }
            else
            {
                return (1);
            }
        });

    ...

 

The logic here is that if no rows are marked then all items appear at full visibility.  To achieve this we need to use the markedRows variable that was created in renderCore().  Unfortunately this is not accessible from the drawchart() function so we need to go back and move markedRows to global scope.  Make sure to remove the "var" keyword from in front of the markedRows variable assignment in renderCore()!

 

var polldata = [];

var color = d3.scale.ordinal()
            .range(["#c7001e", "#f6a580", "#cccccc", "#92c6db", "#086fad"])
            .domain(["Strongly disagree", "Disagree", "Neither agree nor disagree", "Agree", "Strongly agree"]);

var markedRows = 0;

var svg;

...

function renderCore(sfdata)
{

...

    // count the marked rows in the data set, needed later for marking rendering logic
    markedRows = 0;

...

 

This gives us our familiar looking marking behavior:

 

 

Adding Marking Selection Logic

JSViz provides a standard rectangle selection mechanism by default.  In order to use this we just need to provide an implementation for the markModel() function.  In our case, the logic is to find which objects on the page intersect with the marking rectangle and submit their marking ids to Spotfire via a call to markIndices().  Here is the code for our markModel() function:

function markModel(markMode, rectangle)
{
    var selsvg = d3.select ( "svg" );

    if ( !selsvg )
    {
        return;
    }

    var indicesToMark = [];
    var markData = {};
    markData.markMode = markMode;

    svgElem = selsvg[0][0];

    var markRect = svgElem.createSVGRect();

    markRect.x = rectangle.x;
    markRect.y = rectangle.y;
    markRect.height = rectangle.height; // + one to get the item under the click
    markRect.width = rectangle.width; // + one to get the item under the click

    var elements = svgElem.getIntersectionList ( markRect, svgElem );

    for (var index = 0; index < elements.length; index = index + 1)
    {
        element = elements[index];

        if ( element.__data__ && element.__data__.boxes )
        {
            if ( element.__data__.boxes.length > 0 )
            {
                indicesToMark.push ( element.__data__.boxes[0].markid );
            }
        }
    }

    markData.indexSet = indicesToMark;
        
    markIndices ( markData ); 
}

 

With this code in place, we can mark items on the chart and the marking set in Spotfire will be updated.

 

Tips and Tricks

While developing code in JSViz, I recommend turning on the "Development" menu which allows you to use the built-in Chromium debugger to step through your visualization code and figure out what is going wrong, or just to introspect variables as the code executes.  To enable this feature, go to Tools->Options and scroll to the bottom of the Application Section:

 

Finished Example

You can download the finished example code and sample data from here:

Feedback (1)

Excellent step through - thanks for sharing.

petecookdav 8:14am Sep. 27, 2017