FAQ: Python Data Functions in TIBCO Spotfire®

By:
Last updated:
10:10am Jun 25, 2020
Table of Contents

This article collects questions and answers related to the native Python data functions introduced in TIBCO Spotfire® 10.7. The article is divided into sections for different categories of questions. Please note that there are a few of other articles that provide introduction and examples related to Python Data functions here.

General

Does Spotfire come with its own Python interpreter?

Yes, Spotfire Analyst bundles a Python interpreter. You can find it in the Modules folder of your Spotfire installation directory.

Can I use a specific python interpreter that I installed on my computer instead of the one that is bundled with Spotfire?

Yes, in Tools->Options->Data functions in Spotfire Analyst you can set a path to a locally installed Python interpreter. Please note that in this case you will need to make sure the locally installed Python environment has the required packages.

How does the Python Data function support in Spotfire 10.7 relate to the Python Data function extension available at the TIBCO Community?

The Python Data function extension is a community extension that works with earlier versions of Spotfire. From Spotfire 10.7 and onwards this extension is no longer needed and will eventually be obsoleted. It will however be available for usage with earlier versions of Spotfire for some time so that users have time to migrate to Spotfire 10.7 or later. 

We previously used the Python Data function extension and since we upgraded to Spotfire 10.7+ we cannot run or edit the data functions we already had created. What do we need to do?

This will happen if you do not install the Python Data function extension on your Spotfire 10.7 system. Even though 10.7 has native python data functions, these are not the same as the data functions created using the extension. The Python Data Function extension not part of the regular Spotfire installation, and if you need to run and edit data functions that are created using the extension you need to add the extension to Spotfire 10.7 (or if you are using a later version). 

I want to migrate my Python data functions that I created using the Python Data function extension to the native python data functions. How do I go about this?

You need to copy the code from your original data function, create a new native python data function and past the code into it, define the corresponding inputs and output parameters, and if you are lucky that is it. If you are not that lucky, you may have to make sure the right packages are available in the native Python interpreter, and possibly you may have to modify the code to some extent as guided by the error messages.

Can the Spotfire Web Player and Automation Services use Python Data functions?

Edit: Yes as of Spotfire 10.10 LTS this is possible through the TIBCO Spotfire Service for Python. Read more in the whats new article: https://community.tibco.com/wiki/whats-new-tibco-spotfirer-1010-lts

(Original answer No, not in Spotfire 10.7. This is a planned future feature though, please vote for this on the TIBCO Idea portal: https://ideas.tibco.com/ideas/TS-I-7546)

Where can I read more about Python data functions in Spotfire?

Here is a good starting point. This is a top level article that links to a few other articles and documentation on specific subjects.


Inputs and outputs

How are inputs and outputs from Spotfire colums, tables etc represented in a Python data function?

When you define inputs and outputs to a data function, you can set them to be of the Spotfire type Value, Column or Table. In all cases, in your Python code they are refered to with the name you defined. So if you name an input of type value "inputValue", you will refer to it in your python code as "inputValue", like: if inputValue > 3 : ...

The Python type of the inputs and outputs will vary based on the type of the input:

Value: this will be a Python int, float or str dependent on the type of the value.

Column: this will be a Pandas Series

Table: this will be a Pandas DataFrame

Read more detail about the mapping of Python data types to Spotfire data types in the section Spotfire and Python data type mapping of this document.

 


Python Package Management

How do I add other packages to be used with my Python data functions?

You use the commandline pip utility with the bundled Python interpreter, like >python.exe -m pip install nltk. Read more in this article.

Note that in Spotfire 10.8 and later you can also use the Python tools located in Tools->Python tools from Spotfire Analyst.

Can I somehow make sure all my users use the same Python packages when developing data functions for Spotfire?

You can create a Spotfire .spk file that bundles a set of Python packages and is deployed to each user as part of the regular Spotfire package deploymen mechanism. Read more about this in the documentation and in this article.

What packages are included with the bundled Python interpreter?

Bitstring, numpy, pandas and a couple more. Please read the documentation for a comprehensive list. Look for the document Python Data functions in Spotfire Analyst.

Can I distribute a Python package we have developed internally in our company using the Spotfire package deployment mechanism? The Python package we have developed is not available via pip.

Yes, when creating your .spk you can include a package from a different repository or in local a file path. In the in requirements.txt file use the option -i or --index-url, followed by the location URL.

#example

#

mylib -i http://my.domain.org/lib/1.0.0/mylib/

Also see the documentation.

When I use the "Python tools" (located in the Tools menu of spotfire Analyst) to install packages, where is Python tools looking for the packages?

Python tools by default look for packages at the PyPI repository. Python tools use PIP "under the hood". If it is desried to have PIP point to a different repository than PyPI, PIP can be configured to look for packages in other locations. Please refer to the pip documentation: https://pip.pypa.io/en/stable/user_guide/#configuration

Machine learning

If I have trained a model using a data function, how can I make use of it for prediction in other data functions?

You can store a trained model in a document property using the pickle package. See the chapter "Using a Model Produced in a Previous Data Function" in the documentation


Python in Earlier versions of Spotfire

I have an earlier version of Spotfire. Can I use Python in some other way?

Yes. First, Python in Spotfire comes in two major forms: IronPython and Python Data Functions. IronPython is used with the Spotfire API for automating actions and creating custom Spotfire application behavior. Data Functions are used for scripting advanced calculations and can use open-source Python libraries. Data Functions have historically been available out-of-the-box with TIBCO® Enterprise Runtime for R (TERR™) allowing R scripting. As of Spotfire 7.13, this capability was extended to Python using a Community Extension add-on and as of Spotfire 10.7, this capability became native with an out-of-the box and fully-supported Python environment. In addition, prior to Spotfire 7.13, some users may have found they may use Python through TERR™ using the “PythonInR” package or similar packages, however, these methods are very limited and are no longer recommended.

This page focuses on the Python Data Function capabilities in Spotfire. For more information on using IronPython for Spotfire API Scripting and Automation, please visit the main Community site here.

Python Data Functions

In order of best functionality, there are 3 possible methods for creating Python Data Functions:

Which method you should use is mostly dependent on which version of Spotfire you are using. As mentioned earlier, using Python through the R language in TERR is not recommended, however, it is the only option for users with Spotfire versions prior to 7.13. Spotfire users between versions 7.13 and 10.6 should use the Python Extension from the TIBCO Exchange which allows full Python functionality, however, it is unsupported and not easily governed across the enterprise. Also note that users of this Extension will not be able to migrate their Data Functions to versions of Spotfire 10.7 and above, and will likely need to manually re-create the Data Functions in the Spotfire-native Python handler (Though almost all python code will work unchanged). Users on Spotfire 10.7 and above should use the native Python Data Function capability that ships with Spotfire.


The native Python Data Function interpreter that ships with Spotfire 10.7 is the most capable and feature-complete method for using Python in Spotfire. The other methods will gradually become obsolete while the native interpreter will continue to be supported and enhanced through future versions of Spotfire.