Image Reader Mod for TIBCO® Data Science - Team Studio
This offering consists of a custom mod that provides the ability to read images using TIBCO Data Science - Team Studio. The mod allows users to read images from a cluster using a no-code UI.
TIBCO® Data Science Team Studio
TIBCO Data Science - Team Studio 6.5
TIBCO Data Science - Team Studio 6.5
The Image Reader MOD for TIBCO® Data Science - Team Studio allows users to read images from a cluster using Apache Spark in the backend.
This ability is particularly useful for manipulating image data and building Computer Vision models using the Team Studio platform.
The output of the mod will be several metadata columns and a data column containing a string of comma separated pixels.
If you want to start using Mod/Custom operator in Team Studio environment, please follow this Knowledge Base article for installation guidelines.
There are currently no reviews for this content.
Why not be the first to review it - click here to login
Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human visual system can do. [Source]
In this page, I will talk about the basics of computer vision and the current industry solutions that solve this AI problem.
Images are a set of integers representing the intensity of different pixels values between 0 and 255. Images can either be Black and White (BW) or colour (RBG).
For instance, examine the BW image below. We can see how one small portion within the larger image looks like. Notice that in the matrix of integers, 0 represents a completely dark pixel and 255 represents a white pixel. All other pixel values that fall in between 0 and 255 are shown in different shades of gray.
Colour images are also very similar to the BW example shown above except for one key difference. Instead of only one channel representing the intensity between Black and White, colour images will have 3 different channels – Red, Green, Blue. These channels are superimposed upon each other to create images that accurately depict objects, people etc. A 4th channel – called alpha - representing transparency factor may also be available.
We can make several different transformations on an image file to alter the image’s appearance or layout. Some of those are –
- Adjust Brightness, Contrast, Hue, Saturation
Several libraries are commonly used for these transformations –
Deep Learning with Images – Intro to Convolution Neural Networks
A convolution neural network (CNN) is a type of feed forward neural network that is very effective in computer vision related tasks. A standard CNN has 2 types of layers in addition to the dense & dropout layers found in a normal feed forward neural network. They are –
- Convolution layer
We can see the working of a convolution layer in the video snippet below. In an operation, we take a matrix of weights and multiply them with a sub-matrix within an image. The result is saved to a new output matrix and the process is repeated after shifting by one cell. This operation will create a smaller dataset for use in subsequent layers.
Image Source: Google developer site
The pooling layer is required in a CNN to down sample the features that we detected in a previous convolution layer. Typically, Max pooling and Average pooling layers are used in this layer.
More information on convolution and pooling layers is available here. A representative picture of a sample convolution neural network is as below.
Earlier layers of a neural network tend to capture low-level features of an image – for example, eyes in an image of a person’s face. As we train a deeper CNN, the network will pool all low-level features and so understand and generate high-level features – like a complete face. See example below.
Challenges in Computer Vision Currently
The accuracy of the computer vision models we can train is highly dependent on the quality of our data. The common challenges we encounter with image data currently are –
- Occlusion : Some object is in the way of the object we would like to detect in the image
- Viewpoint variation : Pictures in the same class were shot from different angles leading to a difficulty in capturing patterns
- Illumination : Pictures in the same class have varying levels of brightness
- Deformation : The object we are trying to detect is deformed in some of the pictures
- Background objects : There are background objects that look similar to the object we are trying to detect in some of the pictures
Intraclass variation : Objects in pictures of the same class look different in different pictures. For example, if we are trying to detect a dog – the breed of dog might lead to variations in shape, size and colour.
Applications of Computer vision
Image recognition tasks are being used in a variety of enterprise and consumer focused technology products. Some examples of image recognition tasks are –
Image Classification: Trying to classify images based on the content of the image. For example – Identifying faulty/ not faulty chips in a manufacturing usecase.
Image Source: CS231.github
Object Detection: Identifying the exact location of objects or people in an image. The most popular example for this task is the Tesla Autopilot driving system, which identifies objects/people and uses these to make decisions.
Source: Sharif Elfoly medium article
Image Captioning: Identifying what is happening in the picture and then translating it into natural language. For example, a convolutional neural network would be able to look at the image show below and translate it into human language.
Image Source: TF website
Computer vision services in the industry
Here is a list of some of the popular sources available for undertaking computer vision tasks -
Open source tools – OpenCV, TensorFlow, PyTorch etc.
These tools provide the fundamental building blocks for computer vision. We can read, transform and manipulate image data. We can use tools like TensorFlow & PyTorch to also CV models.
Pre-trained AI models eg: Inception, VGG
These are open source CV models that can be used for techniques such as Transfer Learning that help new users achieve greater accuracy in shorter period of time.
Models-as-a-Service eg: AWS Rekognition, Azure Cognitive Services, Google AI platform
These CV models are made available by large multinational corporations via web servers.
AI development options eg: AWS Sagemaker, Azure ML, GCP AI platform etc.
These services will allow us to use the open source tools mentioned above to train and infer new computer vision models
TIBCO Solutions for Computer vision
Dr Data Science – A picture is worth a million bytes
A video walkthrough of the all the content mentioned above and a demo using a standard MNIST handwritten numbers dataset is shown in the video.
Accessing pre-trained models from cloud services
With the main cloud providers all offering image recognition services, we wanted to utilise their existing functionality in Spotfire® if possible. This would not only mean Spotfire could be used to run image recognition models but also connect to cloud services.
Summary of cloud providers image services
To build this image recognition Spotfire tool we would need to be able to do the following:
- Read images into Spotfire and extract metadata i.e. image name, dimensions, etc
- Submit images to a cloud service and handle the returned results
Visualise the results in Spotfire
Accessing image data at scale using TIBCO Team Studio
Mod (custom operator) for reading image data can be downloaded for free from this community exchange page.
Venkata Jagannath is a lead data scientist working on the Data Science team in TIBCO, in the US East region. He works on problems that are at the intersection of Artificial Intelligence, Big Data and Cloud computing. While not at work, he likes to work on his pet projects related to personal finance, home improvements etc.