TIBCO Cloud Integration Connect ... How It Works

THREE INTERFACES FOR THREE USER TYPES

TIBCO Cloud? Integration is a leading integration platform as a service (iPaaS), as rated in Forrester?s most recent Forrester Wave. It offers a broad range of capabilities for all levels of users from business users to integration experts and handles use cases from simple SaaS integrations to those supporting API-led application development. Each of its services can be accessed through three user interfaces:

??Connect: For business users and business application owners to simplify and accelerate cloud connectivity for data across SaaS and other business applications.

??Integrate: For integration specialists to implement mission-critical integration and orchestration services, and deploy anywhere including containers.

??Develop: For developers to build event-driven microservices and functions to support API-led application development.

This paper focuses on the Connect capability, its architecture and how it works.

ARCHITECTURE

TIBCO Cloud Integration has a multi-tenant architecture that scales by distributing workloads across agents that belong to each tenant. The cloud handles administrative tasks such as user management, connection management, integration design, scheduling, and monitoring. Agents carry out instructions they

receive from the cloud, handling integration jobs and reporting results back to the cloud, as shown in the diagram below.

A comprehensive security model encompasses all aspects of TIBCO Cloud Integration. The user interface, agents, and third-party applications communicate

with the cloud through secure channels. Connectors use secure protocols to communicate with APIs for their respective applications. Credentials are stored in encrypted containers, and IP safe lists restrict access to specific tenants.

3 FAST FACTS ABOUT TIBCO CLOUD INTEGRATION:

• Connectors use secure protocols
• Credentials are stored in encrypted containers
• Functionality is embeddable in partner apps using the REST API 

ACCOUNTS AND USERS

Each tenant in TIBCO Cloud Integration is called an account, which represents the company using the service. Most cloud applications restrict each user to a single account, but TIBCO Cloud Integration allows users to belong to more than one account. Users can switch from one account to another by logging out and logging in.

An administrator in an account can invite other users to that account and remove users from the account. Each account has one or more subscriptions to TIBCO Cloud Integration services.

SERVICES

TIBCO Cloud Integration offers two services:

1. The integration service performs bidirectional integration between two or more endpoints with scheduling, support for complex logic and transformations, and the ability to recognize records that are new or changed since the last integration job ran.
2. The replication service creates and maintains a copy of the data in a cloud application such as Salesforce.com or Microsoft Dynamics 365 in a database. The database can be on-premises or in the cloud.

See the TIBCO Cloud Integration Pricing Page for information on what is included in each available tier.

CONNECTOR FUNCTIONALITY

Connectors allow TIBCO Cloud Integration to communicate with specific applications. They interact on an open interface, allowing anyone to create a connector.

A connector links an application, database, or file system to TIBCO Cloud Integration. Most connectors support two-way communication, and they have several basic functions, including:

• Authenticating through a set of user credentials
• Discovering metadata, often including custom objects and custom fields
• Sending of queries, usually with filters, and receiving the results for processing
• Accepting inbound messages for processing
• Translating TIBCO Cloud Integration operations such as Create, Lookup, Update/Upsert, and Delete into specific API calls that the application understands

Connectors can process large datasets using paging, batches, or bulk loads. They can process hundreds or even thousands of records at once in a single API call, improving performance and reducing the number of API hits.

The TIBCO Cloud Integration user interface includes a Connector Marketplace. Through the Marketplace, users enable the connectors they need in their accounts. TIBCO provides many of the connectors, and third parties have created many others.

CONNECTIONS

Once a connector is enabled in a TIBCO Cloud Integration account, it can be used to make a connection to a specific instance of an application, database, or file system. In most cases, a connection includes user credentials for the application or database and a URL or other location of the application.

TIBCO Cloud Integration tests the credentials before creating the connection and stores the connection details in an encrypted container for later use when running integration maps. Connections are reusable?an existing connection can be used again and again in different maps.

AGENT TYPES AND FUNCTIONS

The TIBCO Cloud Integration architecture includes separate design and run-time components. The cloud platform handles administration, design, and monitoring, while delegating execution of replication and integration jobs to its agents. An agent retrieves its commands from the cloud, carries out the instructions, and reports the results to the cloud.

TIBCO Cloud Integration offers two types of agents:

• Cloud agents handle integrations between cloud-based endpoints such as SaaS applications, cloud databases, and cloud file systems like Dropbox. A cloud agent can be used to integrate leads between Marketo and Microsoft Dynamics 365 Sales. TIBCO hosts the agents in the TIBCO Cloud.
• On-premises agents handle integrations where one or more endpoints exist behind a firewall, inaccessible to a cloud agent. Users install an on-premises agent behind the firewall where the agent can communicate with the local endpoint as well as any cloud endpoints. You would use an on-premises agent to replicate Salesforce data to a local database.

All agents run on the same codebase and operate semi-autonomously, increasing the resilience and availability of the overall system. Even if an agent loses communication with the cloud, it continues to run the latest compiled versions of its maps according to the schedule. Once connectivity is reestablished, it reports the results to the cloud. This means agents continue to run even during scheduled maintenance of the cloud platform.

When TIBCO releases a new version of TIBCO Cloud Integration or a connector, the agent finishes any work it is doing, then downloads and runs the new code. This ensures that new software does not interrupt integration jobs.

MAPS

TIBCO Cloud Integration provides a graphical design environment where users create and maintain integration maps. Each map begins with one or more connections from the set defined in an account.

TIBCO Cloud Integration offers three types of maps:

1. Query maps use one of the connections associated with the map to get data for processing. The query can include filters to restrict the results. In most cases, this will include a net change filter, which TIBCO Cloud Integration uses to retrieve only records that are new or changed since the last time the map ran.
2. Message maps allow a cloud application like Salesforce.com to push messages to the map. Each message map has a unique URL. Users configure the cloud application to deliver records to the URL. When records arrive, the agent executes the map and uses one of the connections in the map to interpret the message payload. Message maps process asynchronously.
3. Request/Reply maps allow an external application to send records to a map in JSON format through a REST API. Like message maps, each request/reply map has its own unique URL. Unlike the other two map types, the application sending the request does not need a connection in TIBCO Cloud Integration. As the agent executes the map, it builds a response, which it sends back to the calling application. Request/reply maps process synchronously.

The TIBCO Cloud Integration user builds the map with the visual designer by dragging blocks onto a canvas to define a logic sequence. Control blocks direct the logic flow. The main control block, for each result, creates a loop that executes once for each record the query produces. Others provide if-then-else and other conditional tests.

Operation blocks take action on one of the connections in the map, such as creating, reading, updating, upserting, or deleting a record. They also handle field mapping and data transformation with a library of dozens of Excel-like functions. For update, upsert, and delete operations, users configure match fields to identify the correct record in the target connection.

TIBCO Cloud Integration also provides a map debugger that allows a user to execute a map one step at a time or to set breakpoints when certain conditions occur. This aids in troubleshooting map design.

SOLUTIONS

A solution is a set of ordered maps that are assigned to run on a specific agent. Solutions can be pre-set to run or run on-demand.

In many cases, an integration process involves more than one map. A solution is a set of ordered maps that are assigned to run on a specific agent. Each time a solution runs, it executes the maps until the final map completes. Solutions can run according to a pre-set schedule or can run on-demand through the user interface or via an API call.

Solutions can be exported and imported. The exported maps contain the definition of the connections used, data flow, and mappings/formulas used. Export and import allow you to move solutions between test and production environments or from one customer?s account to another.

REST API

TIBCO Cloud Integration offers a REST API that allows you to embed its functionality into another SaaS application of your choice.

TIBCO Cloud Integration offers a REST API that provides application programmers with access to much of its functionality. You can embed its functionality in another SaaS application and automate processes for:

• Creating users
• Installing agents
• Adding connectors and creating connections
• Creating a solution with a set of maps
• Scheduling solutions
• Monitoring the results of integration tasks

TIBCO Cloud Integration enforces a safe list for the API based on the IP address making the API call. Users configure one or more IP address ranges that can make API calls for an account.

NEW CONNECTOR DEVELOPMENT

SaaS companies and integration consulting firms can create their own connectors for TIBCO Cloud Integration. The scope of a connector development project includes authentication, metadata discovery, queries, and target operations like create, update, and delete.

There are three options available for building connectors for TIBCO Cloud Integration:

• The Messaging Framework works with applications that send outgoing webhooks in either JSON or XML format. This framework allows you to quickly build a connector from a sample message in Microsoft Visual Studio. You can use the connector as the source in a message map, but target operations are not supported.
• The Fast Connector Framework is a wizard-driven tool that helps you build a connector to an HTTP web service by completing a Microsoft Visual Studio Solution with the specific details of the application you?re connecting to. This approach allows you to build connectors that support both source and target operations.
• The Software Development Kit (SDK) offers more flexibility and access to additional functionality, such as complex filtering in queries, picklist values in metadata, additional target operations beyond create and update, and support for related entities.

TIBCO offers a certification process for those who wish to make a connector available to others. Once certified, your connector can be listed in the TIBCO Cloud Integration Connector Marketplace.

MONITORING AND NOTIFICATIONS

Each time an agent runs an integration job, it reports the results back to TIBCO Cloud Integration. Results include how long the job ran, how many records were processed, how many failed, and any errors that occurred. Users can view the execution history related to a solution in the TIBCO Cloud Integration user interface. TIBCO Cloud Integration can reprocess failed records for up to 45 days.

Users can also configure TIBCO Cloud Integration to send notifications via email when certain events occur. Notifications are available for solution and record failures, system updates, connector installation, and lack of agent communication with the cloud. These settings are specific to each user in each account because users can belong to multiple accounts and might want different settings for each one.

CONCLUSION

Whether you are a business analyst, citizen integrator, integration specialist, or app developer, TIBCO Cloud Integration makes it easier than ever to connect all the applications you use. Its UI and pre-built connectors let you see value faster by accelerating not just the initial design, but the ongoing maintenance of your integrations.

Start a free trial today to experience its power and ease of use for yourself.