When you build an integration in TIBCO BusinessWorks, you almost always need to connect two systems that use different field names for the same data. One system calls it firstName, another calls it givenName. One calls it city, another calls it municipality. One uses genderCode, another uses sex. Manually matching hundreds sometimes thousands of fields like this is slow, tedious, and error-prone.The AI Mapper introduced in 6.12.0 Hotfix 3 is a feature in BW6 Studio that does this matching automatically. You open the mapping editor, right-click on the target element and choose AI Map and the AI Mapper reads every field name on both sides - the source schema and the target schema - and proposes which source field should connect to which target field, ranked by confidence. You review the suggestions, accept the ones that are correct, reject the ones that aren't, and the mappings are written directly into your BW process as XPath/XSLT expressions, ready to deploy.What sets the AI Mapper apart from its predecessor, the legacy Smart Mapper, is that it genuinely understands meaning not just spelling. It knows that dob and birthDate refer to the same concept, that routingNumber and sortCode are the same thing in different countries. It does this through a two-stage AI pipeline that combines large-language-model embeddings with a local ONNX-based cross-encoder reranker, all running inside the Studio IDE with no external service dependency at scoring time beyond the initial embedding call.

This article covers everything - what it is, how it works step by step, how to set it up, how to get the most out of each operating mode, what the real-world accuracy looks like across various tested industry scenarios, and the technical architecture underneath for those who want to understand it.

💡 Note

AI Mapper- Online LLM(Tech Preview) support in BW 6.12.0 HF3 is currently in Tech preview mode. The functionality is intended for evaluation and demonstration purposes, and users should be aware that APIs or behavior may change in future releases.By using this Tech Preview, you acknowledge and agree that:

• It is provided "as is" with no warranties, express or implied.

• It may not be deployed in production environments.

• You are responsible for ensuring that any data processed complies with all applicable laws, regulations, and thirdparty rights.

• Use of this software to analyze content or intellectual property of Cloud Software Group is strictly prohibited.

AI Mapper at a Glance

The infographic below provides a high-level overview of the AI Mapper architecture, execution modes, recommendation workflow, learning capabilities, and overall mapping process described throughout this article. The subsequent sections explore each of these concepts in greater detail.

Why It Matters: The Mapping Problem at Scale

Schema mapping is one of the most underestimated costs in enterprise integration. A typical REST integration between an ERP and a fulfillment system might involve 150 fields on each side - even if 80% of those can be matched by exact name, the remaining 20% (30 fields) require domain knowledge to connect correctly. At 2–3 minutes per ambiguous field, that's an hour or more of skilled developer time on a single integration. Multiply across dozens of integrations in a large project and across an enterprise portfolio of hundreds, and the total cost is significant.

Beyond time, manual mapping is fragile. A developer who doesn't know banking terminology might not know that SWIFT and BIC are the same identifier. A developer unfamiliar with US healthcare abbreviations might not connect SSN to socialSecurityNumber. These gaps create silent data errors that surface only at runtime often in production.

The AI Mapper addresses this directly by:

• Reducing time-to-mapping - a schema with 200 target fields that would take a developer 3–4 hours to map manually can be auto-mapped in under a minute, with 60–90% of mappings correct on first pass

• Encoding domain knowledge - the OpenAI embedding model has been trained on a vast corpus that includes financial, healthcare, retail, HR, and IoT terminology; it knows SWIFT = BIC, SSN = socialSecurityNumber

• Handling naming convention differences - camelCase, snake_case, PascalCase, abbreviations, unit suffixes, and acronyms are all normalized before comparison

• Learning from your corrections - every mapping you confirm or correct is stored and used to improve future auto-mapping runs, both within your session and (optionally) shared across your team

From Smart Mapper to AI Mapper : A Brief History

The predecessor to the AI Mapper is the Smart Mapper, which has been part of BW6 Studio for several releases. The Smart Mapper uses weighted heuristics: it splits field names by camelCase and underscore boundaries, looks up each word in a bundled precomputed GloVe-style word-vector dictionary & computes cosine similarity between source and target vectors and blends te score in final mapping. This approach works well for common English words and conventional naming. For e.g orderDate → placedAt succeeds because the vectors for "order date" and "placed at" are reasonably close.

What the legacy Smart Mapper cannot do is resolve synonyms that require domain context or training beyond a general word-vector corpus. It lacks in things like physics-domain synonyms or the financial acronym equivalencies that require financial-domain training data. For example socialSecurityNumber → ssn fails because the conceptual link between "social Security Number" and "ssn" is a domain interpretation not a lexical similarity.

In BW 6.12.0 HF3, the Smart Mapper was extended with the AI Mapper, which wraps the legacy engine with a two-stage AI pipeline:

1. Remote embedding via OpenAI's API - field descriptions are sent to OpenAI's text-embedding-3-small model (or another configured model), which converts them into high-dimensional semantic vectors. Fields with the same meaning even if they look nothing alike get similar vectors.

2. Local ONNX cross-encoder reranking - for the top-5 cosine candidates per target field, a local quantized model (ms-marco-MiniLM-L-6-v2, 23 MB) jointly reads both field descriptions together and produces a precise relevance score. This runs entirely on your machine; no data leaves after the embedding call.

Three significant changes accompany this architecture:

• On-premises inference: ONNX scoring happens locally with no latency cost beyond the initial model download

• Dual-mode operation: "online" mode calls OpenAI for embeddings and uses ONNX for reranking; "offline" mode falls back to the legacy word-vector engine (no API call, air-gap compatible)

• Persistence layer: an embedded Apache Derby database (or optional remote database) accumulates user-confirmed mappings as a recommendation store, enabling reinforcement learning across sessions and teams

How It Works: The Four-Step Pipeline

Here is what the AI Mapper does when you trigger AI Map in online mode

Step 1 - Read and Parse Every Field Name

The AI Mapper reads every field in your source schema and every field in your target schema. It doesn't just read the raw name - it tokenizes it. streetLine1 becomes {street, line, 1}. legalFirstName becomes {legal, first, name}. dateOfBirth becomes {date, of, birth}. It handles camelCase splits, acronym splits , underscore separators, and alpha-to-numeric boundaries - all automatically.

For each field, it also reads the parent path (so it knows whether a field called name is inside shippingAddress or billingAddress), the data type (string, decimal, boolean, etc.), and whether the field repeats (array vs single value). All of this becomes the field's "description."

Step 2 - Turn Descriptions into Meaning

Each field description is sent to an AI language model - OpenAI's embedding API by default - that converts it into a list of hundreds of numbers called an embedding. Think of it like a GPS coordinate system for meaning: fields that mean the same thing end up close together on this meaning-map, even if the words are completely different. dateOfBirth and birth_date land in almost exactly the same spot. routingNumber and sortCode land close together because the model has been trained on text that uses both terms for the same banking concept.

This happens for every source field and every target field, using the full description (name + path + type + cardinality), not just the bare name.

Step 3 - Find the Best Match and Double-Check It

For each target field, the system performs a fast nearest-neighbor search in the meaning space to find the top-5 source fields whose meaning-coordinates are closest (cosine similarity). This is fast but coarse - it might return five plausible candidates without knowing which one is truly best.

The top-5 candidates then go through a second, more precise AI model: a 23 MB cross-encoder model (ms-marco-MiniLM-L-6-v2) running locally on your machine using ONNX Runtime. This model reads both field descriptions together - the target description and each candidate source description as a pair and produces a relevance score for each. Unlike the embedding model, which scores each field in isolation, the cross-encoder understands the full context of both sides simultaneously, like a human expert comparing the two. The highest-scoring candidate is selected as the proposed mapping.

Step 4 - Apply Mappings and Show Suggestions

The proposed mappings are written into the Studio mapper as XPath/XSLT expressions. For repeating fields (arrays or lists), the AI Mapper automatically wraps the mapping in an xsl:for-each loop - you don't have to configure this manually.

The top-3 candidates with confidence percentages are also stored per field and are accessible via right-click → "Recommended Mappings" dialog. This lets you review close calls and substitute the second or third choice if the AI's first choice was wrong. When you do this, the system records your correction.

Three Modes of Operation

The AI Mapper operates in three distinct modes, selected through Window → Preferences → BusinessWorks → Mapper → AI Mapper.

Mode 1: AI Online (Tech Preview)

What it does: Calls OpenAI to embed all source and target field descriptions, runs ONNX cross-encoder reranking locally, applies user-override scoring from the learning database.

Best for: Complex schemas with hard synonyms, domain-specific terminology, heavy use of abbreviations, cross-industry vocabulary mapping (e.g., US banking to UK banking terms, EMR to registration portal field names, physics units to full scientific names).

Requires: OpenAI API key configured in BusinessWorks AI preferences; ONNX model files downloaded (automatic on first use, or manual pre-download for air-gapped environments); internet connectivity for the embedding call.

Trade-off: Field names are sent to OpenAI's API - a data-governance consideration. Only field names, paths, types, and cardinalities are sent; no actual data values.

Mode 2: AI Offline (Local)

What it does: Uses the same legacy word-vector scoring algorithm as the Smart Mapper but with the streamlined AI Mapper UI - no parameter wizard, full-schema depth , learning always on.

Best for: Air-gapped environments; integrations where field names follow simple conventions (camelCase, snake_case, direct abbreviations); cases where OpenAI access is not available or not approved.

Requires: No external services. The precomputed GloVe-style word-vector dictionary is bundled with the product.

Limitation: Cannot resolve hard domain synonyms (socialSecurity→ ssn). These require semantic understanding that goes beyond word-vector cosine similarity.

Important clarification: AI Offline mode uses the same scoring algorithm as Legacy Smart Mapper. What differs is the wrapper: no parameter wizard, default depth (vs. user-chosen depth), and learning always active. If you need smarter matching, you need Online mode.

Mode 3: Legacy Smart Mapper

What it does: The original Smart Mapper with full parameter exposure - depth sliders, weight tuning, explicit enable/disable.

Best for: Teams who need precise control over matching parameters; scenarios where the simpler algorithm is preferred for auditability.

JVM flag: -Dcom.tibco.bw.mapper.useLegacySmartMapper=true - bypasses the AI Mapper infrastructure entirely. This is configured in TIBCOBusinessStudio.ini file under vmargs

Mode comparison table:

The Learning Loop : Getting Smarter Over Time

One of the most powerful features of the AI Mapper is its persistent learning system. Every time you confirm a mapping either by accepting a suggested one or by manually drawing a connection the AI Mapper records the pairing and uses it to boost that same pairing in future runs.

How the Recording Works

When you accept a mapping from the Recommended Mappings dialog or manually create a mapping in the editor, the AI mapper

• Stores the pairing in memory as "targetFieldName targetTypeName" → "sourceFieldName sourceTypeName"

• Persists it to the database immediately (Apache Derby embedded by default, or a remote database for team-wide sharing)

The key is deliberate schema-agnostic aspect where it stores only the field name and type, not the full schema path. This means a confirmed pairing( for e.g managerId → reportsTo) applies as a recommendation boost for any future mapping activity that contains a field called managerId targeting a field called reportsTo, regardless of the surrounding schema structure.

How the Boost Works at Runtime

On the next auto-map run, AI mapper reads all confirmed pairings from the database. In AI Online mode, if a confirmed pairing is found for the current target, the override source is used. The override source always wins, but the AI's top-3 candidates remain visible in the recommended mappings dialog , so you can still see what the AI would have chosen and switch if the context is different.

Local Derby vs Shared Database

By default, confirmed pairings are stored in an embedded Apache Derby database inside the Studio workspace. This means your learning accumulates locally across your Studio sessions.

For team-wide sharing where one developer's corrections become everyone's recommendations you can enable "Use a remote database for shared mapping recommendations" in AI Mapper preferences and configure a DB connection. Once enabled, all confirmed pairings from all developers go to the shared database, and all developers benefit from each other's corrections on the next auto-map run.

To reset learning and start fresh, right-click in the mapper editor and choose "Reset to Default Recommended Mappings". This clears both the in-memory cache and the database.

Setup and Configuration

Prerequisites

• BW 6.12.0 HF3 or later

• OpenAI API key (for AI Online mode only)

• ONNX model files — downloaded automatically on first use, or manually for air-gapped environments

Step-by-Step Setup for AI Online Mode

Step 1 : Enable BW6 Design Assistant(Tech Preview). Go to Window → Preferences → BusinessWorks AI & check the option as shown below

Click Apply and Close. You will be presented with below dialog

Click OK.

Step 2 : Configure OpenAI key. Go to Window → Preferences → BusinessWorks AI → AI Mapper Models. Add a model using your OpenAI API key. Select an embedding model (e.g., text-embedding-3-small).

Click Apply and Close.

Step 3: Enable AI Online Mode. Go to Window → Preferences → Mapper → AI Mapper → Mapping Engine

• Enable AI Mapper: Online LLM (Tech Preview). Default is Local

• Embedding model: Select the model you configured in Step 2

Step 4 : Configure ONNX assets

Still in Preferences → Mapper → AI Mapper, under "Reranker models location" You have two options:

• Download mode (recommended): AI Mapper will automatically download model_quantized.onnx and tokenizer.json from HuggingFace on first use. Specify a download folder, or leave blank to use the user home default directory.

• Existing assets mode: if you have pre-downloaded the files (for air-gapped environments or corporate proxy restrictions), point to the folder containing both files.

The download is idempotent.If the files already exist and are non-empty, no re-download occurs.

Step 5 : Configure the recommendation database (optional)

For team-wide learning, enable "Use a remote database for shared mapping recommendations" and configure your DB connection details. Otherwise, leave this at the default (local Derby).

Step 6 : Test with a simple process

Create or open a BW process with 2 mapper activity with 2 different schemas. In the mapping editor, drag Source tree to target tree on a target element and select "AI Map." You should see the progress dialog appear (first indeterminate, then switching to "Mapping N of M nodes..." once the target embedding is complete). On first run, the ONNX model is downloaded in the background.

Demo Scenarios : Seeing It In Action

The sample project included here AIMapperOnlineSample.zip showcases demo scenarios covering a range of industries and mapping challenges. Each scenario is a BW process with two Mapper activities(Source & Target) whose schemas are ready to try. The mapper activities are preconfigured with the schema pairs used in each scenario, but no mappings have been implemented yet. This allows you to execute the AI Map Drop Wizard and observe how different options generate mappings & recommendations .

The screenshots below shows the imported project in Business Studio, where the process names correspond directly to the Smart Mapper options being demonstrated.

NOTE: After importing the project, design-time validation errors may appear in Studio.These errors are expected and can be safely ignored, because the mapper activities intentionally do not contain mappings yet. The scenarios are designed specifically to demonstrate how each AI Mapper option generates mappings during the scenarios described later in this article.

How to Use the Demo Project

1. Import the archive into BW6 Studio (File → Import → General → File System)

2. Open any process to see the paired mapper activities with corresponding schemas

3. Select the Target mapper activity and open its input configuration in the properties panel.

4. In the Data Source panel, locate the root data element (for example, the source schema root). Drag the source root element and drop it onto the corresponding target element root in the mapping panel.

5. Choose AI Map option and click Finish. You should see the progress dialog appear (first indeterminate, then switching to "Mapping N of M nodes..." ). Wait for completion.

6. To see recommendations, right-click any mapped target element → Recommended Mappings

Below is a walkthrough of each scenario, including what capabilities it exercises and observed real-world results.

Scenario 1 : IoT Sensor Telemetry

Source: An IoT gateway emits SensorReading with abbreviated field names.

Target: An analytics platform consumes Measurement with full scientific names.

Description: A sensor device sends readings with heavily abbreviated field names (ts, tempC, humPct, co2Ppm, lux). An analytics platform expects full scientific names (recordedAt, temperature, humidity, carbonDioxide, illuminance)

Here is how Source & Target schemas look:

Result:

Key mappings to observe

Comparision with Pre-HF 3 legacy SmartMapper

This IoT Sensor Telemetry scenario is the sharpest possible illustration of what the AI Mapper actually delivers because it is precisely the kind of schema where the pre-HF3 Smart Mapper fails catastrophically(with default tuning params), not just partially as seen below.While Smart Mapper can sometimes be improved through multiple iterations of parameter tuning, threshold adjustments, and manual refinement, achieving acceptable results often requires significant trial and error. In contrast, the AI Mapper(Online) is able to identify these semantic relationships with minimal or no tuning.

Scenario 2 : Healthcare Patient Record

Source: A hospital EMR exports PatientRecord in camelCase.

Target: A registration portal consumes RegistrationForm in snake_case with clinical abbreviations.

Description: A hospital EMR exports patient data in camelCase (dateOfBirth, genderCode, socialSecurityNumber). The registration portal uses snake_case (birth_date, sex, ssn). The genderCode → sex mapping is the hardest one here - the words look completely different but mean the same thing.

Here is how Source & Target schemas look:

Result:

Key mappings to observe

The only single error was a phone number source swap between the patient's own phone and the emergency contact's phone both named phone with similar contexts.

Learning Override Step

Manually correct next_of_kin/phone → emergencyContactDetails/PhoneNumber using Right Click -> Recommended mappings dialog and choosing the second option.

On the next auto-map for any other activity in the same Studio session, the user-confirmed pairing gets weightage. You can delete the mapping and redo it using AI Map again & confirm

Scenario 3 : E-Commerce Order Processing

Source: A retail OMS

Target: A warehouse fulfillment system expecting FulfillmentOrder.

Description: An order management system sends an order, and a warehouse fulfillment system receives it. The fields represent the same business data but use different terminology, such as firstName vs givenName, city vs municipality, and quantity vs qty. Order lines are represented as a repeating list.

Here is how Source & Target schemas look:

Result:

Key mappings to observe

Scenario 4 : Type-Aware Matching

Source: A generic business record produced by an upstream application.

Target: A target entry structure consumed by a downstream system.

Description: A Record source with mixed types mapping to an Entry target with different type assignments.Both schemas contain fields with similar names, but not all fields represent the same type of data.

Here is how Source & Target schemas look:

Result:

Key mappings to observe

The AI Mapper evaluates both semantic similarity and data types when generating mappings. For example, isActive maps confidently to active because both are boolean fields with similar meaning. This example demonstrates that the AI Mapper considers data type compatibility in addition to field names when determining mappings.The code field demonstrates an explicit type conversion during mapping. Even though the source value is a string, the mapper applies an xsd:int() cast to convert it into an integer in the target schema. This ensures type compliance on the target side.

Scenario 5 : Name Ambiguity

Source: A UserProfile system

Target: A Contact system expecting a single name field

Description: A UserProfile contains multiple candidate fields representing a person’s name in different contexts, including fullName, displayName, and loginName. The target Contact schema expects a single name field, which introduces ambiguity in mapping.

Here is how Source & Target schemas look:

Result:

Key mappings to observe

The AI Mapper resolves this by ranking candidate fields based on semantic similarity and contextual relevance. For example, fullName is typically ranked highest due to direct semantic equivalence, followed by displayName, and then loginName, which is less formal and more system-oriented.

Scenario 6 : Deep & Wide Schema

Source: A travel booking system (PNR-based reservation data)

Target: A travel reservation system expecting a normalized Reservation structure

Description: A Passenger Name Record (PNR) contains deeply nested travel data including booking information, passengers, flight segments, and baggage details. The target Reservation system expects a structured reservationRecord with normalized entities such as travelers and flightLegs.

Here is how Source & Target schemas look:

Result:

Key mappings to observe

Despite the depth and repetition in the source schema, the AI Mapper processes the structure efficiently by batching embeddings and handling hierarchical nodes in chunks. Repeating structures such as passengers and segments are mapped into corresponding repeating target structures like travelers and flightLegs.The system is optimized for deep hierarchies and uses pre-embedded target caching and chunked embedding with retry logic to maintain performance and reliability. A progress dialog ("Mapping m of n nodes…") is displayed during execution to provide user visibility into long-running mapping operations, demonstrating that the process is scalable and non-blocking even for large, multi-level schemas.

Scenario 7 : Extreme Scale: 10,000-Node Enterprise Schema

Source: EnterpriseDataPlatform (10,000-node enterprise master schema representing a full organizational data model).A 775 KB, 11,632-line schema with around 10000 nodes representing 50 enterprise domain sections (Customer, Account, Order, Product, Shipment, Invoice, Employee, Asset, Regulatory, and more).

Target: A smaller domain-specific operational schema (e.g., Order Fulfillment), containing only a subset of entities such as buyers, financials, fulfillment, offers, and payments

Description: The source is a highly complex, monolithic enterprise schema consisting of ~10000 elements across 50 business domains including Customer, Order, Product, Invoice, Shipment, Compliance, Risk, and more. Each domain is further divided into Core, Extended, Reference, Operational, and Analytical sections, resulting in deeply nested structures with high redundancy and variation in naming conventions.The target schema is significantly smaller(around 150 Nodes) and represents a focused operational model (for example, order fulfillment ) containing only essential business entities and flattened structures.

Here is how Source & Target schemas look:

Result:

Key mappings to observe

• The progress bar starts indeterminate (target embedding in progress)

• Switches to a determinate "Mapping N of M nodes..." bar once target count is known

• Advances one tick per processed node as the mapping proceeds

Think about what a human mapper would face: 10,000 fields to read, understand, and pair one by one. At a generous pace of 30 seconds per field, that is 83 hours of manual work. Even with a small schema of 200 fields, most integration developers budget a full day or more just for the mapping step.

The AI Mapper does the same job in under a minute. You click AI Map, watch the progress bar sweep across the screen, and the entire schema is wired up before you have had a chance to make a coffee. The speed comes from three overlapping efficiencies working together:

• All target fields are embedded in a single batch before the mapping starts — one call to the OpenAI API sends all field descriptions at once rather than one per field.

• The local ONNX model runs in parallel across up to four CPU cores simultaneously, processing up to 512 field pairs per inference pass.

The result is a complete, reviewed, deployment-ready mapping for a 10,000-node enterprise schema in the time it would take a developer to open the file

Data Privacy and Security Considerations

What Is Sent to OpenAI

In AI Online mode, the following information is sent to the OpenAI embedding API for each schema field:

• Field's local element name (e.g., dateOfBirth, routingNumber)

• Ancestor path chain (e.g., personal inside patient record)

• Data type name (e.g., string, decimal, boolean)

• Cardinality (e.g., repeating, only-one)

What is NOT sent: Actual data values, runtime payloads, process logic, activity configuration, or credentials of any kind. Only schema metadata — field names and types — is transmitted.

What Stays Local

• All ONNX cross-encoder scoring runs locally on your machine

• The recommendation database (Derby or remote DB) stays within your infrastructure

• The bundled word-vector dictionary (offline mode) is entirely local

• The process files and all generated XPath expressions are local

  If your organization's policy prohibits sending schema metadata (field names) to external services, use AI Offline mode or Legacy Smart Mapper. Both are fully air-gapped and produce no external network calls.

Demo Recording

In addition to this article, a short video recording is available that walks through the demo and highlights some of the key features in AI Mapper :

Summary

The AI Mapper in TIBCO BusinessWorks 6.12.0 HF3 is a genuinely useful addition to the integration developer's toolkit. Its two-stage architecture - OpenAI embeddings for semantic recall, local ONNX cross-encoder for precise reranking brings real AI capability to the routine but expensive task of schema mapping.The learning database is a multiplicative force: every manual correction you make today improves tomorrow's auto-mapping, and with the shared Database option, every correction from any team member benefits everyone.