Transformers

​

Overview

The Transformers package provides pre-configured examples for transforming, processing, and preparing data within Alumio integration flows. These examples show how transformers can be used to modify data structures, apply business logic, validate information, and prepare data before it is sent to another system. Transformers are an essential part of an integration flow because they allow data to be adjusted between different systems with different data structures or requirements. They can be used to split large datasets, transform values, filter records, generate files, organize data, and perform advanced data manipulation. This package includes examples covering common transformation scenarios, from basic data mapping to advanced processing operations. These templates can be used as a starting point and customized based on specific integration requirements.

​

General Information

In Alumio, Transformers are used to process and modify data during an integration flow. They can be added to Incoming Configurations, Routes, or Outgoing Configurations to change the structure, format, or content of data before it continues to the next processing step. The Transformers package shows different transformation capabilities, including:

1. Splitting items from API responses into individual records
2. Writing processed data into files
3. Combining records into batches
4. Filtering data based on conditions and validation rules
5. Performing operations on arrays and values
6. Rendering dynamic templates
7. Updating stored values
8. Grouping related records
9. Moving data between different accessors
10. Mapping and modifying data structures

These examples show how Alumio transformers can be combined to create flexible and reusable data processing workflows.

​

Package Contents

The following Transformer examples are included in the package:

​

1. Split Items

Split Items: Get Items from HTTP Request with Pagination
Shows how to split multiple items retrieved from an HTTP request into individual records that can be processed separately within an integration flow. The package includes three transformer examples using different HTTP pagination approaches:

1. Pagination Follow Next Links in HTTP Header: shows how to retrieve and split items from an HTTP response where the next page reference is provided through HTTP response headers
2. Pagination Increase Query Parameter: shows how to retrieve and split items by increasing a query parameter value for each request, such as page number or offset
3. Pagination Follow Next Link: shows how to retrieve and split items by following the next page reference provided in the HTTP response body

Use cases:

• Processing paginated API responses
• Splitting API response items into individual records
• Creating separate tasks for each item
• Supporting different API pagination strategies

​

2. Write Items

Write Items to a File
Shows how to write processed items into files stored on a local filesystem using the Write Items to a File transformer. The package includes two transformer examples with different file format types:

1. CSV Format: shows how to convert processed data into a CSV file structure and store it on the local filesystem. Commonly used for exporting structured tabular data
2. JSON Format: shows how to convert processed data into a JSON file structure and store it on the local filesystem. Commonly used for exchanging structured data between applications

Use cases:

• Exporting processed integration data
• Generating CSV or JSON files
• Preparing files for external systems or further processing

​

3. Batch Items

Create Batches by Combining Items
Shows how to combine multiple items into a single batch before continuing the integration process. Batching helps optimize data processing by reducing the number of operations when handling large datasets. Use cases:

• Creating batch API requests
• Combining multiple records into one payload
• Improving processing efficiency for large datasets

​

4. Filtering

Filtering Transformer Examples
Shows how to filter data using different filtering methods. Filtering allows you to control which records should continue through the integration flow based on specific conditions or validation rules. The package includes multiple filtering examples:

1. Filter Based on Transformer: shows how to filter data based on the output or result of another transformer
2. Filter by Storage Entities: shows how to filter data by comparing values stored in Alumio storage entities
3. Key Condition: shows how to filter records based on the existence or value of a specific key
4. Not Condition: shows how to exclude records that match a specific condition
5. Validate by JSON Schema: shows how to validate and filter data by checking whether the payload matches a defined JSON schema structure
6. Value Condition: shows how to filter records based on specific field values

Use cases:

• Applying business rules
• Removing unnecessary records
• Validating incoming data
• Processing only matching records

​

5. Operator

Operator Transformer Examples
Shows how to use the Operator Transformer to perform operations on data values and arrays. The package includes examples using different operator types:

1. Array Intersect: shows how to compare two arrays and return only values that exist in both arrays
2. Array Merge: shows how to combine multiple arrays into a single array
3. Array Diff: shows how to compare arrays and return values that are different between them

Use cases:

• Comparing datasets
• Combining collections of data
• Identifying differences between records
• Preparing data structures

​

6. Render Template

Render a Template Transformer
Shows how to generate dynamic content using templates based on incoming data. The transformer allows data to be combined with predefined templates to create formatted output structures. Use cases:

• Creating dynamic payloads
• Generating formatted messages
• Preparing structured content for external systems

​

7. Storage

Update a Storage Entity Transformer
Shows how to use the Storage Transformer to update values in Alumio storage entities during an integration flow. Storage entities allow reusable values to be maintained and accessed during processing. Use cases:

• Updating stored values
• Maintaining integration state
• Saving temporary information
• Reusing data between processing steps

​

8. Grouping

Group Records Transformer
Shows how to use the Grouping Transformer to organize multiple records into groups based on specific fields or conditions. The transformer helps structure related data before further processing or sending it to another system. Use cases:

• Grouping records by attributes
• Creating grouped payload structures
• Preparing aggregated data
• Organizing related items

​

9. Move Data Accessor

Move Data Between Accessors Transformer
Shows how to move data between different locations within an Alumio data structure. The package includes multiple accessor movement examples:

1. Pattern to Children Accessor: moves data from a matching pattern into a children accessor structure
2. Key to Key Accessor: moves data from one key accessor to another key accessor
3. Key to Structure Accessor: moves data from a key accessor into a structured data accessor
4. Pattern to Key Accessor: moves data from a matching pattern into a specific key accessor

Use cases:

• Restructuring payload data
• Moving values between fields
• Preparing data for mapping
• Adjusting payload structures

​

10. Mappers

Mapper Transformer Examples
Shows how to use the Mapper Transformer to modify and transform data values using different mapper operations. The package includes the following mapper examples:

1. List: Split into Chunks: shows how to split a list into smaller groups containing a defined number of items
2. List: Fill List with Value: shows how to add or fill list elements with a specified value
3. List: Flip Keys and Values: shows how to swap keys and values within a list structure
4. String: PCRE Replace: shows how to replace parts of a string using PCRE (Perl Compatible Regular Expression) patterns

Use cases:

• Restructuring list data
• Formatting values
• Replacing text values
• Preparing data for outgoing requests

​

Typical Use Cases

The Transformers package can be used for various integration scenarios, including:

• Transforming API responses before sending them to another system
• Processing large datasets from APIs
• Creating export files
• Preparing batch requests
• Applying validation and filtering rules
• Mapping data between different applications
• Generating custom payload structures
• Managing temporary integration data
• Reorganizing complex data structures

​

Customization

These transformer examples are designed as reusable templates. After importing the package, you can customize:

• Transformation logic
• Input and output data structures
• Filtering conditions
• Mapping configurations
• File formats
• Batch processing rules
• Template content
• Storage values

This allows you to adapt the examples to different business requirements while following Alumio’s recommended transformation patterns.

​

Integration Flow Example

A typical transformation flow works as follows:

1. Incoming Configuration retrieves data from a source system.
2. Transformer processes and modifies the data based on the required logic.
3. Outgoing Configuration sends the transformed data to the target system.
4. Route manages the complete integration process.

By using these transformer templates, you can quickly create reusable data processing workflows and adapt them to different integration scenarios.

​

Template Import

Follow these steps to import a template to Alumio:

1. Log in to your Alumio dashboard.
2. Navigate to More > System > Import/Export

3. Click to browse or drag and drop ndjson file to upload it.

4. Click Import Configurations to start the import.