Data transformers

Data transformers can be configured as part of an entity data transformer or an entity node transformer.

Data transformers do not have access to the entity type so they can only perform on the data level. Most data transformers use !! accessors !! to access the data.

Transformer types

The following data transformers are available by default.

When additional logic is required to correctly transform data, !! a new data transformer !! can be created.

Value mapper

Code: value-mapper

Maps values using a !! mapper !!.

In the following example, a mapper is used to make the SKU of a product uppercase.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "value-mapper",
"parameters": {
"mappers": [
{
"prototype": "string-upper"
}
],
"accessor": {
"prototype": "key",
"parameters": {
"keys": [
"sku"
]
}
}
}
}
]
}
}

Input:

{
"sku": "tnt"
}

Output:

{
"sku": "TNT"
}

Value filter

Code: value-filter

Filters values using a !! condition !!.

In the following example, the enabled attribute is moved when it is not a boolean.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "value-filter",
"parameters": {
"accessor": {
"prototype": "key",
"parameters": {
"keys": [
"enabled"
]
}
},
"conditions": [
{
"prototype": "logic-not",
"parameters": {
"conditions": [
{
"prototype": "type-boolean"
}
]
}
}
]
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true
},
{
"sku": "dynamite",
"enabled": false
},
{
"sku": "timer",
"enabled": 1
}
]

Output:

[
{
"sku": "tnt",
"enabled": true
},
{
"sku": "dynamite",
"enabled": false
},
{
"sku": "timer"
}
]

Key mapper

Code: key-mapper

Maps keys using a !! mapper !!.

The following shows how to translate keys in different structures.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "key-mapper",
"parameters": {
"mappers": [
{
"prototype": "standard",
"parameters": {
"to": "active",
"from": "enabled"
}
}
],
"accessor": {
"prototype": "key",
"parameters": {
"keys": [
"enabled"
]
}
}
}
},
{
"prototype": "key-mapper",
"parameters": {
"mappers": [
{
"prototype": "dictionary-map",
"parameters": {
"map": [
[
"color",
"label"
],
[
"yield",
"power"
]
]
}
}
],
"accessor": {
"prototype": "structure",
"parameters": {
"key": "attribute_code",
"root": "custom_attributes",
"value": "value"
}
}
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "color",
"value": "red"
},
{
"attribute_code": "yield",
"value": 1200
}
]
},
{
"sku": "dynamite",
"enabled": false
}
]

Output:

[
{
"sku": "tnt",
"custom_attributes": {
"2": {
"value": "red",
"attribute_code": "label"
},
"3": {
"value": 1200,
"attribute_code": "power"
}
},
"active": true
},
{
"sku": "dynamite",
"active": false
}
]

Key filter

Code: key-filter

Filters keys using a !! condition !!.

The following filters keys that are not in a list of allowed keys.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "key-filter",
"parameters": {
"accessor": {
"prototype": "structure",
"parameters": {
"key": "attribute_code",
"root": "custom_attributes",
"value": "value"
}
},
"conditions": [
{
"prototype": "not-one-of",
"parameters": {
"value": [
"yield",
"color"
]
}
}
]
}
},
{
"prototype": "value-filter",
"parameters": {
"accessor": {
"prototype": "key",
"parameters": {
"keys": [
"custom_attributes"
]
}
},
"conditions": [
{
"prototype": "empty"
}
]
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "color",
"value": "red"
},
{
"attribute_code": "yield",
"value": 1200
},
{
"attribute_code": "weight",
"value": 12
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "color",
"value": "red"
},
{
"attribute_code": "yield",
"value": 1200
}
]
},
{
"sku": "dynamite",
"enabled": false
}
]

List mapper

Code: list-mapper

Maps a list of keys and values using a list-* !! mapper !!.

The following example uses a list mapper to add default attributes to a product.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "list-mapper",
"parameters": {
"mappers": [
{
"prototype": "list-inherit-from",
"parameters": {
"array": {
"certified": false
}
}
}
],
"accessor": {
"prototype": "structure",
"parameters": {
"key": "attribute_code",
"root": "custom_attributes",
"value": "value"
}
}
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"value": false,
"attribute_code": "certified"
}
]
}
]

Chain transformer

Code: chain

Chains multiple transformers into one. Chaining a transformer can be beneficial when sharing configuration, like an !! accessor !!.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "chain",
"parameters": {
"accessor": {
"prototype": "structure",
"parameters": {
"key": "attribute_code",
"root": "custom_attributes",
"value": "value"
}
},
"transformers": [
{
"prototype": "list-mapper",
"parameters": {
"mappers": [
{
"prototype": "list-inherit-from",
"parameters": {
"array": {
"certified": false
}
}
}
]
}
},
{
"prototype": "key-mapper",
"parameters": {
"mappers": [
{
"prototype": "string-upper"
}
],
"accessor": {
"prototype": "key",
"parameters": {
"keys": [
"weight"
]
}
}
}
}
]
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": {
"1": {
"attribute_code": "certified",
"value": true
},
"2": {
"value": 12,
"attribute_code": "WEIGHT"
}
}
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": {
"1": {
"value": false,
"attribute_code": "certified"
},
"2": {
"value": 12,
"attribute_code": "WEIGHT"
}
}
}
]

Conditional transformer

Code: conditional

Applies a !! condition !! to a transformer as a !! data-filter !! applies to the data.

The example shows how products inherit default custom attributes, but only if the product is marked as enabled.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "conditional",
"parameters": {
"filters": [
{
"prototype": "value-condition",
"parameters": {
"accessor": {
"prototype": "key",
"parameters": {
"keys": [
"enabled"
]
}
},
"conditions": [
{
"prototype": "equals",
"parameters": {
"value": true
}
}
]
}
}
],
"transformers": [
{
"prototype": "list-mapper",
"parameters": {
"mappers": [
{
"prototype": "list-inherit-from",
"parameters": {
"array": {
"tested": false,
"certified": false
}
}
}
],
"accessor": {
"prototype": "structure",
"parameters": {
"key": "attribute_code",
"root": "custom_attributes",
"value": "value"
}
}
}
}
]
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
},
{
"value": false,
"attribute_code": "tested"
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Accessor move

Code: accessor-move

Moves values between two !! accessors !!. This can be used to convert from one structure to another structure.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "accessor-move",
"parameters": {
"source": {
"prototype": "key",
"parameters": {
"keys": [
"enabled"
]
}
},
"destination": {
"prototype": "structure",
"parameters": {
"key": "attribute_code",
"root": "custom_attributes",
"value": "value"
}
}
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
},
{
"value": true,
"attribute_code": "enabled"
}
]
},
{
"sku": "dynamite",
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"value": false,
"attribute_code": "enabled"
}
]
}
]

Accessor copy

Code: accessor-copy

Copies values between two !! accessors !!. This can be used to convert from one structure to another structure.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "accessor-copy",
"parameters": {
"source": {
"prototype": "key",
"parameters": {
"keys": [
"enabled"
]
}
},
"destination": {
"prototype": "structure",
"parameters": {
"key": "attribute_code",
"root": "custom_attributes",
"value": "value"
}
}
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
},
{
"value": true,
"attribute_code": "enabled"
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"value": false,
"attribute_code": "enabled"
}
]
}
]

Pattern move

Code: pattern-move

Moves data matched with a !! pattern !!, to a configured !! replacement !!.

The following example will rename the attribute_code inside custom_attributes to code.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "pattern-move",
"parameters": {
"pattern": "custom_attributes.*.attribute_code",
"replacement": "custom_attributes.$1.code"
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"value": 12,
"code": "weight"
},
{
"value": true,
"code": "certified"
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"value": 12,
"code": "weight"
}
]
}
]

Pattern copy

Code: pattern-copy

Copies data matched with a !! pattern !!, to a configured !! replacement !!.

The following example will copy the attribute_code inside custom_attributes to code.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "pattern-move",
"parameters": {
"pattern": "custom_attributes.*.attribute_code",
"replacement": "custom_attributes.$1.code"
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12,
"code": "weight"
},
{
"attribute_code": "certified",
"value": true,
"code": "certified"
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12,
"code": "weight"
}
]
}
]

Group records

Code: group-by

Group records using a pattern, path and destination.

The following example groups custom_attributes by their attribute_code.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "group-by",
"parameters": {
"path": "attribute_code",
"pattern": "custom_attributes.*",
"destination": "grouped_attributes"
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "weight",
"value": 14
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "weight",
"value": 14
},
{
"attribute_code": "certified",
"value": true
}
],
"grouped_attributes": {
"weight": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "weight",
"value": 14
}
],
"certified": [
{
"attribute_code": "certified",
"value": true
}
]
}
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
],
"grouped_attributes": {
"weight": [
{
"attribute_code": "weight",
"value": 12
}
]
}
}
]

Expand path transformer

Code: expand-path

By expanding the path, its key can be re-used as a value within its node. This can be useful to create line numbers when a set order is required.

The following example will give all custom_attributes nodes a property number, holding the value attr- followed by the key of the custom_attributes node.

{
"prototype": "data",
"parameters": {
"transformers": [
{
"prototype": "expand-path",
"parameters": {
"value": "attr-$1",
"source": "custom_attributes.*",
"destination": "custom_attributes.$1.number"
}
}
]
}
}

Input:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
},
{
"attribute_code": "weight",
"value": 14
},
{
"attribute_code": "certified",
"value": true
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12
}
]
}
]

Output:

[
{
"sku": "tnt",
"enabled": true,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12,
"number": "attr-0"
},
{
"attribute_code": "weight",
"value": 14,
"number": "attr-1"
},
{
"attribute_code": "certified",
"value": true,
"number": "attr-2"
}
]
},
{
"sku": "dynamite",
"enabled": false,
"custom_attributes": [
{
"attribute_code": "weight",
"value": 12,
"number": "attr-0"
}
]
}
]