Transformation Rules Specification

A transformation rule contains a source field and target field string. The source field references the json field tag (dot notation) for the source json object, and the target field references the json field in the destination object model. A rule is valid if it satisfies the following properties for the source and destination objects:

• The source field type matches the target field type post enumeration. Take the example objects below. The transformation rule orderKey to orderNumber would not be valid because the types are not the same (number & string respectively). A valid rule would be orderKey to orderId, or orderNum to orderNumber . We also support nested object transformation rules as long as their enumerated cardinality does not have a conflict. An example of a valid nested rule would be orderKey to shippingAddress.orderId since there exists a 1:1 correspondence between these two fields for N amount of source objects ingested.

  // example source object
  {
      "orderKey": 123,
      "orderNum": "ABC",
      "items": [
          {
              "id":    1,
              "sku": "A",
              "requiresShipping": true,
              "qty": 2,
              "itemTags": ["light", "fragile"]
          },
          {
              "id":    2,
              "sku": "B",
              "requiresShipping": true,
              "qty": 1,
              "itemTags": ["heavy"]
          },
          {
              "id":    3,
              "sku": "C",
              "requiresShipping": false,
              "qty": 1,
              "itemTags": ["flammable"]
          }=
      ],
      "otherItems": [
          {
              "id": 1,
              "name": "name1"
          }
      ]
      "shipping": {
          "city": "Mesa",
          "state": "AZ",
          "zip": 85213
      }
  } 
  

  // example destination object
  {
      "orderId": 11111,
      "orderNumber": "ABC",
      "orderItems": [
          {
              "id":    1,
              "productId": "ABC",
              "requiresShipping": true,
              "quantity": 5,
              "price": 55.60,
              "tagName": "tag",
              "itemTags": []
          }
      ],
      "shippingAddress": {
          "orderId": 11111,
          "city": "ABQ",
          "state": "New Mexico",
          "zipCode": 63245,
          "addressLine1": "123 Lane",
          "country": "United States"
      }
  }
  

  • enumeration is when a list field is provided and the resulting value is a list. These elements must be enumerated to the target field which must also be a list.

• The cardinality of the source must always be equal to cardinality of the target field. Otherwise, there would not be a 1:1 correspondence between the elements in source to target. Take the example above. A transformation cannot be applied from orderKey → orderItems.id because the cardinality for orderId is 1 for the source and N for the destination. The same is true for orderItems.sku → orderNumber as it would be N to 1. In contrast, a valid transformation rule can be made for orderKey → orderId because they have equal cardinalities for the object and a 1:1 correspondence can be made. If the cardinalities for enumerations are equal while testing a transformation rule in an integration, ensure that the cardinalities are equal for every instance of the source events.

• In the event that multiple transformation rules which are targeting the same list parent object, but the cardinalities of the source lists are different, the amount of elements in the largest source list will be the amount in the destination target list and any remaining items that cannot be transformed due to lack of elements will be null. Take the example above, the following transformation rules:

  • items.sku → orderItems.productId
  • otherItems.name → orderItems.tagName

  would result in the following orderItems:

  "orderItems": [
      {
          "id":    null,
          "productId": "A",
          "requiresShipping": null,
          "quantity": null,
          "price": null,
          "tagName": "name1"
      },
      {
          "id":    null,
          "productId": "B",
          "requiresShipping": null,
          "quantity": null,
          "price": null,
          "tagName": null
      },
      {
          "id":    null,
          "productId": "C",
          "requiresShipping": null,
          "quantity": null,
          "price": null,
          "tagName": null
      }
  ]