Connect processes raw data (i.e. unfiltered, as received from a source) in pre-programmed ways to standardise behaviour across all connectors. Part of this process is that some data will be discarded, and other data will be transformed. However, if you want to set up special data rules, it's often necessary to access the raw data rather than the standardised data. You can use this raw data to transfer it into standardised fields or to store it in custom meta fields for further modification.
In this article, you'll find the following:
How do I find the raw data?
Raw data is included with every data object that Connect receives:
- Contacts
- Contact Lists
- Organisations
- Orders
- Products
- Checkouts
- Shipments
Raw data is visible in the object detail view. We show this for Contacts in the example below, but it applies to all data types.
Step 1: Go to the Contact overview
Select the contact overview under 'data' in your dashboard menu, then click any contact to view its data.
Step 2: Go to the list view
Switch to the contact list view by toggling the top-right icon.
Step 3: scroll down for raw (original) data
Scroll down through the list. Below the list, you will find a code format containing all the raw data for that contact as Connect received it in the last update.
How do you use the raw data to make field names for rules?
Suppose you want to build a rule that stores raw data in a newly specified metadata field for further processing outside Connect. For example, a phone number. For this example, we assume that Connect receives but does not store phone numbers by default (in reality, it does).
To manually save the phone number from the raw data, create a rule with an action that creates a phone number metadata field with the value that's in the raw data. In this process, the rule builder will ask you for a metadata field, an operator and a value field:
- Field: Where the system stores the raw data you want to work with (i.e. the metadata field you want to create).
- Operator: The comparison to find the data, in this situation always '='.
- Value: The field where the (raw) data can be retrieved from (i.e. the field where the phone number is located in the raw data).
Value field
It's easiest to start with the value field. Check the raw data in your contact to see where a phone number is stored. You will find a field 'phoneNumber', nested in the properties data, as you can see in the screenshot in Step 3.
To translate this to a field name in the rule builder, use the prefix 'ctx:Raw' followed by the nested data fields, separated with dots. So for the phone number, the field name would be 'ctx:Raw.properties.phoneNumber'. If you put this in the rule builder value field, Connect will know where to retrieve the data you want to work with.
While this is an example of a nested data field, top-level data fields behave similarly (for example, 'ctx:Raw.externalId' or 'ctx:Raw.properties').
Pay attention to case
These data fields are case sensitive. Make sure to write raw data names as they're displayed in Connect. If the raw data says 'phoneNumber', 'PhoneNumber' will not be recognised.
Metadata field
Next, you can make meta field to store the phone number from the raw data. Metadata fields can be created at will, as long as you use the prefix 'meta.' To follow along with this example, an appropriate metadata field for the raw phone number would be 'meta.phoneNumber'.
The screen capture below shows you how to create the rule from this example in the rule builder.