DataX Formats, Schemas, Identifiers & Limits

Abstract

Describes the ingestion formats, partner identifiers, rate-limited quotas for taxonomy and audience posts, and data retention for audiences for DataX partners. Error code responses are also described in this section.

Ingestion Formats

Partner will call the POST /v1/usermatch endpoint with their assigned OAuth 2.0 credentials.

Partner Match data is shared in one of the following methods:

CSV File With bzip2 Compression

Define Schema in the .csv file header.

Example 1:

Example 2:

Example 3:

HTTP /POST

Note that before uploading your JSON, you’ll need to encrypt it with SHA256. That means, you must convert your email and phone number list to a hash, which you would then place in your file. For example, if the original email is [email protected], the hashed value in the file would be:

d48adb3c108a657adf7597921f3bfc591ee3f00d658d2d288e0bb396ac0d5964

OR

Phone Number: +911112341234
Would have a hashed value of: 8e4b975211195dc6ddc68c4faaad597a715f30dd46e06f6c20e8ebd363105232

Important

Hashed email: Your file name must be properly normalized using lowercase and contain no spaces.

Hashed Phone Numbers: Before SHA-256 hashing phone numbers, here are the normalization rules based on E.164:

• E.164 phone numbers can have a maximum of 15 digits.

• Normalized E.164 phone numbers use the following syntax, with no spaces, hyphens, parentheses or other special characters:

  • [+] [country code] [subscriber number including area code]

  • Examples:

    • US: 1 (123) 456-7890 is normalized to +11234567890.

    • Singapore: 65 1243 5678 is normalized to +6512345678.

    • Sydney, Australia: (02) 1234 5678 is normalized to drop the leading zero for the city plus include the country code: +61212345678.

• Additional external documentation:

  • E.164: The international public telecommunication numbering plan

  • List of country calling codes

Additional Hashed Phone Number Test Dataset

Examples listed here have already been normalized. This is for validating the phone number hashing.

Regular Expression to Validate Normalized Phone Number

Pattern I164_PATTERN = Pattern.compile("^\\+[1-9]\\d{1,14}$");

DataX only supports pre-encrypted files. Once the files are encrypted, all the personal data that resides on the files will be protected, using the SHA256 function, so that no raw emails or phone numbers are ever stored.

If an email address or phone number is not hashed in the proper format, DataX will not process the audience records.

Follow these steps:

1. Define schema parameter during a POST call.

2. Introduce a comma-separated schema parameter that defines the column order and the name values.

3. Schema MUST be defined in the headers of the file itself.

Example

POST /v1/usermatch?schema=PXID,SHA256EMAIL,SHA256PHONE
        Partner ID1, HashedEmail Value 1,HashedPhone Value 1
        Partner ID2, HashedEmail Value 2,HashedPhone Value 2
        Partner ID3, HashedEmail Value 3,HashedPhone Value 3

Notes:

• A Partner has the option to define the schema as they wish.

• There is a priority of the schema defined at different places: schema parameter > schema header in files > default schema.

• If a schema is not defined during ingestion, the default schema format will be PXID|SHA256EMAIL.

Data Validation

When calling the POST /v1/usermatch endpoint, the uploaded bz2 file’s format and content will be validated. The header line and up to the first 10 lines in the content will be checked. If the validation is not passed, the following error code and message will be returned:

DataX Partner Identifier

Rate Limitation

DataX is rate-limited per the following quota limits for taxonomy and audience posts:

Data Retention

Segment Expiration:

• Audiences will be linked to the active segment for 45 days.

• Audience refreshes can be posted at any time (for example, daily, weekly, monthly, etc).

• If audiences are not refreshed by the default expiration TTL, they will be unlinked from the segment(s).

User Match Expiration:

• Partner IDs will be stored in-house for measurement.

• In-house data will be stored/ linked to a Verizon Media ID for 90 days to support audience refreshes.

Error Responses