Product Yahoo Conversion API

The product level integration to the Yahoo Conversion API (CAPI) can be used to send transaction data for campaign measurement, attribution and optimization using Yahoo In-Flight Outcomes (IFO). Transaction data includes product ID, product quantity, dollar amount and other conversion-related attributes.

The end-to-end process for activating against CAPI works as follows:

1. Deliver a Product Catalog via s3 to the Yahoo Partner Data Store (PDS).

2. Create authentication credentials via an OAuth 2.0 connection and Yahoo enables the seat for IFO.

3. Create a Pixel ID and set as the default pixel for IFO Conversions.

4. Send Transaction data via the CAPI spec.

   Transaction Data can be shared in one of the following ways. These endpoints support sending transaction data via CAPI (preferred method) or sending transaction data using multiple methods (for example, online data via DOT JS Pixel (Browser to Server) and/or offline data via CAPI (Server to Server).

   • Conversions via DOT JS Pixel (Browser to Server): Send online transaction data, including product codes for individual transactions, using your existing pixel credentials.

   • CAPI (Server to Server): Send transaction data using your existing pixel credentials to track individual transactions.

5. Work with the Yahoo Account Team to finalize the catalog and attribution rules in the Yahoo DSP.

Deliver Product Catalog Through S3/Partner Data Store (PDS)

Deliver the product catalog to the Yahoo S3 endpoint, which is updated on a recurring basis (preferably daily). This catalog contains a 3-level product hierarchy that users can search and select from the DSP UI when creating IFO rules. It includes the exact product IDs/SKUs provided in the sales data feed, ensuring precise tracking and accurate attribution of product-level sales transactions.

Example Catalog and Yahoo DSP Rule Structure:

Advertiser: Coca-Cola Company

>High Brand: Coca-Cola

>Low Brand: Coca-Cola Zero Sugar

>Product ID: 049000042559

The following section details the high- level architecture and steps to set up delivery of product catalog data to Yahoo.

Architecture and Data Flow

File Format and Contents

File Type: Tab delimited gzip/bzip2 file.

File Schema: Product ID, Product Owner, Product Brand, Product Name, Partner Attributes

Data type: STRING for all columns.

File format and Contents of “_manifest”

The unit of the file size is “byte”.

<.schema size><SPACE><.schema> --- <SPACE> delimited

<file_1 size><SPACE><file_1.csv.bz2>

<file_2 size> <file_2.csv.bz2>

...

<file_n size> <file_n.csv.bz2>

File format and Contents of “<.schema>”

The schema file, <.schema>, could be either of the following two formats:

1. A Pig header file, ”.pig_header”, in CSV column header format with extended data type support:

   • <column_header:data_type>[,<column_header:data_type>]{*}

   • Supported data types: http://pig.apache.org/docs/latest/basic.html#data-types

   • https://pig.apache.org/docs/r0.17.0/api/constant-values.html#org.apache.pig.data.DataType

2. The standard Apache Pig schema file, “.pig_schema”, in JSON file format.

Requirements from Data Provider

Please provide the following to allow for load estimation of the data to be sent:

1. Upload frequency (daily or hourly)

2. Number of files per batch upload and their size after compression, .bz2 preferred.

3. Projected data volume per day

Load data into your S3 bucket and provide necessary access to Yahoo, or upload data to a Yahoo S3 bucket. If using a Yahoo S3 bucket to deliver the data, Yahoo will need the public IP address ranges (in the form of IP CIDRs) of the hosts from which the data provider will upload files to the S3 location provided. A sample IP range list: "188.12.111.0/24, 188.12.112.0/30, 198.11.23.23/30".

High-Level Guidelines: Privacy, Security and Performance

The following are best practices for sharing data via S3:

1. Do not send data for opted out users.

2. Protect S3 bucket/location with security policy, such as

   1. Disable public access

   2. Enforce HTTPS (TLS1.2 or above) connection

   3. Enable server side encryption (SSE-S3) with cypher key rotation at least every 12 months.

3. Group files by feed types and day/hour and upload them to folders named by feed types and date/time. Refer to Directory Layout and File Format.

4. Limit the file size to around 1 GB (after .bz2 compressed)

   1. For small data sets, limit the number of files to under 5 per hour.

5. Avoid many small files.

6. Support credential rotation: annually or in need.

7. Secrets—for example, credentials—should be delivered in encrypted format: GPG public key to be provided by the receiver of the credentials. Refer to Sharing Secrets with External Partners using GPG.

Directory Layout and File Format

s3://<bucket name>/<3p-m>/product_catalog/yyyyMMdd[/hh]

_manifest -- upload this after all other files are uploaded

<.schema>

<file_1.csv.bz2> --- data file; TAB, ‘\t’, delimited

<file_2.csv.bz2>

...

<file_n.csv.bz2>

Pig Data Types

Contacts of Operational alerts

<provider-specific-name>_s3_feed_alerts@<provider-specific-name>.com

Sharing Secrets with External Partners using GPG

Background

This document describes how company A, the secret sender, can share secrets with company B, the secret recipient, over the internet securely with GPG encryption.

In a nutshell,

1. Company B uses GPG to generate a pair of public key, b_armor.pub, and private key, b_armor.

   1. Company B needs to provide a password, password_B,  in generating the public/private keys.

   2. Company B will need this password in decrypting the encrypted secret from company A.

2. Company B emails b_armor.pub to company A.

3. Company A uses b_armor.pub to encrypt the secret, and emails the encrypted blob to company B.

4. The company B uses b_armor (and password_B when prompted) to decrypt the encrypted blob.

Detailed Steps

1. Company A requests company B to generate (if not yet) and export their GPG public key. Company B follows instruction here: https://kb.iu.edu/d/awio

   1. Generate a key, assume company B’s email address is: “b_poc@b.com”

      1. gpg --gen-key

         1. Choose “(1) RSA and RSA (default)”

         2. Key length: 2048

         3. Expire in 2 weeks: 2w

         4. Full Name

         5. Email address: b_poc@b.com

         6. Comment: GPG key w/ company A

         7. Enter passphrase: <password_B> (company B need in decrypting the secret)

   2. gpg -o b_armor.pub -a --export b_poc@b.com

   3. Company B emails “b_armor.pub” as attachment to company A.

2. Company A imports company B’s public key

   1. gpg --import “b_armor.pub”

3. Company A puts secrets in “my_secret.txt” and encrypt it using b_armor.pub identified as “b_poc@b.com”

   1. gpg -o my_secret_armor.txt -a -e -r b_poc@b.com my_secret.txt

4. Company A emails “my_secret_armor.txt” as attachment to company B.

Create a Pixel ID (yp ID) and set as the Default Pixel for IFO Conversions

Important

In regards to multi-advertiser integrations, multiple advertisers will have different pixel IDs belonging to the same multi-vendor integration owner. Contact your Yahoo Account team for further details.

Create a New Pixel and Assign It as the Default Pixel

1. Under your advertiser in the DSP, navigate to the Tracking tab.

2. Click Create New and Pixel.

   The unique pixel ID is generated.

3. Copy the pixel ID from this newly created pixel to use in the POST endpoint.

   Do not use this pixel code for website tracking.

   

4. Reach out to the Yahoo Account Team to set the pixel as the default pixel for the IFO conversions.

Transaction Data Delivery Through Browser to Server

1. Place the Javascript Dot tag, which is linked to the default pixel ID you chose, on your product website (refer to example below).

   <script>(function(w,d,t,r){var q=[];w.ypr=function(e,t,d){q.push({e:e,t:t,d:d})};var s=d.createElement(t);s.src=r;s.async=!0;s.onload=s.onreadystatechange=function(){var y,rs=this.readyState;if(rs&&rs!="complete"&&rs!="loaded"){return};try{w.ypr=function(e,t,d){d.pixelId=<pixel_id_value_here>;YAHOO.ywa.I13N.firePostBeacon(e,t,d)};for(var n=0;n<q.length;n++){var o=q[n];w.ypr(o.e,o.t,o.d)};q=[]}catch(e){}};var scr=d.getElementsByTagName(t)[0],par=scr.parentNode;par.insertBefore(s,scr)})(window,document,"script","https://s.yimg.com/wi/ytc.js")</script>

   Important

   • If it is desired to use an existing online conversion pixel or IFO pixel as the default pixel, leave the existing code on the page and append the new pixel code from the IFO rule list page. Both the existing and new codes can coexist on the same page, but only one instance of the new code should be present.

   • If the default pixel needs to be changed at any time, contact your Yahoo Account team to update the default pixel assignment. Once the default pixel is changed, replace the existing default pixel code on your product website with the new code pointing to the new default pixel ID.

   • The default pixel can have multiple rules attached to it and the code for the default pixel only needs to be implemented once on your product website, provided the default pixel does not change.

2. Make a call to the Javascript tag with transaction events (refer to example below). For more details on what fields are required for the call, refer to Required and Supported Fields.

   <!-- retailer's code -->
   <script>
   var config = {
   pageUrl: 'https://example.com',
   userData: {
   ids: {
   EMAIL: ['hashed_value'],
   PHONE: ['hashed_value'],
   },
   },
   eventData: {
   totalPrice: 123.45,
   currency: "usd",
   products: [
   {
   id: "12345SKU", // unique id for product / service
   unitPrice: 123.45,
   quantity: 1, // count of product in transaction
   customKeyValues: {
   custom: "value",
   key: "hole"
   }
   }
   ],
   customKeyValues: {
   key: 'value',
   }
   }
   };
   function fireBeacon() {
   window.ypr('collect', 'purchase', config);
   }
   </script>

Required and supported fields

The required fields for a successful call are described in the table below.

Note

Yahoo DSP supports up to four (4) customKeyValues via CAPI. These values represent custom reporting dimensions.

Transaction Data Delivery Through Server to Server

The specification shown below identifies the key components of the CAPI request, including the POST url for streaming and batch conversions endpoint, header types and the POST body.

Endpoints

Batch Conversions Endpoint

This is the default endpoint for all integrations. When using the batch conversion endpoint, data is processed once a day.

POST https://batch.datax.yahoo.com/v1/events/<pixelId>

Streaming Conversions Endpoint

When using the streaming conversion endpoint, data is processed multiple times a day.

POST https://streaming.datax.yahoo.com/v1/events/<pixelId>

Headers

Content-Type: application/json

Accept: application/json

Authorization: Bearer <access_token>

POST body - JSON keys

Required and supported fields

The required fields for a successful JSON body POST are described in the table below. Normalize all values to lower case.

Note

CAPI supports multiple identity keys, the “userData” fields listed in the table below, to match the event. Multiple keys can be sent to improve match rates. One or more device IDs, email addresses, phone numbers or Partner Match IDs (pxid) are required.

Sample Without Privacy Object

[
  {
    "eventName": "addToCart",
    "eventId": "event1",
    "eventTs": 1713545795,
    "actionSource": "web",
    "actionSourceUrl": "http://store.com",
    "country": "USA",
    "region": "NA",
    "userData": {
       "email" : [ "email1_hash", "email2_hash" ],
       "gpsaid" : ["gpsaid_hash"],
       "phone" : ["phone_hash","phone2_hash"],
       "pxid" : ["pxid_key:pxid_value"],
       "ip_address" : "clientIp_hash",
       "userAgent" : "Y Browser"
    },
    "eventData": {
      "price": 1.22,
      "currency": "USD",
      "products" : [
        {
          "id": "abc1",
          "name": "duck",
          "brand": "Rubber",
          "quantity": 2,
          "unitPrice": 0.11,
          "category": "bath",
          "subCategory": "toys",
          "customKeyValues" : {
              "vendorModel" : "abc"
          }
        },{  
          "id": "abc2",
          "name": "rake",
          "brand": "Wood",
          "unitPrice": 1.0,
          "category": "garden",
          "subCategory": "tools"
        }
      ],
      "customKeyValues" : {
          "orderId": "order1",
          "promo": "abc1",
          "store": "store1"
      }
    }
  }
]

Sample With Privacy Object

[
  {
    "eventName": "addToCart",
    "eventId": "event1",
    "eventTs": 1713545795,
    "actionSource": "web",
    "actionSourceUrl": "http://store.com",
    "country": "USA",
    "region": "NA",
    "userData": {
       "email" : [ "email1_hash", "email2_hash" ],
       "gpsaid" : ["gpsaid_hash"],
       "phone" : ["phone_hash","phone2_hash"],
       "pxid" : ["pxid_key:pxid_value"],
       "ip_address" : "clientIp_hash",
       "userAgent" : "Y Browser"
    },
    "privacy" : {
      "privacy_type": “OPTOUT”
    },
    "eventData": {
      "price": 1.22,
      "currency": "USD",
      "products" : [
        {
          "id": "abc1",
          "name": "duck",
          "brand": "Rubber",
          "quantity": 2,
          "unitPrice": 0.11,
          "category": "bath",
          "subCategory": "toys",
          "customKeyValues" : {
              "vendorModel" : "abc"
          }
        },{  
          "id": "abc2",
          "name": "rake",
          "brand": "Wood",
          "unitPrice": 1.0,
          "category": "garden",
          "subCategory": "tools"
        }
      ],
      "customKeyValues" : {
          "orderId": "order1",
          "promo": "abc1",
          "store": "store1"
      }
    }
  }
]

OAuth 2.0 Authentication

CAPI is a server-to-server implementation that requires OAuth 2.0 authentication before any data can be posted to Yahoo Ad Tech servers.

OAuth 2.0 is a mechanism that relies on continuously refreshing authentication tokens. Clients are providing those tokens during the posting of data.

Important

Yahoo does not support OAuth 1.0, which depends on static tokens.

Step 1 Request Client Credentials

Step 2 Generate the JSON Web Token (JWT)

Step 3 Complete the Post-Credential OAuth 2.0 Workflow

Step 4 Use Access Token to Call CAPI Endpoint

Request Client Credentials

Note

Requesting client credentials includes an internal allowlist approval process that may require additional time for the setup to be completed.

To complete the steps below, you first need a Client ID and a Client Secret. Follow the steps outlined below to request them.

1. Generate a private key.

   >> openssl genpkey -aes256 -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out private_key.pem

2. Generate a public key using the above private key.

   >> openssl rsa -in private_key.pem -out public_key.pem -outform PEM -pubout

3. Send the public key to the Yahoo Account Team.

4. Yahoo will then send a file containing credentials encrypted with the above public key for you to use.

5. Decrypt the file with the private key.

   >> openssl rsautl -decrypt -inkey private_key.pem -in credential.enc -out my_credentials.txt

Generating JSON Web Token (JWT)

The JSON Web Token is composed of three main parts:

• Header: normalized structure specifying how the token is signed (generally using the HMAC SHA-256 algorithm).

• Free set of claims embedding whatever you want: client_id, aud, expiration date, etc.

• Signature ensuring data integrity.

The signature mechanism is HMAC_SHA256 as defined by the JOSE specifications:

https://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31

JWT Header
{
"alg": "HS256",
"typ": "JWT"
}

JWT Claims
{
"aud": "https://id.b2b.yahooinc.com/identity/oauth2/access_token?realm=dataxonline,
"iss": "{client_id}",
"sub": "{client_id}",
"exp": {expiry time in seconds},
"iat": {issued time in seconds},
"jti": "{UUID}"
}

Note the following:

• “exp” and “iat” values should be numeric. Do not set them as strings.

• “exp” value is currentTime + 3600 (i.e. 60 minutes).

• Don’t use currentTime + (24 * 60 * 60). You may get a “JWT has expired or is not valid” error.

• UUID - A Universally Unique IDentifier, https://www.ietf.org/rfc/rfc4122.txt

Walking through manual steps to build this JWT value

jwt_signing_string = base64url_encode(jwt_header) + '.' + base64url_encode(jwt_body)

jwt_signature = base64url_encode(hmac_sha256(jwt_signing_string, client_secret))

JWT = jwt_signing_string + '.' + jwt_signature

JWT libraries can be found at https://openid.net/developers/libraries/.

Post-Credential OAuth 2.0 Workflow

Start the workflow after the encrypted credentials are received from the Yahoo Account Team.

1. Call the ID B2B server to get the access_token, which is valid for 60 minutes.

2. Call the POST /identity/oauth2/access_token endpoint of ID B2B with the JWT token created out of the provided client_id and the client credential.

Sample Request

curl -X POST 'https://id.b2b.yahooinc.com/identity/oauth2/access_token' \
-H 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_assertion_type=urn:ietf:params:oauth:client-assertion-type:jwt-bearer' \
--data-urlencode 'client_assertion=<jwt_token>' \
--data-urlencode 'scope=conversion-event' \
--data-urlencode 'realm=dataxonline'

Sample Response

{
  "access_token": "wcf1011c-70fe-4740-b8a1-781d2b4dd3q3",
      "scope": "conversion-event",
      "token_type": "Bearer",
      "expires_in": 3599
 }

Use Access Token to Call CAPI Endpoint

Call the CAPI endpoint with the access_token in the Authorization header. The API will verify the client access_token and identify the client and determine whether it is authorized for the pixel ID.

Sample Request

curl -X POST \
    https://streaming.datax.yahoo.com:443/v1/events/123456 \
    -H 'authorization: Bearer 9f4c74cf-5bb9-45ce-987c-e5240e5710b8' \
    -H 'cache-control: no-cache' \
    -H 'content-type: application/json' \
    -d '[
 {
    "eventName": "PURCHASE",
    "eventId": "4CE3F5FD-B203-45EA-98BE-79FB28E92DF5",
    "eventTs": 1733508168,
    "actionSource": "web",
    "actionSourceUrl": "http://store.com",
    "country": "USA",
    "region": "NA",
    "userData": {
        "email": [
            "536a09742acb5b4ec7c7d6c0e20a5d3f4318817817353b69f8ee15f27d3fc9fa",
            "7ebd20af4a7ff32eb6331c108cb1aa2176c195f7f1b472fc7ebfd2051f89f8a1"
        ],
        "gpsaid": [
            "c2f11fe5-3600-4ade-901e-5cf84f2d71a5"
        ],
        "phone": [
            "1036636844eea8b0c54623eee63eb5d83ad5b86c02cfa7a8da29a3c140c9b100",
            "f4ef23f72996f81f2bbc90929eb7d1e6397cba597cbbd70b5afa717ad41e500f"
        ],
        "pxid": [
            "999:XY50038zETeXJBOYNTRn7Z3T6VSkxDF5ZpRz3wvPEVmt1ZXHo"
        ],
        "ip_address": "77785fc7b151a325a39d2c40a7701cb57736f2ce34b7edbef4e538f42c1509d3",
        "userAgent": "Y Browser"
    },
    "order": {
        "orderId": "order1",
        "price": 1.22,
        "currency": "USD",
        "products": [
            {
                "id": "abc1",
                "name": "duck",
                "brand": "Rubber",
                "quantity": 2,
                "unitPrice": 0.11,
                "category": "bath",
                "subCategory": "toys",
                "customKeyValues": {
                    "vendorModel": "abc"
                }
            },
            {
                "id": "abc2",
                "name": "rake",
                "brand": "Wood",
                "unitPrice": 1,
                "category": "garden",
                "subCategory": "tools"
            }
        ],
        "customKeyValues": {
            "orderId": "order1",
            "promo": "abc1",
            "store": "store1"
        }
    }
}

Sample Response

{
"success": "COMPLETE"true
}

Rate Limitation

A rate limit is the number of events that are allowed within an API call an advertiser can make within a given period.

For the batch endpoint, the 2 rate limits are:

1. 200 events per second

2. 10MB of data per second

For the streaming endpoint, 2 rate limits apply:

1. An advertiser can send up to two hundred (200) events per second.

2. An advertiser can send up to 1MB of data per second.

Error Responses

The following return codes can come back in response.