Marketing Channels in JavaScript

All Web Analytics vendors provide out-of-the-box configurations to setup Marketing Channels and attribution reporting. However there will always be a case that you need to implement custom marketing channel detection logic in JavaScript through the tag management system. Even though it is unlikely the result will be identical to the one vendors provide, you need to get as close as possible.

I would like to highlight a few points that can help you get as accurate data as possible. Imagine the following scenario:

The marketing team wants to feed to their tools/systems with the marketing channel information customers are clicking through. This can be based either on tracking codes, referring domains or just be a direct visit. This information needs to be evaluated client-side using JavaScript. For simplicity, let’s assume that we do not need to care about “remembering” the last touch channel (attribution period); the marketing tool takes care of that. We just need to pass the information on a click-through basis.

What is your process?

Know your requirements

There is no saying how many times we make assumptions about our stakeholders’ needs. Make none! Keep in mind that Marketing Channels relate to marketing activities which translates to budget spent (£££). You need to provide the highest quality of work in terms of accuracy, robustness and maintainability; that includes knowing your end-goal. A few examples of simple requirements:

  • Marketing Channel is set to “XYZ” depending on the value of the tracking code (cmp) prefix
  • Marketing Channel Detail is set to the campaign code (cmp) of the respective channel (i.e. read the tracking code value)

However we can add some more complexity.

  • Distinguish Natural Search and Paid Search
  • Distinguish Natural Social and Paid Social
  • Capture referring domains but disregard known internal domains (i.e. payment gateways)
  • All channels should be checked and captured through-out the visit
  • Set as Direct only if it occurs at the beginning of the visit
  • The list of social, referring and internal domains is dynamic

This includes basic requirements and assumes that some processing takes place server side. However it is good starting point to start thinking how to proceed. Let’s visualise the high level requirements

marketing channels hierarchy
Channels Hierarchy

Write robust and simple solutions

Will try to break it down to some key points that most of the time affect accuracy and I have seen them repeat over-time:

Browser Support

Browsers are changing constantly, so does the JavaScript engine. It is expected from us to write modern code that takes advantage of the most recent features. let and const keywords, arrow functions etc. Until you realise things are not always peachy! In the chart below (taken from statcounter.com), IE11 has a significant market share in UK desktop traffic (7%). If your target audience has similar behaviour, you might want to think twice before using fancy features. Otherwise you will end up throwing exceptions and breaking the data collection process. True story! 🙂

statscounter browser market share
Browser market share – UK – Desktop

In general it is highly advisable to check browser support for any recently released feature before committing to it. Great source of such information is https://developer.mozilla.org/en-US/.

Know when a click-through occurs

You will need a generic way to identify when a click-through for any for the channels takes place. You can abstract the logic and cater for all channels. Let’s call it isChannelClickthrough(campaignCode, referrer). The campaign code (cmp, utm’s etc.) and the page referrer (document.referrer) are the all the information you will need (almost!).

The above implements the following logic checks in a waterfall fashion:

  1. If a channel parameter (campaign code – ?cmp=<…>) is set, then a paid channel is present
  2. Else if the referral is in place, check if the referring domain:
    1. matches known search engine domains
    2. matches know social media domains
    3. does NOT match known internal domains
  3. Else if no referral is present, check if its the first hit of the visit (aka Direct visit).

Note the order of identifyChannels() calls! Referring domains are at the end. This is important to first check for Natural Search and Natural Social. The reason is that first we are checking for a positive match while on the referring domains we are checking for the absence of a positive match across multiple domains (see below). If the above returns TRUE, you can then start populating and/or trigger marketing vendors that except this data. Let’s elaborate on how identifyChannel() works.

First point is to have a list of major domains that your audience goes through. This includes Search Engines, Social Media and known “friendly” domains. The last one are usually domains your customers interact with but are considered internal for the purposes of marketing attribution. Most likely this will include payment services, sites that span across multiple domains (www.my-site.com & www.my-site.co.uk).

For natural search and natural social, the referring domain needs to match any of the ones in the existing list. So the use of channel_domain_mapping.some() and .indexOf() do the job. However if we are checking for the referring domains, we need it to NOT match any of them. In this instance channel_domain_mapping.every() is used.

Where does Tealium fit in?

Since I have implemented the above in Tealium, had to use specific features which can be easily re-created in other Tag Management Systems:

  1. Within extensions and conditions, a data layer variable named “a” is available. It has two possible values; view and link depending on the type of event triggered (through utag.view or utag.link). It is suggested to trigger the whole marketing channel evaluation logic only when a page view is triggered. It’s not necessary you will have issues but being explicit and strict always helps in debugging data accuracy issues. So within extensions use the check a===’view’.
  2. b.tealium_session_event_number parameter is incremented every time a page view (utag.view) or link tracking (utag.link) is triggered within the session. It is an out of the box data layer variable that helps.

Maintainability & robustness in mind

One recurring issue I have faced is URL query string parameters’ names capitalisation. Depending on which person or application is creating the tracking code templates, you mind end up with multiple combinations. Some times people manually type the tracking parameters, in others different campaigns have different tracking templates. The same parameter (i.e. campaign code) might be configured as cmp, Cmp, CMP etc. The list goes on. A simple solution is to lower-case your keys only.

// ES6 fancy version with arrow function
var sanitised_search = window.location.search.replace(
/([^&?\/=]*)=/gi , x => x.toLowerCase()
);                  

This converts https://analyticsmayhem.com/?Cmp=CAMPAIGN_1 into ?cmp=CAMPAIGN_1. Note that the value of the parameter remains unaffected.

// ES5-compatible version
var sanitised_search = window.location.search.replace(
/([^&?\/=]*)=/gi , function(x) {return x.toLowerCase()}
);

Working with URL query string parameters can be tedious some times. One of the easiest way to manipulate them is using the URLSearchParams api. The main constructor expects a string in the format of “?key1=val1&key2=val2”. This can be easily extracted from window.location.search or manually create a URL object and read the search property.

// Create a URL object from a string
var url = new URL("http://example.com/search?query=lorem");
// Extract the URL query string parameters
var searchParams = new URLSearchParams(url.search);

searchParams.has("query") // returns true
searchParams.get("query") // returns "lorem"

The only downside is browser support; no support in IE. So this goes back to my first point; pay attention to your audience browser details!

Extensive QA

As expected, we always to QA complex/critical solutions. This is one of them. The main reason is not complexity but rather the important of data that it generates. You do not want to build something and 3 months down the line to figure out there was a deficiency with a particular channel (i.e. Affiliates) and you ended up paying incorrect commissions. Play it safe and test it! Test across multiple browsers, tracking code combinations, channel combinations, even across different campaigns within each channel.

Furthermore, include your stakeholders in this process. You want them to QA the data, sense-check them and make sure they have a say. An extra pair of eyes always helps! On top of that, they usually have much better knowledge of the metrics. So if something is off, they are in a much better position to spot it.

One point that I did not touch in this post is client-side persistence of the marketing channels (especially cross-domain). This requires a whole piece of analysis by itself! Maybe in a future post!

Enjoy!

Tealium GitHub Integration

The first time I heard about the Tealium Github integration was almost a year ago, around the time when Digital Velocity ’18 was in London. It sounded like an exciting feature to enhance your data collection process. And it finally got here: Tealium GitHub Integration documentation.

Tealium initially introduced this feature as part of their beta features and it got me really excited! It really changes your daily work and it becomes much smoother. You can find more information about the setup of the extension in the documentation link. However I would like to mention a few points that I personally find helpful.

Delivery from Visual Studio Code to Tealium extensions

Visual Studio Code is the code editor of my choice. I do all my JavaScript/Python work there. Till recently I felt I was under utilising it for my data collection tasks. Before this update this is how it looked like:

  1. Copy code from the extension (need to have latest version)
  2. Edit code
  3. Copy code from editor
  4. Paste within the extension
  5. Pray that you did not copy any accidental change!

Far from ideal. Now with the integration:

  1. Pull latest version locally from Github
  2. Edit code and commit changes
  3. Push to repository
  4. Synchronize extension

It really makes your life smoother when it comes to editing big and complicated JavaScript files. Below a few useful links that can assist in the overall setup:

Cross-profile code re-use

In some cases companies have multiple profiles. How would you manage 5, 10 or more different profiles? How would you manage a change that needs to be deployed across multiple profiles and/or accounts? You can easily reference the same file in all of them and all you have to do is sync to the latest version.

Streamline the delivery of updates

The most beneficial scenarios that come to my mind is managing marketing tags IDs and generic functions (i.e. cleaning URLs, parsing names, managing cookies/storage). You can maintain a central lookup of all your Facebook IDs, AdWords IDs/labels, Google Analytics IDs, Adobe Analytics report suites IDs etc. and make sure you enforce consistency across your sites. Think about how much more streamlined your work can be! The only consideration in the above is that you need to make sure that your code is cross-profile compatible. But I am a firm believer that this exercise will benefit you a lot in the long run. So do it!

Collaboration and code management

This is applicable to bigger teams or teams with tighter release permissions. With the extra features Github offers, you can now organise, review and scrutinise your work much more. For example, you are allowed to release on the site once every 2 weeks. You want to make sure that all the developers have done their work properly. The native version control in Tealium is good but let’s be honest, it has limitations. Now they are gone! Let’s investigate the following:

  • Organising GitHub
  • Using single or multiple extensions per file

Organising GitHub

You can have 3 distinct branches in Github (development, QA, production).

github branches
GitHub Branches

You development work will take place in the Dev branch where multiple people can contribute. Once the work is considered ready, you have to create a pull request from your Dev environment into the QA.

Provide all the details (i.e. what is the purpose of the update).

Pull request from Dev into QA

Approve (or reject the work) performed. This will lead into merging your changes into the next environment.

All set. Your QA environment just got updated.

Successful merge

The workflow is more or less the same regardless of how you configure the actual extensions within Tealium. What might change is the number of branches. You might decide to opt-in for just 2; dev/qa and production.

Option A – Single Extension with multiple files

This is relatively simple to implement and does not complicate your overall setup. You need one extension per file i.e. for Adobe Analytics doPlugins file in GitHub you will create only one doPlugins extension in Tealium. Within the extension you can link the same file for each branch (Dev, QA, Prod).

Github files
Linking single extension to multiple branches – branch is visible only during mouse over.

Time to fetch the code into Tealium. Go into the relevant file and click “Sync File” (this also triggers automatically when you expand the extension).

Synchronize Tealium with GitHub

You can switch between branches by clicking the appropriate GitHub file from above, sync it and publish it to the respective environment.

GitHub publish locations
Publishing to QA environment

The main caveat I have spotted so far is the branch name is only visible if you hover over the file (easy to mis-publish branches). Maybe Tealium can change that in the future.

Option B – Multiple Extensions

The alternative option is having 3 extensions, one for each branch.

tealium-extensions
Tealium Extensions

Each extension is configured to be deployed in a different environment through detecting the Tealium current environment.

Environment Condition – Dev Environment

The data layer variable has as a source the variable “ut.env”. That is a default Tealium data layer variable that depicts the current Tealium environment (dev/qa/prod).

udo variable
Tealium Environment Data Layer Variable configuration

Now you can link the doPlugins file from the GitHub Dev branch with the equivalent extension in Tealium; same applies for the rest of the branches (just make sure you use the correct path for each branch).

Linking extensions – The underlined part represents the branch.

This approach provides more clarity to your work but also creates a lot of duplication within the UI.

Regardless of which option you select (or if you come up with your own structure) you can now have a much smoother merge issues resolution and code review.

Same functionality, 3 extensions, 3 deployment environments

As a recap, your BAU will look something like this:

  1. Do you work on the dev branch.
  2. Commit/push to GitHub and perform pull request to QA.
  3. After merge, go in Tealium and sync your QA branch. Time for testing!
  4. Once happy, merge your code into the production branch.
  5. Back into Tealium, sync the production version. Ready to go live!

Overview

Obviously you can simplify/complicate things as much as you want – totally up to your personal and business requirements. The important thing is that now you have option to do adjust your processes in much more flexible way than before!

Hints:

  • Make sure you create all your repositories under an organisation in GitHub. Highly recommended to avoid issues with individual accounts.
  • Tealium offers also resource locks which limit edit access to particular resources (tags, extensions etc.). This is an alternative method to manage access.

Data Persistence in the ITP world

Update: since Apple announced ITP 2.3 the below solution is not functional any more.

With the introduction of ITP 2.1 recently (Webkit blog), it becomes even more complicated to maintain consistency in Web Analytics and make sure reporting accuracy is maintained. Furthermore sometimes it is not just Web Analytics that we care about; Digital Marketing and other forms of data lie within our realm.

A lot of the times we want to store in cookies information that is not related to identifying customers i.e. actions the customers did and reference them later, capture sequence of actions and process them at different point in the journey (the list goes on). With the introduction of ITP 2.1, this is a good reminder that cookies are not meant to store information that the server does not care about:

  • Previous page name
  • Name of links/buttons
  • Page load timings
  • Marketing channels tracking codes & related combinations
  • Rolling e-commerce basket information

However, we often tend to store this information in cookies because “it is easy”. Yes it is, but also comes with a cost. We tend to overload the HTTP requests with useless data which and – if you really want to be nit-picky – consume resources and customers pay more for on their PAYG phone plans.

At least let’s try

I was experimenting with an idea similar to the below. Store in localStorage or sessionStorage data while also having the option to sync between cookies and storage critical data (i.e. Adobe Marketing Cloud ID). Have drafted the below small piece of code.

The above defines a handy object with convenient methods to save and retrieve values from your storage method of choice. The key things to know are:

  • storeValue & getAndSyncValue are the key methods (example below).
  • Switch between localStorage and sessionStorage options
  • When syncing is used, a value will be set in both storage and cookie.

Now it is up to you to decide what is critical and must persist in cookies versus what can be saved in storage and make internet a bit nicer place!

How to use it

When a value is saved (i.e. user_id = 123), then the value ends up both in cookies and also the storage namespace used as part of the package.

Cookies
Local Storage

If the 3rd parameter was set to false (i.e. “storageManagement.storeValue(‘user_id’, 123, true)” ) then the user_id would be stored only in localStorage.

Hopefully the above will give you an indication of how easy it is to use an alternative storage method, make your work more elegant and better utilise Web APIs.

Check the following Stack Overflow post for more details about the API differences. Differences between Cookies and localStorage/sessionStorage