Connecting the omni-channel jigsaw puzzle

I recently read a very nice book that I purchased from Kindle and wanted a few of my friends/colleagues to read that. When I recommended the book to them I also asked them to check if their Amazon/Kindle are also recommending the same book for them. For some people, it was there somewhere in the 9^th or 10^th spot and in many cases, the book did not even feature in their recommendation lists. So, it looks like even the most sophisticated ML/Analytics-driven recommendation engines are not yet as 'knowledgeable' as us.

We are slowly getting into an 'analytics-led' age where every B2C (or even B2B) company is trying very hard to build the profile or the genome of their customer. Understanding the customer is key for any B2C/B2B business and that starts with an accurate customer profile that is created by collecting lots of data about the customer. The lure of ‘data as the new currency’ forced these companies to invest quite a bit on their 'big data' and ‘analytics’ strategies around this and most of them are yet to see any real ROI. Most companies now do have an omni-channel presence to some extent, and have invested a lot to make their presence felt across multiple channels – but, that’s about it.

So what went wrong?

The first problem was that they went too fast too soon. Most companies went from no customer information to a lot of data (not insight – plain data) in a very short period of time. They just went crazy collecting customer data without a real strategy to use them. Since the cost of data storage reduced dramatically, it became cheap to store them in data lakes. In a sense, the data hype was similar to the dotcom boom, when everyone went to the internet without a proper internet ready business plan or strategy.

"Gartner has predicted that through 2018, 90% of deployed data lakes will be useless. That word “useless” should grab your attention. It’s worse than a “failed project,” Source: Forbes

The second problem was the lack of maturity of data science. Companies hired a lot of data scientists to slice and dice their data to extract magic out of that in order to up-sell and cross-sell. The algorithms were neither robust nor proven and hence the big data pretty much created mere classifications and segments. The 'Netflix' model of recommendation was probably pushed to a lot of use cases where it does not even apply. Soon these companies realized that actionable insight is not the same as information when it comes to creating a customer profile.

The third bit that went wrong was the customer profile model itself. This, in fact, is the biggest problem of a big data implementation. Let's take the customer genome of my friends who are buying books online as an example. There are probably at least 50 attributes that affect the customer's book buying pattern (recommendation from friends, review in a newspaper, price of the book, mood to name a few). Some of these factors are understood by data scientists to some reasonable extent, but there are attributes that are still way to fuzzy to factor in a real algorithm (emotional attributes like mood for example).

Even if a recommendation engine factors 50% of the attributes - say 25 of this 50 - that means there are 126 trillion different ways to pick 25 attributes from a set of 50. So, what are the REAL chances that the algorithm picked the right set of attributes? Currently, using a large number of attributes in any calculations is probably out of question for most algorithms as it needs an enormous computing power. Hence most companies are choosing the easy attributes and that reduces the accuracy and quality of the outputs.

Disconnected omni-channels do not work

Most companies created their omni-channel presence in a disconnected manner. They have their presence for many use cases across multiple channels, but the critical piece of stitching them together is missing. Mere presence on eCommerce platforms such as YouTube, Twitter, and Facebook; as well as tracking them individually not only give limited data, it sometimes creates a false customer genome. As mentioned earlier, there are trillions of different combinations possible with customer attributes. Now imagine these attributes are spread across multiple channels and no one is connecting them back. It’s like solving a jigsaw puzzle with trillions of pieces in them - can’t be that easy, right?

Then there is also the issue of missing data. Even if a company is present across multiple channels, a lot of critical customer behavior happens outside these channels. These data points are not captured and hence never factored in the algorithms. For example, Amazon has my book browsing and searching history and is trying to put together a profile of me, but Amazon does not have access to the discussions I had with my friend on a topic and books related to that. Moreover, they don’t have access to my Google calendar to see what type of meetings I am attending. To build a correct customer profile with such missing pieces is almost like solving a jigsaw puzzle with 100 trillion pieces where half of them are missing.

Figure 1 depicts how my customer profile data is distributed across different channels during the day:

Fig 1: Customer data owners during the day

It is clear that not a single enterprise has access to all the profile data and hence can’t really build a 360° profile or genome of a customer. Also, note the amount of offline data that we still generate.

Connected omni-channel is the future

The three key aspects that need to be considered while building a customer genome are:

1. Focus on more attributes - try to capture as many attributes as possible. The larger the number of attributes, the smaller are the missing pieces in the jigsaw puzzle. Focus on the emotional attributes as well and capture them, as analytics algorithms mature. They will be able to use these data in future.
2. Connected omni-channels - the data collected from different channels will need to be stitched back together to create the full customer genome. Connecting the browsing data from Amazon with the video watching patterns from YouTube will reveal more deep profiles.
3. Data sharing with others - as it's very clear that one company will never own the data for any customer, it's important to build a data sharing mechanism with other players. If a company can combine the YouTube data with the cable TV data, that will build a more complete jigsaw picture. In many cases, it will mean collaborating with rivals, but that’s a different problem that needs to be addressed.

It is evident that building a customer genome from omni-channel data is an enormous jigsaw puzzle that will never have all the pieces. But, when companies introduce real-time personalization to their omni-channel strategy, customer profiles will have fewer gaps.  A robust omni-channel strategy is a good start, but connecting them will get it closer.