Data-driven market segmentation – more effective marketing to segments using AI

We are currently working with a client in the FMCG who is trying to unpack better customer segments or “clusters” in order to engage and market to them better.

 

The problem is we are trying to solve is “How might we find customers that allow for better ROI on marketing initiatives?”

 

Its interesting project. Currently most companies that face this challenge. Imagine however the ability to segment customers in a totally new way? Discover common needs or characteristics not humanly possible or conceivable? What if we allowed artificial intelligence to apply its own way to “segment” or “cluster” ?

 

At HIVERY, we apply artificial intelligence to business problems. With this FMCG, we set the challenge to unpack new customer segments for our client using our proprietary machine learning framework, and run A/B market experiments to ‘test’ the effectiveness of these new “machine-conceived” segments.

 

Below is our highlight approach, at the end we are able to create a custom application application to allow the discovery of new segments and communicating and measuring that new segment possible using our proprietary machine learning framework.

 

 

Screen Shot 2016-04-03 at 10.17.44 AM
Gather the data, apply our AI framework (unsupervised learning algorithm), once new clusters are identified work with our client (i.e. domain expertise) to unpack and refine what they mean, communicate on new segments to stakeholders for buy in (i.e. marketing teams), test and measure marketing campaigns using A/B testing method, fine tune marketing actions and deploy wider.

 
So do this? Well, lets talk supervised and unsupervised learning algorithms. Supervised learning is when the dataset you feed your algorithm is done with “pre-define tags”. So this classification algorithm requires training data. Once the “machine learning training” is completed (i.e. a classification model is created), than the classification model is used to classify new datasets and help identify common needs or characteristics (based on pre-define tags). This how most customer segments are done. This is a “this is a female, age 40-50”, every time the machine recognises a “female, age 40-50”, it group them together.

 

 

In unsupervised learning algorithms, there are no “pre-define tags”, hence “”machine learning training” is not done at all. Here we allow the “machine learning” to identify pattens on its own. The way MACHINES SEE DATA is COMPLETELY different to the way HUMANS SEE DATA. This is why at HIVERY we say Data Has A Better Idea™.

 

Here is data set compared to the same cluster by an unsupervised learning algorithms.

Original vs clustered dataset using unsupervised learning algorithm

Original vs clustered dataset using unsupervised learning algorithm

 

 

In our example of “males vs females”, unsupervised learning algorithms might cluster based on a specific characteristic seen in the data itself (beyond what humans segment), instead of “gender” and “age”, it might be some other variable like “likely to commit fraud” or “strong likely to purchase an up-sell” or “will re-purchase within next 4 days”

 

New market segments based on unsupervised learning algorithm

New market segments based on unsupervised learning algorithm approach

 

 

I like Saimadhu Polamuri explanation of the difference between supervised and unsupervised learning algorithms, he talks about basket and it is filled with fresh fruits.

 

The task is to arrange the same type fruits at one place. Assuming the fruits are apple, pomegranate, banana,cherry and grape only.

 

So you already know from your previous learnt knowledge (which was trained in the past) to recognise the shape of each and every fruit so it is easy to arrange the same type of fruits at one place. As these fruits come in, you recognise them arrange or “cluster” the same type of fruits together, forming a different segments. This type of learning is called as supervised learning.

 

 

In unsupervised learning, suppose you are an alien from another world, and had the same basket and it is full with same fruits. Like before your task is to arrange the these one place. But this time you don’t know any thing about “fruits” – you are an alien after all! You are seeing these fruits for the first time, so how will you arrange them? You might decide to select any physical characteristic of that particular fruit. Suppose you take colour. Then you will arrange them base on the colour and go something like this:

  • Red colour group: apple, pomegranate & cherry
  • Green colour group: banana & grapes

Now you will take another physical characteristic like size, so now you groups them things like:

  • Red colour AND big one: pomegranate, apple
  • Red colour AND small one: cherry
  • Green colour AND small one: grapes
  • Green colour AND big one: banana

Here you haven’t learnt any thing before about “fruits” means no train data and non-response variable. This type of learning is know unsupervised learning.

 

Summary:

 

Supervised learning: discover patterns in the data that relate data attributes with a target (class) attribute. These patterns are then utilised to predict the values of the target attribute in future data instances.

 

Unsupervised learning: The data have no target attribute.

 

Both are type of classification techniques.