Blog Post

Paid search, toddlers, and CMAB: a Relevance Calculator

Marketing is a complex discipline generating vast amounts of data daily. Lots of data, tracked through different resources, waiting to be combined and interpreted.

It gets even more complex with PPC marketing. In paid search marketing many factors affect customer buying decisions in real-time, which in return affect the marketer’s ability to adjust the paid search strategy accordingly.

Paid advertising, especially Google Ads, is expensive. Not only keyword bidding is expensive, but also only a small percentage of the leads convert right away from an ad. Google Advertising makes spending easy, meanwhile bidding optimization gets harder.

Paid search marketing is a thin line between too general and too specific keywords, derived from your customers’ needs and biggest pain points. Did you make the right keyword bidding decision? The reports show you that. The reports also affect any future decisions regarding the paid search campaigns and, ideally, show which keywords perform better than others and which to abandon.

So, your future paid search strategy success depends on your past data. And the way you collect it. And the way and time you spend analyzing it. And this success will only get the customer to your door (your website or landing page), it does not guarantee the purchase/revenue.

paid advertising marketing

PPC advertising is an opportunity to bring highly qualified leads to the door to ensure a higher % of conversations. We are here to help you improve your paid search strategy with a few clicks and some historical data. It is called the Relevance Calculator and here is how it works.

The model we are using is called a contextual multi-armed bandit (CMAB), an algorithm from the field of reinforcement learning. Reinforcement learning is a field focusing on developing models aimed at discovering optimal strategies in complex and often not very well understood environments, taking a lot of their principles from psychology.

We came up with a non-traditional way of using the CMAB algorithm to test how your historical bidding strategies work, how much you could have earned and how to tweak it for higher future returns based on your specific goals and target customers.

The way you can understand how the model works is by comparing it to a child and how they learn from their environment. When the toddler is crawling around the house, they sometimes find things that are fun and want to do them over and over again.

Sometimes they also find things that are not so fun like for example might accidentally burn their hand on the stove or bump their head on a table.

In all those circumstances you can think of the concept of rewards. In circumstances that lead to a negative outcome, you get a negative return, and in desired outcomes, you get a positive reward.

The model learns exactly as the toddler from the feedback that they continuously get from the environment. In our case, the feedback is in the form of business KPIs.

reward for bidding optimization
As the model learns from past behavior and by exploring new campaign strategies, it learns which actions lead to the best increase in KPIs defined. Over time, the model explores different strategies to get the best return via defined KPIs and then exploits what it’s learned in order to maintain the level of these KPIs at their new height.

And much like the behavior of a toddler, an adult- a human expert- can step in to guide the CMAB on its path. A marketing expert or a campaign manager can give intuition to the model to improve the future bidding strategies based on historical insights and patterns by allowing the campaign manager to override model decisions in favor of their own bids – giving the model a chance to learn from its human peers.

Bidding optimization by human expert


It is an ideal solution for a live deployment that can actively engage with and streamline the marketing decision-making process.

This same concept holds for new campaigns defined by the marketing team. Initial bidding strategies defined by campaign managers give the model an initial boost on its way to learning the campaign dynamics during the bidding process until real-time signals via tracked KPIs stream in after launch.

Another natural consequence of a CMAB’s ability to learn over time and to be trained by its human counterparts is its ability to adapt to an ever-changing marketing landscape, with changing client needs and tastes, changing search patterns, and changing competitor dynamics over time.

It is an ideal solution for a live deployment that can actively engage with and streamline the marketing decision-making process.


Indeed, a CMAB is more than just a model for savvy smart bidding optimization. CMABs are general models able to discover new and better strategies in many areas within the marketing space.

Whether it be optimizing email campaigns, optimizing customer interactions and customer outreach, or rolling out of targeted campaigns, any domain requiring in-depth knowledge from expert campaign managers with years of experience and which has ever-changing dynamics across time is fair game for a CMAB approach. You can read about many more interesting applications in work by Bouneffouf et al. to build an intuition of its usage.

CMABs can hence be used in many applications to streamline and automate many marketing decision-making processes, not only saving on time and resources but also potentially discovering new and more efficient processes over time as a result of its inherent exploration of new and diverse strategies.


Result? 40% (*estimated for the results with the test data) higher paid search ROI on average (sometimes reaching 60-80% KPI uplift) versus the historical bidding results for our client, JOT Internet Media. 

We partnered with JOT Internet Media, a company that focuses on lead generation, services monetization, digital marketing, media, and investments, through European Data Incubator Challenge. Our challenge to solve was Pattern Recognition Campaign Performance Indicators, which became our Relevance Calculator.


40% higher paid search ROI*