Use unconventional AI to amplify performance of Amazon SP

The Challenge

The Solution

Our Approach

Utilising advanced data science techniques like reinforcement learning and neural networks in a E2E daily automated way promises more won opportunities.

Amazon works with Ad Exchange to offer a real-time bidding environment where advertisers can participate in order to select Sponsored Products. Manufacturers (advertisers) choose to select the frequency of bids to be placed in a day. The bid price in this scenario is placed on the search keywords, which means advertisers have the option to bid for multiple keywords in a day.

The decision on the required keywords for which a bid needs to be placed is made by competitors’ analysis. The list of search terms on which competitors gain conversions is identified and is considered to be the target keywords to bid on. However, these keywords are quite dynamic in nature and are expected to change because of how customer search evolves and changes over time. Using these keywords as a feature for the model would ask for updating the model architecture every time.  As an alternative, we built a model architecture that is independent of the number of keywords by ensuring that:

1. The number of keywords enters as a “batch” dimension instead of a “feature” dimension. This means that predictions on one keyword are treated independently from predictions on the other keywords.
2. The keywords are represented as continuous vectors in space. The model outputs a vector in that exact space, and we find the K-nearest neighbours to that prediction.

Upon identification of target keywords, the optimal bid value of each keyword is identified using a dual-agent reinforcement learning model. One agent predicts the bid amount to be placed for a given keyword and the other predicts the long-term performance in the form of conversion for the suggested bid amount. The two agents are dependent on each other such that one agent helps to win the real-time auction and the other ensures that the bid results in maximum Return of Ad Spend (ROAS).

We used the open-source AuctionGym – a simulated environment created by developers from Amazon to enable a close replica of the actual bidding environment at Amazon for researchers and professionals. It helped us fine-tune our reinforcement learning models and strategise on deciding the target keywords, which could lead to better conversion.

The open-source simulated environment presents an impression opportunity to the bidders, on which the bidders decide a bid price and the ad to be shown. The environment then decides the winning ad and price, which is then shown as a Sponsored Product, possibly leading to conversion. The internal system in AuctionGym environment consists of Bernoulli process to simulate whether the allocation decision leads to conversion, enabling an end-to-end auction scenario.

The model shows the critical parameters used to select the most optimum bid and the estimate of the performance based on the bid price set. As time progresses, the agent learns the bidding strategy better and starts outperforming manually suggested bids for the keywords. For a completely trained model, RL agent’s bid suggestion provided approximately 30%  than a human agent (a manual process).

Already showing great tangible results

We initially implemented our approach within Google Ads, which resulted in a 30% improvement in Click-through rate (CTR) within 6 months. The increased visibility led to an 8% higher ROI, and the optimisation suggestions generated by our algorithms enabled the Account manager to allocate 35% of his time to more strategic tasks previously devoted to manual, operational work.

A friction-less implementation

Our solution can be an additional feature to what you currently use and an upgrade of your dashboard approach, so the investment in terms of time and resources would be minimal. Alternatively, we can deliver a stand-alone front end to manage input & output.

The Success

Maximise efficiencies & business impact from your sponsored product investments

Consumers’ shift to digital media for shopping, entertainment, education, and other purposes has resulted in the accumulation of mountains of data that can be used for in-depth data analysis to depict complex consumer behaviour. Meanwhile, businesses around the world are recovering from the effects of COVID-19, and marketing budgets are limited. In such a case, implementing an intelligent bidding strategy that extracts meaningful information from vast amounts of data as efficiently and cost-effectively as possible aids in making the best use of the marketing cost constraints.

By automating performance optimisation, you or your Amazon ads partner can transform execution and operation time into strategic thinking to make better decisions. You get more actionable insights, which would help improve your KPIs, like CTR, ROAS and Sales Volume. Replacing the manual daily work with our reinforcement learning model saves time and generates more sales using fewer resources while being a source of extra intelligence for your media and marketing teams Finally, adding this unique AI layer to your Amazon SP provides a competitive advantage.

Our model is not only tested and approved, but it also is Amazon’s algorithm update-proof. It is change-resistance and is less dependent on historical data and trends vs other, widely used techniques. It perfectly mimics the search algorithms in the digital advertising space.

It’s not a black-box solution. Taking the user on the journey is our speciality! We can explain any context and actionable insights that come out of the data model, which we believe is also key to building trust in the solution and using it optimally.