“The recommender system from Devoteam G Cloud will enable us to deliver a greatly improved personalised experience to all of our users, and tackle more complex use-cases such as ads recommendations and content recommendations”
About the customer
The Royal Belgian Football Association (RBFA) was founded on 1 September 1895. It is the official representative of Belgian football. They have many responsibilities going from managing the Belgian football teams like the Belgian Red Devils and Belgian Red Flames to assisting local teams with their organisation. In short, almost anyone interested in football in Belgium will at some point interact with the RBFA.
As an important part of their business, the RBFA provides specialised football content to supporters through articles, videos, and quizzes. This ensures that football fans are always up-to-date with the latest news, in addition to putting interesting matches/interviews/initiatives in the spotlight.
The Challenge
To keep users engaged, the RBFA wants to create a personalised experience for each of them. This concerns all the content that the RBFA produces, but also important advertisements from their partners, and products in their web shop. All these elements share one common denominator: they contain tags. Tags are minimalist word-length descriptions of what the entity (content, ad, or product) is about. For example, #red-devils, #futsal, or #disability-teams.
Therefore, instead of directly recommending content, ads, or products, we can first recommend the most relevant tags for a user, and then join them to the related content, ads, and products.
The Goal
The focus of the project is on recommending content to users. Being able to recommend personalised ads and products was left for the future, as it requires more complex integrations with RBFA’s existing tools.
To this end, we first developed tag recommendations using known interactions between users and tags through the smartphone application or website, that we then joined to content through tags.
However, when developing recommender systems, one must tackle several challenges:
- The deployed solution must provide fast but accurate recommendations, in order to avoid having long loading times for the users.
- In practice, recommender systems are inherently complex, hard to maintain, and subject to data drift.
For this project, we had no choice but to tackle all these challenges for the RBFA simultaneously, so that they could take advantage of the benefits of a modern recommender engine.
The Solution
In order to understand how Devoteam G Cloud’s recommender system technology works, we need to define its main working entities: embeddings. Embeddings are high-dimensional vectors, mathematical representations of business entities. In the case of the RBFA, where the goal was to make tag recommendations, Devoteam G Cloud created a neural matrix factorisation model that produces embeddings for users and for tags.
The great aspect of embeddings is that, once they are constructed, you can easily make 4-ways recommendations based on the closest neighbours. Four-ways recommendations allow you to cover much more than just a simple content recommendation use case. These are the possibilities it offers:
- Tag to tags: Similar tag recommendations on a web page (“Users also viewed”).
- Tag to users: Marketing targeting of users (“Newsletters”).
- User to tags: Classical tag recommendations (“Your newsfeed”).
- User to users: Users persona identification for marketing purposes (“User categorization”).
For the RBFA, the four recommendation ways were a great asset, because they enabled them to understand their users better, while also improving their overall experience. To this end, the RBFA is able to use their embeddings to serve more personalised content to their users, but also to create relevant newsletters and increase engagement.
To ensure proper scalability of the solution while having low response times, a fully serverless architecture was deployed using Vertex AI Endpoints, App Engine, and Firestore. These tools constituted a proxy that caches and serves recommendation requests to the end-users. This proxy was revealed to be important in serving accurate results fast, as caching recommendations enabled the set up of an SLO requiring that 95% of requests should be served under 100ms.
Finally, once tag recommendations have been made, the proxy joins the results with the database of the different types of content, so that user-level personalised recommendations are possible.
The Methodology
The project was divided into two parts:
- First, developing a model that is able to make user-tags recommendations and present the results in a Data Studio dashboard;
- Then, deploy the model in production and serve recommendation requests with an API that has the role of proxy in front of the machine learning model.
This separation allowed us to first validate the performance of the model with the stakeholders, and then implement it for usage in production.
All along the project, Devoteam G Cloud closely worked with the RBFA in an agile way to make sure that the project evolves in the right direction. To this end, bi-weekly steering meetings were organised, as well as daily standups to tackle potential blocking factors. This allowed Devoteam G Cloud’s consultants to evolve at their full speed, delivering business value to the RBFA as soon as possible.
The Result
Thanks to Vertex AI Pipelines, the recommender system model was fully automated from start to finish. This means that the RBFA can at any time, and at the cost of a click, spin up a new pipeline to retrain the model using the latest data available, and deploy it in production with zero downtime. This makes it significantly easier for RBFA’s technical teams to manage recommender systems on a day-to-day basis, while also being able to react to data drift.
From a users’ perspective, their way of interacting with RBFA’s platforms is enhanced, since they can now experience truly personalised content recommendations, without too much impact on the application or website’s latency.
In the future, Devoteam G Cloud’s recommender engine technology will enable the RBFA to propose personalised advertisements and products to their users. This improvement will not only greatly impact the user experience, but also generate a lot of business value through the sales it will generate.
Did this case inspire you? Then download the RBFA app to discover how it works from an end-user perspective.
