ML15146 blog AWS Personalized News Arch new 001 cropped
With a multitude of articles, videos, audio recordings, and other media created daily across news media companies, readers of all types—individual consumers, corporate subscribers, and more—often find it difficult to find news content that is most relevant to them. Delivering personalized news and experiences to readers can help solve this problem, and create more engaging experiences. However, delivering truly personalized recommendations presents several key challenges:
In this post, we describe how Amazon Personalize can power a scalable news recommender application. This solution was implemented at a Fortune 500 media customer in H1 2023 and can be reused for other customers interested in building news recommenders.
Amazon Personalize is a great fit to power a news recommendation engine because of its ability to provide real-time and batch personalized recommendations at scale. Amazon Personalize offers a variety of recommendation recipes (algorithms), such as the User Personalization and Trending Now recipes, which are particularly suitable for training news recommender models. The User Personalization recipe analyzes each user’s preferences based on their engagement with content over time. This results in customized news feeds that surface the topics and sources most relevant to an individual user. The Trending Now recipe complements this by detecting rising trends and popular news stories in real time across all users. Combining recommendations from both recipes allows the recommendation engine to balance personalization with the discovery of timely, high-interest stories.
The following diagram illustrates the architecture of a news recommender application powered by Amazon Personalize and supporting AWS services.
This solution has the following limitations:
Let’s walk through each of the main components of the solution.
To implement this solution, you need the following:
interactions datasetitems datasetTo train a model in Amazon Personalize, you need to provide training data. In this solution, you use two types of Amazon Personalize training datasets: the interactions dataset and items dataset. The interactions dataset contains data on user-item-timestamp interactions, and the items dataset contains features on the recommended articles.
You can take two different approaches to ingest training data:
In this solution, you can also ingest certain items and interactions data attributes into Amazon DynamoDB. You can use these attributes during real-time inference to filter recommendations by business rules. For example, article metadata may contain company and industry names in the article. To proactively recommend articles on companies or industries that users are reading about, you can record how frequently readers are engaging with articles about specific companies and industries, and use this data with Amazon Personalize filters to further tailor the recommended content. We discuss more about how to use items and interactions data attributes in DynamoDB later in this post.
The following diagram illustrates the data ingestion architecture.
The bulk of the model training effort should focus on the User Personalization model, because it can use all three Amazon Personalize datasets (whereas the Trending Now model only uses the interactions dataset). We recommend running experiments that systematically vary different aspects of the training process. For the customer that implemented this solution, the team ran over 30 experiments. This included modifying the interactions and items dataset features, adjusting the length of interactions history provided to the model, tuning Amazon Personalize hyperparameters, and evaluating whether an explicit user’s dataset improved offline performance (relative to the increase in training time).
Each model variation was evaluated based on metrics reported by Amazon Personalize on the training data, as well as custom offline metrics on a holdout test dataset. Standard metrics to consider include mean average precision (MAP) @ K (where K is the number of recommendations presented to a reader), normalized discounted cumulative gain, mean reciprocal rank, and coverage. For more information about these metrics, see Evaluating a solution version with metrics. We recommend prioritizing MAP @ K out of these metrics, which captures the average number of articles a reader clicked on out of the top K articles recommended to them, because the MAP metric is a good proxy for (real) article clickthrough rates. K should be selected based on the number of articles a reader can view on a desktop or mobile webpage without having to scroll, allowing you to evaluate recommendation effectiveness with minimal reader effort. Implementing custom metrics, such as recommendation uniqueness (which describes how unique the recommendation output was across the pool of candidate users), can also provide insight into recommendation effectiveness.
With Amazon Personalize, the experimental process allows you to determine the optimal set of dataset features for both the User Personalization and Trending Now models. The Trending Now model exists within the same Amazon Personalize dataset group as the User Personalization model, so it uses the same set of interactions dataset features.
When a reader visits a news company’s webpage, an API call will be made to the news recommender via Amazon API Gateway. This triggers a Lambda function that calls the Amazon Personalize models’ endpoints to get recommendations in real time. During inference, you can use filters to filter the initial recommendation output based on article or reader interaction attributes. For example, if “News Topic” (such as sports, lifestyle, or politics) is an article attribute, you can restrict recommendations to specific news topics if that is a product requirement. Similarly, you can use filters on reader interaction events, such as excluding articles a reader has already read.
One key challenge with real-time recommendations is effectively including just-published articles (also called cold items) into the recommendation output. Just-published articles don’t have any historical interaction data that recommenders normally rely on, and recommendation systems need sufficient processing time to assess how relevant just-published articles are to a specific user (even if only using user-item relationship signals).
Amazon Personalize can natively auto detect and recommend new articles ingested into the items dataset every 2 hours. However, because this use case is focused on news recommendations, you need a way to recommend new articles as soon as they’re published and ready for reader consumption.
One way to solve this problem is by designing a mechanism to randomly insert just-published articles into the final recommendation output for each reader. You can add a feature to control what percent of articles in the final recommendation set were just-published articles, and similar to the original recommendation output from Amazon Personalize, you can filter just-published articles by article attributes (such as “News Topic”) if it is a product requirement. You can track interactions on just-published articles in DynamoDB as they start trickling in to the system, and prioritize the most popular just-published articles during recommendation postprocessing, until the just-published articles are detected and processed by the Amazon Personalize models.
After you have your final set of recommended articles, this output is submitted to another postprocessing Lambda function that checks the output to see if it aligns with pre-specified business rules. These can include checking whether recommended articles meet webpage layout specifications, if recommendations are served in a web browser frontend, for example. If needed, articles can be reranked to ensure business rules are met. We recommend reranking by implementing a function that allows higher-ranking articles to only fall down in ranking one place at a time until all business rules are met, providing minimal relevancy loss for readers. The final list of postprocessed articles is returned to the web service that initiated the request for recommendations.
The following diagram illustrates the architecture for this step in the solution.
Personalized news dashboards (through real-time recommendations) require a reader to actively search for news, but in our busy lives today, sometimes it’s just easier to have your top news sent to you. To deliver personalized news articles as an email digest, you can use an AWS Step Functions workflow to generate batch recommendations. The batch recommendation workflow gathers and postprocesses recommendations from our User Personalization model or Trending Now model endpoints, giving flexibility to select what combination of personalized and trending articles teams want to push to their readers. Developers also have the option of using the Amazon Personalize batch inference feature; however, at the time of writing, creating an Amazon Personalize batch inference job doesn’t support including items ingested after an Amazon Personalize custom model has been trained, and it doesn’t support the Trending Now recipe.
During a batch inference Step Functions workflow, the list of readers is divided into batches, processed in parallel, and submitted to a postprocessing and validation layer before being sent to the email generation service. The following diagram illustrates this workflow.
To effectively scale, you also need the news recommender to accommodate a growing number of users and increased traffic without creating any degradation in reader experience. Amazon Personalize model endpoints natively auto scale to meet increased traffic. Engineers only need to set and monitor a minimum provisioned transactions per second (TPS) variable for each Amazon Personalize endpoint.
Beyond Amazon Personalize, the news recommender application presented here is built using serverless AWS services, allowing engineering teams to focus on delivering the best reader experience without worrying about infrastructure maintenance.
In this attention economy, it has become increasingly important to deliver relevant and timely content for consumers. In this post, we discussed how you can use Amazon Personalize to build a scalable news recommender, and the strategies organizations can implement to address the unique challenges of delivering news recommendations.
To learn more about Amazon Personalize and how it can help your organization build recommendation systems, check out the Amazon Personalize Developer Guide.
Happy building!
Whether a company begins with a proof-of-concept or live deployment, they should start small, test…
Digital tools are not always superior. Here are some WIRED-tested agendas and notebooks to keep…
Machine learning (ML) models are built upon data.
Editor’s note: This is the second post in a series that explores a range of…
David J. Berg*, David Casler^, Romain Cledat*, Qian Huang*, Rui Lin*, Nissan Pow*, Nurcan Sonmez*,…
Qualcomm did not violate a license with Arm when it acquired Nuvia for $1.4 billion,…