[RMP] Quick Start for Session-Based Recommendation

[EvenOldridge]

Problem:

Merlin provides documentation and a number of example notebooks on how to use tools like NVTabular, Dataloader and Merlin Models. In order to build a pipeline for training and evaluation purposes, a Data Scientist needs to analyze that material, copy-and-paste code snippets demonstrating the API and glue that code together to implement scripts for experimentation and benchmarking.
It might also not be clear to the users the advanced API options featured by Merlin Models that can be mapped as a hyperparameter, and potentially improve models accuracy.

Goal:

This RMP provides a Quick-start for building a training pipeline for session-based recommendation.
It addresses the ranking models part of this larger RMP #732, in particular the steps 4-7 of the Data Scientist journey when experimenting with Merlin.

The Quick start for ranking is composed by:

Template scripts

• Generic template script for preprocessing data for session-based recommendation
• Generic template script for building and training models for session-based recommendation, exposing the main hyperparameters for ranking models .
  It includes support to sequential models like YouTubeDNN, RNNs and Transformers (backed by HuggingFace library).

Documentation

• Documentation of the scripts command line arguments
• Documentation of best practices learned from our experimentation:
  • Hyperparameter tuning: search space, most important hyperparameters and best hparams for REES46 dataset
  • Intuitions of API options (building blocks, arguments) that can improve models accuracy

Constraints:

• Preprocessing - The pre-processing template notebook will perform some basic feature encoding for categorical (e.g. categorify) and continuous variables (e.g. standardization). It will also group interactions by session, sorted by timestamp
  The customer can expand the template with advanced preprocessing ops demonstrated in our examples.
• Training - The training and evaluation script for Merlin Models should be totally configurable, taking as input the parquet files and schema, and a number of hyperparameters exposed via command line arguments. The output of this script should be the evaluation metrics, being optinally logged to Weights&Biases and Tensorboard.

Investigations

• POC on how to build a session-based recommendation pipeline that can deal with the item cold-start problem #1051
• POC on how to build a session-based recommendation model that can be used to re-rank candidate items #1052

Starting Point:

• Script for Transformers4Rec paper reproducibility using Transformers4Rec library
• Script that @sararb has been porting from T4Rec repo to use Merlin Models API

Tasks

Dataset choice

• Choose the dataset for the Quick-start for session-based recs #951

Preprocessing script

• Change the preprocess.py script to support session-based recommendation #940
• Preprocessing - Make it possible to first group interactions per user or session before data split. #941

Modeling script

• Enable LogQ correction for ContrastiveOutput when negative sampling is used #942
• Session-based quick start: Support pointwise prediction tasks #943
• Configure how sequential and non-sequential features are combined #944
• Include support to basic mean sequence aggregators (YouTube DNN) #945
• Rename/create command line args and standardize training_trask and evaluation_task (e.g. "causal" x "all"): #946
• Quick-start for session-based script and CLI args refactored to be more modular #960

Experiments

• Quick-Start experiments: Run a session-based recommendation HPO for the selected dataset for next-item prediction task #947

Documentation

• Document the new CLI args for session-based preproc of the preprocessing script #948
• Document the CLI args of the session based recsys training script  #949
• Document the best practices on building a pipeline for session-based recommendation #950

Deployment and inference with Triton

• Create Quick-start script to build and export a Triton ensemble (NVT + Models) using Systems

• Create a Quick-start notebook demonstrating how to prepare an inference request to Triton

• Create markdown documentation on how to use the quick-start deployment script

• Starting point: https://github.com/NVIDIA-Merlin/models/blob/main/examples/usecases/transformers-next-item-prediction.ipynb

Tasks moved from #433 - Tensorflow support for session based recommendations integration in Merlin

Reproducibility of Transformers4Rec results and integration tests (23.01)

• Reproduce selected results from Transformers4Rec paper with Merlin Models API models#806 - Reproduce selected results from Transformers4Rec paper to ensure the implementation is correct
• Create integration tests for session-based recommendation models#807 - Integration tests based on #806 to ensure to track regressions in the API, accuracy and performance

Support of advanced sequential tasks and the definition of examples (22.11)

• [RMP] Support of advanced sequential tasks and the definition of examples #472 - The main objective is to support advanced session-based tasks and create examples of common session-based and sequential-based architectures.