We are thrilled to announce the publication of the paper “Why Ignore Content? A Guideline for Intrinsic Evaluation of Item Embeddings for Collaborative Filtering” in the Proceedings of the Brazilian Symposium on Multimedia and the Web (WebMedia’24) [link].
Authored by Pedro R. Pires (UFSCar), Bruno B. Rizzi (BTG Pactual), and Tiago A. Almeida (UFSCar), this work tackles the challenges of scalability and sparsity in recommender systems. The paper provides a comprehensive guideline for intrinsic evaluation of item embeddings, focusing on their qualitative properties in collaborative filtering systems.
Key contributions include:
- A novel approach to evaluating the intrinsic quality of embeddings using techniques adapted from Natural Language Processing.
- A comparative analysis of matrix factorization and neural embedding models with traditional extrinsic evaluation methods.
- A proposed strategy for evaluating embeddings in content-based scenarios, enriching the assessment pipeline for recommendation systems.
- Findings that emphasize how embeddings with suboptimal extrinsic performance can excel in tasks demanding intrinsic knowledge, such as similarity detection and autotagging.
This research highlights the importance of intrinsic evaluation for understanding the latent properties of embeddings, providing valuable insights for developing more robust recommender systems. Congratulations to the authors for advancing the field of collaborative filtering!