Information Retrieval and Recommender Systems

Guide to AI and Data Courses · Part IV: Data
IndexingQuery and rankingCollaborative filteringEvaluationDense vector searchNeighboring courses
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

What Question Does It Ask?

How are the most relevant documents or products picked from millions of candidates?

Information retrieval studies systems that find the documents most relevant to a query among millions and rank them. Recommender systems do the same job without a query: they predict what a user will like from past behavior. Both return a ranked list, not an answer.

At a glance

Approach

matching and ranking

Input

query, documents, user history

Output

ranked result list

Success measure

precision, recall, NDCG

Math load

medium: vector spaces

Data needs

lots of interaction data

Introduction

A Short History

1972

Spärck Jones proposes inverse document frequency (IDF): a rare term says more.

1975

Salton and colleagues publish the vector space model: documents and queries are term vectors.

1992

Xerox PARC's Tapestry system coins the term "collaborative filtering".

1998

Brin and Page introduce the Google search engine, built on PageRank.

2006

The Netflix Prize begins; matrix factorization stands out in the winning solutions.

2020

Dense vector retrieval and retrieval-augmented generation (RAG) begin to spread.

The Course

Core Concepts

Inverted index

For each term, a list of the documents containing it.

TF-IDF and BM25

Scoring documents by term frequency and rarity.

Vector space model

Measuring similarity by the cosine of the angle.

Precision, recall

Share of results that are relevant; share of relevant ones found.

Ranking metrics

MAP, MRR, NDCG: position in the list matters.

Collaborative filtering

Recommending from what similar users liked.

Matrix factorization

Splitting the user-item table into latent factors.

Cold start

New users or items with no history.

The Course

A Typical Workflow

Collectcrawls,interaction logsIndexinverted index,vector indexRetrievethousands offast candidatesRanklearnedrankerEvaluateNDCG,A/B test
Example: A video platform collects watch logs, indexes videos as vectors, retrieves a few hundred candidates from millions, ranks them by probability of being watched, and compares the new ranker with the old one in an A/B test.

The Course

What Does the Course Cover?

  1. Boolean retrieval and the inverted index
  2. Index construction, compression and dictionaries
  3. Text preprocessing: tokenization, stemming
  4. Scoring: TF-IDF and the vector space model
  5. Probabilistic retrieval and BM25
  6. Evaluating retrieval: precision, MAP, NDCG
  7. Link analysis: PageRank and HITS
  8. Learning to rank
  9. Dense retrieval and approximate nearest neighbours
  10. Content-based recommendation
  11. Collaborative filtering and matrix factorization
  12. Evaluating recommenders: offline metrics, A/B tests

Skills you will gain

building a search index in ElasticsearchBM25 experiments with Pyserinivector search with FAISStraining recommenders with Surprisemeasuring rankings with NDCG and MAP

A typical semester; the order and coverage vary between universities.

Try It

Searching a Help Center

A shopping site's help center has nine articles, and customers type questions in their own words. Pick a sample question or type your own; articles are ranked by TF-IDF weights and cosine similarity.

IDF down-weights words such as “my” and “order” that appear in many articles, so the distinctive “damaged” stands out. Compare with the TF button. Cutting each word to its first four letters is a crude stemmer: damage, damaged → dama.

Context

Prerequisites

Courses worth knowing before this one, and why they are needed.

CourseWhy it is needed
Linear Algebravector space model, matrix factorization
Data Structureshash tables, trees and priority queues: the index and the top k results
Algorithmssorting, merging and complexity: speed on large indexes

Context

Are You Ready for This Course?

Ask yourself; for every "no", the right-hand column shows where to start.

Ask yourselfIf not, start with
Can you compute and interpret the cosine similarity of two vectors?Linear Algebra
Can you explain why lookup in a hash table takes constant time?A data structures course: hash tables and trees
Can you compute precision and recall from a confusion matrix?Machine Learning
Can you split a text into words and count word frequencies?Natural Language Processing and Text Mining
If you can say "yes" to most of these, you are ready; one or two gaps can be closed during the semester.

Context

Neighbouring Courses

subfieldbuilds onshares methodsapplication

Overlap

Overlapping Courses

RelationCourseShared ground
builds onThis course builds on Linear Algebravector space model, matrix factorization
shares methodsShares methods with Natural Language Processing and Text Miningtext representation, TF-IDF, semantic search
shares methodsShares methods with Data Miningassociation, collaborative filtering
shares methodsShares methods with Social Network AnalysisPageRank, link analysis

Differences

Where It Differs (1/2)

CriterionIR & RecommendersData MiningNLP
Core questionHow are the most relevant documents or products picked from millions of candidates?How are hidden, useful patterns found in large piles of data?How does a computer analyze, understand and generate human language?
Approachmatching and rankingdiscovery-driven analysislinguistics, statistics and deep learning
Inputquery, documents, user historylarge tables or transaction datatext
Outputranked result listrules, clusters, outlierslabel, translation, summary, answer
Success measureprecision, recall, NDCGinterestingness, support, confidenceaccuracy, BLEU, human evaluation
Math loadmedium: vector spacesmediummedium
Data needslots of interaction datalargelarge text corpora
  • Data Mining: Data mining discovers patterns in data; this course turns discovery into a ranked list for one user's current need.
  • NLP: NLP analyzes the structure and meaning of language; this course represents text only to score and rank relevance.

Differences

Where It Differs (2/2)

CriterionIR & RecommendersMachine LearningSocial Network Analysis
Core questionHow are the most relevant documents or products picked from millions of candidates?How does a program generalize from examples?How are networks of people and their relationships analyzed?
Approachmatching and rankingstatistical learninggraph theory and statistics
Inputquery, documents, user historylabeled or unlabeled examplesnodes and links
Outputranked result lista model that makes predictionscentrality, communities, diffusion
Success measureprecision, recall, NDCGerror on unseen datamodularity, interpretability
Math loadmedium: vector spaceshigh: linear algebra, probability, optimizationmedium: graphs, matrices
Data needslots of interaction datamoderate to largenetwork data
  • Machine Learning: ML tries to predict individual examples correctly; this course cares about list order, above all the top few results.
  • Social Network Analysis: Social network analysis studies the structure of ties; this course uses that structure (e.g. PageRank) only as a ranking signal.

Practice

Applications

Web search

Ranking the pages most relevant to a query.

E-commerce

Customers who bought this also bought...

Video and music

Personal feeds and playlists.

Enterprise search

Searching company documents and e-mail.

News feeds

Ranking posts by relevance.

RAG

Fetching the right documents for a language model.

Practice

Risks and Ethics

Filter bubbles

Always showing more of what a user liked narrows the range of views; polarization from news and video feeds is debated.

Popularity bias

Popular items get recommended more, so they get clicked more; new and small producers stay invisible.

Query logs give people away

From search logs that AOL anonymized and released in 2006, journalists identified some of the users.

Ranking manipulation

Link farms, fake reviews and keyword stuffing distort rankings; the system must be designed against adversaries.

In depth: Explainable AI and AI Ethics.

Practice

Tools and Datasets

Tools and libraries

Apache LuceneElasticsearchFAISSPyseriniSurprisescikit-learn

Well-known datasets

MovieLensNetflix PrizeMS MARCOTREC collectionsCranfield collectionAmazon product reviews

Practice

Common Misconceptions

Common beliefIn fact
A search engine just finds the words in the query.The real work is ranking thousands of matches; users rarely look past the first page.
Recommending the top-rated items is enough.The same list for everyone is not personalization; diversity and novelty are measured too.
High accuracy means good recommendations.A model with low rating error can still rank badly; rank metrics such as NDCG are needed.
Dense vectors have made classic search obsolete.BM25 is still a strong baseline; many systems combine the two.

Next Steps

After This Course

Courses that build on or extend this one

This course is mostly an end point: it builds on the others.

Related careers

Search engineerRecommender systems engineerData scientistRanking researcher
After this course you can move on to semantic search with natural language processing, link-based ranking with social network analysis, and indexing billions of documents with big data systems. Retrieval-augmented generation connects this course directly to large language models.

Next Steps

Further Reading

  • Manning, Raghavan and Schütze, Introduction to Information Retrieval, Cambridge University Press, 2008.
  • Baeza-Yates and Ribeiro-Neto, Modern Information Retrieval, 2nd ed., Addison-Wesley, 2011.
  • Aggarwal, Recommender Systems: The Textbook, Springer, 2016.
  • Koren, Bell and Volinsky, "Matrix Factorization Techniques for Recommender Systems", IEEE Computer 42(8), 2009.

The End

Chapter 28: Information Retrieval and Recommender Systems
← Previous chapter · Map of the field · Next chapter →
1 / 1 Sercan KÜLCÜ. All rights reserved.