Information Retrieval and Recommender Systems
Introduction
What Question Does It Ask?
Information retrieval studies systems that find the documents most relevant to a query among millions and rank them. Recommender systems do the same job without a query: they predict what a user will like from past behavior. Both return a ranked list, not an answer.
At a glance
matching and ranking
query, documents, user history
ranked result list
precision, recall, NDCG
medium: vector spaces
lots of interaction data
Introduction
A Short History
Spärck Jones proposes inverse document frequency (IDF): a rare term says more.
Salton and colleagues publish the vector space model: documents and queries are term vectors.
Xerox PARC's Tapestry system coins the term "collaborative filtering".
Brin and Page introduce the Google search engine, built on PageRank.
The Netflix Prize begins; matrix factorization stands out in the winning solutions.
Dense vector retrieval and retrieval-augmented generation (RAG) begin to spread.
The Course
Core Concepts
For each term, a list of the documents containing it.
Scoring documents by term frequency and rarity.
Measuring similarity by the cosine of the angle.
Share of results that are relevant; share of relevant ones found.
MAP, MRR, NDCG: position in the list matters.
Recommending from what similar users liked.
Splitting the user-item table into latent factors.
New users or items with no history.
The Course
A Typical Workflow
The Course
What Does the Course Cover?
- Boolean retrieval and the inverted index
- Index construction, compression and dictionaries
- Text preprocessing: tokenization, stemming
- Scoring: TF-IDF and the vector space model
- Probabilistic retrieval and BM25
- Evaluating retrieval: precision, MAP, NDCG
- Link analysis: PageRank and HITS
- Learning to rank
- Dense retrieval and approximate nearest neighbours
- Content-based recommendation
- Collaborative filtering and matrix factorization
- Evaluating recommenders: offline metrics, A/B tests
Skills you will gain
A typical semester; the order and coverage vary between universities.
Try It
Searching a Help Center
A shopping site's help center has nine articles, and customers type questions in their own words. Pick a sample question or type your own; articles are ranked by TF-IDF weights and cosine similarity.
Context
Prerequisites
Courses worth knowing before this one, and why they are needed.
| Course | Why it is needed |
|---|---|
| Linear Algebra | vector space model, matrix factorization |
| Data Structures | hash tables, trees and priority queues: the index and the top k results |
| Algorithms | sorting, merging and complexity: speed on large indexes |
Context
Are You Ready for This Course?
Ask yourself; for every "no", the right-hand column shows where to start.
| Ask yourself | If not, start with |
|---|---|
| Can you compute and interpret the cosine similarity of two vectors? | Linear Algebra |
| Can you explain why lookup in a hash table takes constant time? | A data structures course: hash tables and trees |
| Can you compute precision and recall from a confusion matrix? | Machine Learning |
| Can you split a text into words and count word frequencies? | Natural Language Processing and Text Mining |
Context
Neighbouring Courses
Overlap
Overlapping Courses
| Relation | Course | Shared ground |
|---|---|---|
| builds on | This course builds on Linear Algebra | vector space model, matrix factorization |
| shares methods | Shares methods with Natural Language Processing and Text Mining | text representation, TF-IDF, semantic search |
| shares methods | Shares methods with Data Mining | association, collaborative filtering |
| shares methods | Shares methods with Social Network Analysis | PageRank, link analysis |
Differences
Where It Differs (1/2)
| Criterion | IR & Recommenders | Data Mining | NLP |
|---|---|---|---|
| Core question | How are the most relevant documents or products picked from millions of candidates? | How are hidden, useful patterns found in large piles of data? | How does a computer analyze, understand and generate human language? |
| Approach | matching and ranking | discovery-driven analysis | linguistics, statistics and deep learning |
| Input | query, documents, user history | large tables or transaction data | text |
| Output | ranked result list | rules, clusters, outliers | label, translation, summary, answer |
| Success measure | precision, recall, NDCG | interestingness, support, confidence | accuracy, BLEU, human evaluation |
| Math load | medium: vector spaces | medium | medium |
| Data needs | lots of interaction data | large | large text corpora |
- Data Mining: Data mining discovers patterns in data; this course turns discovery into a ranked list for one user's current need.
- NLP: NLP analyzes the structure and meaning of language; this course represents text only to score and rank relevance.
Differences
Where It Differs (2/2)
| Criterion | IR & Recommenders | Machine Learning | Social Network Analysis |
|---|---|---|---|
| Core question | How are the most relevant documents or products picked from millions of candidates? | How does a program generalize from examples? | How are networks of people and their relationships analyzed? |
| Approach | matching and ranking | statistical learning | graph theory and statistics |
| Input | query, documents, user history | labeled or unlabeled examples | nodes and links |
| Output | ranked result list | a model that makes predictions | centrality, communities, diffusion |
| Success measure | precision, recall, NDCG | error on unseen data | modularity, interpretability |
| Math load | medium: vector spaces | high: linear algebra, probability, optimization | medium: graphs, matrices |
| Data needs | lots of interaction data | moderate to large | network data |
- Machine Learning: ML tries to predict individual examples correctly; this course cares about list order, above all the top few results.
- Social Network Analysis: Social network analysis studies the structure of ties; this course uses that structure (e.g. PageRank) only as a ranking signal.
Practice
Applications
Ranking the pages most relevant to a query.
Customers who bought this also bought...
Personal feeds and playlists.
Searching company documents and e-mail.
Ranking posts by relevance.
Fetching the right documents for a language model.
Practice
Risks and Ethics
Always showing more of what a user liked narrows the range of views; polarization from news and video feeds is debated.
Popular items get recommended more, so they get clicked more; new and small producers stay invisible.
From search logs that AOL anonymized and released in 2006, journalists identified some of the users.
Link farms, fake reviews and keyword stuffing distort rankings; the system must be designed against adversaries.
In depth: Explainable AI and AI Ethics.
Practice
Tools and Datasets
Tools and libraries
Well-known datasets
Practice
Common Misconceptions
| Common belief | In fact |
|---|---|
| A search engine just finds the words in the query. | The real work is ranking thousands of matches; users rarely look past the first page. |
| Recommending the top-rated items is enough. | The same list for everyone is not personalization; diversity and novelty are measured too. |
| High accuracy means good recommendations. | A model with low rating error can still rank badly; rank metrics such as NDCG are needed. |
| Dense vectors have made classic search obsolete. | BM25 is still a strong baseline; many systems combine the two. |
Next Steps
After This Course
Courses that build on or extend this one
Related careers
Next Steps
Further Reading
- Manning, Raghavan and Schütze, Introduction to Information Retrieval, Cambridge University Press, 2008.
- Baeza-Yates and Ribeiro-Neto, Modern Information Retrieval, 2nd ed., Addison-Wesley, 2011.
- Aggarwal, Recommender Systems: The Textbook, Springer, 2016.
- Koren, Bell and Volinsky, "Matrix Factorization Techniques for Recommender Systems", IEEE Computer 42(8), 2009.