04 Architecture of retrieval system 2

Question 1

indexing

Answer 1

create a bag-of-words representation of each document by text in a fast look up structure

Question 2

inverted index

Answer 2

primary data structure generated by the indexing process

make a dictionary of all words in the collection
for each word, list all the docs it occurred in

Question 3

indexing steps

Answer 3

1. lexical analysis (tokenisation)
2. stop word removal
3. stemming
4. index structure creation

Question 4

relevance estimation

Answer 4

compute the relevance of a document for a query
- term weighting scheme which allocate a numeric to each term reflecting their importance
- similarity coefficient: use term weight to compute an overall degree of similarity

Question 5

search algorithm

Answer 5

binary AND search

best match algorithm
- for each document, score=0
- for each query term, search vocab list, pull out posting list
- for each document in the list, score += 1