Natural Language Processing

Question 1

NLP

What are three techniques for handeling word variation?

Answer 1

Tokenisation, Stemming, Lemmatisation

Question 2

What is Tokenisation?

Answer 2

break a sentence or paragraph into individual words

Question 3

What is stemming?

Answer 3

reducing a word to its root form

Question 4

What is lemmatisation?

Answer 4

reducing a word to its lemma

Question 5

What is the difference between stemming and lemmatisation?

Answer 5

Stemming simple cuts off any additional characters used to added context (adjustable -> adjust) whereas lemmatisation will change the word to its base case (better -> good, was -> (to) be)

Question 6

What is a “bag-of-words” representation?

Answer 6

All words in the corpus are considered without context and only as single items

Question 7

What does TF-IDF stand for?

Answer 7

term frequency - inverse document frequency

Question 8

How are keywords identified?

Answer 8

Calculate the frequency of every term
for each document, calculate all TF-IDF scores
for n keywords, take the top n TF-IDF scores

Question 9

How is TF-IDF calculated?

Answer 9

Multiply a terms frequency within a document (TF) by its inverse document frequency across a collection of documents

Question 10

How is IDF calculated?

Answer 10

log(n/nt)
n = number of docs in corpus
nt = number of docs in corpus with term in

Question 11

How is text represented?

Answer 11

Using a sparse vector

Question 12

What values are recorded in a sparse matrix?

Answer 12

1. Word Count
2. Presence of a word
3. TF-IDF

Question 13

What is topic modelling used for?

Answer 13

Dimensionality reduction

Question 14

What are the advantages of topic modelling?

Answer 14

1. It is language agnostic - only uses a document-term matrix as input
2. There is no need to know in advance what the topics are
3. A quick and common way to summarise text

Question 15

4 Answers

What are the disadvantages of topic modelling?

Answer 15

1. It is not well defined so it is difficult to generalise to new unseen documents
2. There are many free parameters so scales linearly
3. Prone to overfitting
4. Doesn’t work well for short text

Question 16

What is word embedding?

Answer 16

Representing every word as a dense vector

Question 17

5 Answers

What can a word embedding vector represent?

Answer 17

1. Word Morphology
2. The contexts where a word appears
3. The global Corpus statistics
4. Concept Hierarchy
5. The relationship between a set of documents and the terms in them

Question 18

How are word embeddings used?

Answer 18

Word embeddings can be fed to a neural network and used to predict the next word in the sentence

Question 19

How does a continuos bag of words model work?

Answer 19

It predicts the next word using multiple other words as context

Question 20

How does the skip-gram model work?

Answer 20

Using a single word as context to predict multiple words

Question 21

How can word embeddings be generated?

Answer 21

By feeding one-hot encoded words into a single layer neural network the weight vector that maximises prediction correctness is the word embedding of a given word.

Question 22

How are semantic relationships between words represented?

Answer 22

They are represented as the relationship between vectors in vector space

Question 23

What is polysemy?

Answer 23

When a word has multiple meanings

Question 24

What is one solution for word embeddings and polysemy?

Answer 24

One embedding is generated for every sense of the word, known as sense embeddings

Question 25

# 5 Answers What are some applications of sentence embedding?

Answer 25

1. Measuring semantic similarity 2. Clustering Similar texts 3. Paraphrase mining 4. Automatic Translation 5. Text summarisation