L15 - NLP

Question 1

Define Bayesian

Answer 1

Statistical method that assigns probabilities to events.

Question 2

What is NLP?

Answer 2

Natural Language Processing is the subset of AI tasked with analysing and interpreting human language.

Question 3

Give some examples of how NLP is used?

Answer 3

Text summation
Sentiment analysis
Topic extraction
Question answering
Spam detection

Question 4

What is the bayesian spam detection formula?

Answer 4

P( message | spam ) = P(spam|message)P(spam ) / P(message)

Question 5

How is NLP used for spam detection?

Answer 5

Historic data of both spam and non-spam emails is fed into some NLP model.

Creates a (message X word) matrix where each row is a message, each col is a word, and the value of the count of that word in the message.

From this matrix, classifications can be made regarding whether a message containing certain words is spam or not. This is the NLP classification model.

New message can be run through the model to determine whether it’s spam.

Question 6

Explain how NLP can be used to determine whether a block of text is lyrics from a rock song…

Answer 6

1. Download a large set of rock song lyrics.
2. Break lyrics down into a large set of words.
3. Perform Stemming or Lemmatisation on each word.
4. Count all occurances of words
5. This can be used to predict whether other lyrics are a rock song or not.

Question 7

Define Stemming…

Answer 7

Process of reducing a word to its root form by removing prefixes of suffixes.

Question 8

Define Lemmatisation…

Answer 8

Reducing words to their lemma (base form), which is the contextual root of the word.

For example, the lemma of studies if study. Whereas the stem of studies is studi ( not a real world )

Question 9

Why do we need stemming, lemmatisation or tokenisation?

Answer 9

Groups words or phrases with similar meanings, thus reducing the dimensional vector space of the models, and improves computability.

Question 10

What is an issue with stemming that lemmatisation solves?

Answer 10

Stemming can lead to non-real worlds. E.g Studies -> Studi

In contrast, the lemma of studies is study.

Question 11

What issue does Stemming and Lematisation solve?

Answer 11

High variation of similar words can lead to high dimensionality in the vector space. This degrades computation efficiency.

Thus, these reduce the dimensionality by grouping similar words to the same numeric value.

Question 12

Define tokenisation…

Answer 12

Break a sentence or paragraph into individual words.

Question 13

Give the Lemma of the following: was, changing, better, worse

Answer 13

be
change
good
bad

Question 14

Why can tokenisation from english to another language cause issues?

Answer 14

Because other languages may not have the same sentence structure, and may have numerous to convey 1 word in english.

Question 15

What is bag-of-words representation?

Answer 15

Vector based way of representing a text. Creates a Word Matrix containing all counts of each word in the text.

Each element int he vector represents the count of that word.

Columns contain all words in the text.

Each row is a sentence.

Can lead to sparse vectors.

Question 16

What does Bag-o-words usually result in?

Answer 16

Long tail distribution.

Common words have high count, less common have low count.

Common words such as ‘the’, ‘a’, ‘of’ etc. which are often meaningless regarding context.

Question 17

What does the Bag-of-word technique give as output? What can this be used to do?

Answer 17

Text in the form of numerical vector representation. This is now computable for operations such as classification and clustering.

Question 18

What are some issues with bag-of-words?

Answer 18

1. High dimensionality
2. Not accounting for word importance. For example, we see long-tail distribution due to high count of common words.
3. Scaling issues

Question 19

What model is built on improvement on Bag-of-words to account for word importance?

Answer 19

TF-IDF approach