Natural Language Processing

Question 1

What is Natural Language Processing?

Answer 1

The act of handling natural language using some form of computational model

Question 2

What is text language similarity?

Answer 2

Detection of language from words in a script

Question 3

What is sentiment analysis?

Answer 3

Classifying the emotional content of a message

Question 4

What is topic extraction?

Answer 4

Detection of a topic from a script

Question 5

What is text summarisation?

Answer 5

Summarisation of text

Question 6

What is relationship extraction?

Answer 6

Extracting the relationships between objects in text

Question 7

What is question answering?

Answer 7

Answering questions provided to an NLP

Question 8

What is language generation?

Answer 8

Generating language given a question

Question 9

What is Bayesian spam detection?

Answer 9

A method of detecting the possibility that a message is spam

Question 10

How is Bayesian spam detection calculated?

Answer 10

The multiplicative probability that each word in a message is spam, multiplied by the probability of spam, divided by the probability of a message

Question 11

What is word variation?

Answer 11

The fact that two words can have the same root/lemma but have different meanings

Question 12

What is the lemma of a word?

Answer 12

The ‘root’ of that word - e.g. ‘speak’ and ‘speaks’ have the same lemma

Question 13

What is tokenisation?

Answer 13

Separating a sentence or paragraph into different tokens or words

Question 14

What is stemming?

Answer 14

Reducing a word to its root form by removing any prefixes and suffixes - e.g. ‘changing’ -> ‘chang’

Question 15

What is lemmatisation?

Answer 15

Reducing a word to its original meaning - e.g. ‘was’ -> ‘to be’, ‘changing’ -> ‘change’

Question 16

Why may tokenisation be different in English compared to other languages?

Answer 16

Two words may be semantically linked - e.g. in Vietnamese, ‘thoi gian’ means a period of time, but separately they are unrelated (‘thoi’ means a shuttle or buffet and ‘gian’ means time) - or a single token may mean multiple words - e.g. in Japanese, 姉 (or ‘ane’) means ‘older sister’ while 妹 (or ‘imōto’) means ‘younger sister’, but ‘sisters’ is a compound of the two: 姉妹 (pronounced ‘shimai’)

Question 17

What is meant by a ‘bag of words’ representation?

Answer 17

An entire corpus/document of words is represented simply as the frequency of their stemmed or lemmatised meanings

Question 18

What is meant by a ‘corpus’ in relation to word processing?

Answer 18

All of the text being used, or a document in its entirety

Question 19

How do word frequency distributions typically look?

Answer 19

Like a long tail, with connectives at the front and descriptions at the end

Question 20

To represent data among multiple documents, we could use…

Answer 20

A document-word frequency matrix

Question 21

We can measure how much a word relates to a type of document by…

Answer 21

Dividing its frequency in that document types by its frequency in other document types

Question 22

What is one of the main problems with ‘bag of words’ representations?

Answer 22

The matrix generated is typically too sparse - there are too many zeroes

Question 23

What is a mathematical topic model used for?

Answer 23

Discovering the different topics in a collection of documents

Question 24

What is Latent Dirichlet Allocation (LDA)?

Answer 24

Separating a document-term matrix into two separate matrices - document-topic, and term-topics - like in 2nd normal form!

Question 25

What does Latent Dirichlet Allocation (LDA) achieve?

Answer 25

The separation of words into topics, grouping them together to reduce dimensionality

Question 26

What is one problem with topic modelling?

Answer 26

It is not great for generative modelling, and does not work well for short texts with little topics to discussW

Question 27

What is word embedding?

Answer 27

Taking every word as a dense vector, where we represent each word with some continuous representation - e.g. the probability that a word will appear in a certain context

Question 28

How can we perform word embedding?

Answer 28

Create one-hot vectors representing each word and their context, then provide them to an autoencoder network to get embedding values

Question 30

What can we do to solve the issues posed by single-word encoding (predicting only one word ahead)?

Answer 30

Predicting multiple words ahead for each y instead, which is called a skip-gram

Question 31

Embedded vectors translate semantic relationships to...

Answer 31

Vector relationships

Question 32

What is meant by polysemy?

Answer 32

Words that have multiple meanings - e.g. queen, queen or Queen (the ruler, the poker card or the band)

Question 33

What are sense embeddings?

Answer 33

Unique lemmatisations that help separate the meanings of the words - e.g. "Queen" -> "Queen, the band" and "queen" -> "queen poker card"

Question 34

What is one way we can adjust our word embeddings for corpora (plural of corpus) with larger scales?

Answer 34

By using sentence embeddings, or even document embeddings, that perform the same sort of task but over a larger corpus!