Lecture 9: NLP

Question 1

NLP VS Speech Processing

Answer 1

Natural Language Processing
= automatic processing of written texts
1.
Natural Language Understanding

Input = text
2.
Natural Language Generation

Output = text

Speech Processing
= automatic processing of speech
1.
Speech Recognition

Input = acoustic signal
2.
Speech Synthesis

Output = acoustic signal

Question 2

What is BOW model

Answer 2

A simple model where word order is ignored

used in many applications:

NB spam filter seen in class a few weeks ago

Information Retrieval (eg. google search)

…

But has severe limits to understand meaning of text…

Maybe we should take word order into account…

Question 3

N-gram model

Answer 3

An n-gram model is a probability distribution over sequences of events (grams/units/items)

Question 4

Why onnly bigram or trigram?

Answer 4

Markov approximation is still costly
with a 20 000 word vocabulary:

bigram needs to store 400 million parameters

trigram needs to store 8 trillion parameters

using a language model > trigram is impractical

Question 5

How would one recognize language?

Answer 5

• train a bigram of which characters are followed up by which

Question 6

problem with n-gram

Answer 6

Natural language is not linear ….

there may be long-distance dependencies.

Syntactic dependencies

The man next to the large oak tree near … is tall.

The men next to the large oak tree near … are tall.

Semantic dependencies

The bird next to the large oak tree near … flies rapidly.

The man next to the large oak tree near … talks rapidly.

World knowledge

Michael Jackson, who was featured in …, is buried in California.

Michael Bublé, who was featured in …, is living in California.

…