class 9

Question 1

what are 3 reasons for computers to do NLP?

Answer 1

1. to communicate with humans
2. to learn
3. to have a better scientific understanding of language and language use

Question 2

what is a language model?

Answer 2

a probability distribution describing the likelihood of any string

Question 3

what is grammars purpose?

Answer 3

to define the syntax of legal sentences

Question 4

what is the purpose of semantic rules?

Answer 4

to define the meaning of the legal sentences

Question 5

what is the bag-of-words model?

Answer 5

the application of Naive Bayes to a string of words

Question 6

what is tokenization?

Answer 6

the process of dividing a text into a sequence of words

Question 7

what is an n-gram model?

Answer 7

use a Markov chain model that considers the dependence between n adjacent words

Question 8

in what cases would you use n-gram models?

Answer 8

in spam detection, author attribution, and sentiment analysis

Question 9

what are other alternatives to n-gram models?

Answer 9

character-level models or skip-gram models

Question 10

what is a structured model that is usually constructed through manual labor?

Answer 10

a dictionary

Question 11

what’s a common model for POS tagging?

Answer 11

the hidden markov model

Question 12

HMM [hidden markov model] combined with what algorithm can produce an accuracy of ~97%

Answer 12

Viterbi algorithm

Question 13

what is the task of assiging a part of speech to each word in a sentence?

Answer 13

part of speech tagging

Question 14

what is the corpus of over 3M words of text annotated with POS tags?

Answer 14

the Penn Treebank

Question 15

what are some types of POS tagging?

Answer 15

logistic regression: but uses a greedy search

Viterbi algorithm: slow

beam search: in between logistic and viterbi, keeps accuracy but drops less-likely tags

Question 16

examples of generative pos tagging models?

Answer 16

Naive Bayes and HMM

Question 17

what is the name for a list of allowable words?

Answer 17

lexicon

Question 18

what are open classes?

Answer 18

nouns, names, verbs, adjectives, and adverbs.

change rapidy

Question 19

what are closed classes?

Answer 19

pronouns, article propositions, etc

change relatively slowly

Question 20

how can dynamic programming be used for parsing?

Answer 20

this method stores the result of every analyzed substring so it doesn’t need to get reanalyzed later.

analyzed substrings are stored in a chart and the algorithm that stores the substrings is called a chart parser

Question 21

what is a chart parser algorithm that uses Chomsky Normal Form grammar?

Answer 21

CYK algorithm

Question 22

is Natural language a context-free grammar?

Answer 22

it is not, it is very dependent on contextual evidence

Question 23

what are some tasks of NLP?

Answer 23

text-to-speech
machine translation
speech recognition
question answering

Question 24

what are some complications of real natural language?

Answer 24

when a word has more than one meaning: lexical ambiguity

when a phrase has multiple parses: syntactic ambiguity

Question 25

what's the word for when a word or phrase is replaced by another word or phrase that has a close association or relationship with the original?

Answer 25

metonymy

Question 26

what's the inside out algorithm?

Answer 26

learns to estimate probabilities of PCFG without trees

Question 27

what is curriculum learning?

Answer 27

starts with short unambiguous sentences and works its way up to 3,4,5 word sentences

Question 28

what is semisupervised parsing?

Answer 28

starts with a small number of trees as data to build an initial grammar and then adds a large number of unparsed sentences to improve the grammar

Question 29

what is unsupervised parsing?

Answer 29

uses a corpus of sentences without trees to learn a new grammar or improve an existing grammar

Question 30

what is decoding?

Answer 30

the process of generating target words from source words

Question 31

what are sequence to sequence models? seq2seq

Answer 31

its a neural network architecture that uses RNNs in conjunction with an LSTM []

Question 32

what's the goal of machine translation?

Answer 32

translating a source language to a target language

Question 33

what are some examples of pretrained word embedded vector dictionaries?

Answer 33

WORD2VEC, GloVe, FASTTEXT

Question 34

what is Q learning?

Answer 34

it's a reinforcement learning algorithm that does not use a model for training. Instead it uses iterations to learn and predict things about its environmnet. It also uses an off policy approach which means that it can redefine the original policy and make its own rules