Lecture 4

Question 1

Syntactic Level Analysis

Answer 1

To analyze how words are put together
to make valid sentences

Question 2

Grammar:

Answer 2

Grammar: the kind of implicit
knowledge of your native language that
you had mastered by the time you were
3 or 4 years old without explicit
instruction

Question 3

Chomsky:

Answer 3

syntactic structure can be independent on the meaning of the sentence

Question 4

Grammars (and parsing) are key
components in many applications:

Answer 4

• Grammar checkers
• Dialogue management
• Question answering
• Information extraction
• Machine translation

Question 5

Two types of Grammars

Answer 5

*Context Free Grammar (CFG), also known
as Phrase Structure Grammar
* Dependency Grammar

Question 6

Context Free Grammar (CFG)

Answer 6

a set of recursive rewriting rules (or productions) used to generate patterns of strings.
CFGs describe the structure of language by capturing
constituency and ordering

Question 7

Constituency

Answer 7

How we group words into units and what we say about how the various kinds of units behave

Question 8

Ordering

Answer 8

Rules that govern the ordering of words and bigger units in the language

Question 9

Notations of CFG
Non-terminal

Answer 9

symbols represent the phrases, the categories of phrases, or the constituents,
e.g., NP, VP, etc.

Question 10

Notations of CFG
Terminals

Answer 10

symbols are the words,
e.g., car. They
often come from words in a lexicon

Question 11

Notations of CFG
Rewrite rules / productions

Answer 11

rules for replacing nonterminal symbols (on the left
side) with other nonterminal or terminal symbols (on the right side)

Question 12

Notations of CFG
Start symbol:

Answer 12

a special nonterminal symbol that appears in the initial string generated by the grammar: S  [NP VP] | VP

Question 13

derivation

Answer 13

a sequence of rules applied to a string that accounts for that string
* Covers all the elements in the string
* Covers only the elements in the string

Question 14

Parsing

Answer 14

is the process of finding a derivation (i. e. sequence of productions) leading from the START symbol to a
TERMINAL symbol (or TERMINALS to START symbol)

Question 15

Challenges for CFG
Agreement

Answer 15

In English, subjects and verbs have to agree in person and number; Determiners and nouns have to agree in
number. S - NP VP

Question 16

Challenges for CFG
Subcategorization

Answer 16

expresses the constraints that a particular verb (sometimes called the predicate) places on the number and
syntactic types of arguments it wants to take (occur with)

Question 17

Dependency Grammar

Answer 17

• Dependency grammars offer a different
  way to represent syntactic structure
• CFGs represent constituents in a parse
  tree that can derive the words of a
  sentence
• Dependency grammars represent
  syntactic dependency relations between
  words that show the syntactic structure
• Syntactic structure is the set of relations
  between a word (aka the head word) and
  its dependents.

Question 18

Parsing

Answer 18

the process of finding a derivation (i. e. sequence of productions) leading from the START symbol to a
TERMINAL symbol (or TERMINALS to START symbol)

Question 19

Top down parser

Answer 19

• starting from the rules
• Only searches for trees that can be answers
• But also suggests trees that are not consistent
  with any of the words

Question 20

Bottom up parser

Answer 20

• starting from the input token list
• Only forms trees consistent with the words
• But suggest trees that make no sense globally

Question 21

Solutions to parsing problems (1)

Answer 21

1. Solve the problem of
   performance with chart parsers that use a
   special data structure (i.e., chart) to get rid of the backtracking

Question 22

Solutions to parsing problems (2)

Answer 22

2. Solve the problems of predefining CFG or other grammars by using Treebanks and statistical parsing. The main use of the Treebank is to provide the probabilities to inform the statistical parsers

Question 23

Solutions to parsing problems (3)

Answer 23

Partially solve the problems of correctly choosing the best parse trees
by using lexicalization (information about words from the Treebank)

Question 24

Probabilistic CFG (PCPG)

Answer 24

The parsing task is to generate the parse tree with the highest probability (or the top n parse trees)

Question 25

Attach probabilities to grammar rules

Answer 25

The expansions
for a given non-terminal sum to 1
VP -> Verb .55
VP -> Verb NP .40
VP -> Verb NP PP .05

Question 26

The probability of a parse tree:

Answer 26

the product of the
probabilities of the rules used in the derivation

Question 27

Word Sense

Answer 27

the Meaning of a Word -
We say that a word has more than
one word sense (meaning) if there
is more than one definition.

Question 28

Word senses may be

Answer 28

Coarse-grained, if not many distinctions are
made
Fine-grained, if there are many distinctions
of meanings

Question 29

Polysemy:

Answer 29

a word with two or more
related meanings

Question 30

Homonymy:

Answer 30

Words spelled (or
pronounced) the same way but
with different meanings

Question 31

Hypernymy:

Answer 31

A more general term
that encompasses a word

Question 32

Hyponymy:

Answer 32

A more specific term
that is contained within a word

Question 33

How Humans Disambiguate

Answer 33

• local context (e.g., book in a sentence
  that has flight, travel, etc.)
• the sentence or other surrounding text
  containing the ambiguous word
  restricts the interpretation of the
  ambiguous word
• domain knowledge (e.g., plant in a
  biology article)
• the fact that a text is concerned with a
  particular domain activates only the
  sense appropriate to that domain
• frequency data
• the frequency of each sense in general
  usage

Question 34

How Machines Disambiguate

Answer 34

Algorithm for simplified Lesk:
1. Retrieve from machine readable dictionary
all sense definitions of the word to be
disambiguated
2. Determine the overlap between each
sense definition and the current context
3. Choose the sense that leads to highest
overlap

Question 35

Example: disambiguate

Answer 35

PINE
“Pine cones hanging in a tree”
* PINE
1. kinds of evergreen tree with needle-shaped leaves
2. waste away through sorrow or illness

Question 36

WSD

Answer 36

Word Sense Disambiguation

Question 37

Classifier Approach to WSD -1

Answer 37

Train a classification algorithm that
can label each (open-class) word with
the correct sense, given the context of
the word

Question 38

Classifier Approach to WSD -2

Answer 38

Training set is the hand-labeled corpus
of senses

Question 39

Classifier Approach to WSD -3

Answer 39

Result of training is a model that is
used by the classification algorithm to
label words in the test set, and
ultimately, in new text examples

Question 40

Word Similarity Features:

Answer 40

• For each word in the context, compute
  a similarity measure between that
  word and the words in the definitions
  to be disambiguated
• Similarity measures can be defined
  from a semantic relation lexicon, such
  as WordNet (hypernym, hyponym)

Question 41

Syntactic features (relationship between the word and the other parts of the sentence)

Answer 41

Predicate-argument relations: Verb-object,
subject-verb

Heads of Noun and Verb Phrases

Question 42

Collocational features:

Answer 42

Information about words in specific positions (i.e., previous word)

Question 43

Associated words features -1

Answer 43

For each word to be disambiguated, collect a small number of frequently-used
context words.

Question 44

Associated words features -2

Answer 44

Represent these words as a set of words feature