# LDA2VEC Flashcards

1
Q

What is Latent Dirichlet Allocation (LDA)?

A

LDA is a probabilistic topic model and it treats documents as a bag-of-words.

2
Q

What is Latent Dirichlet Allocation (LDA) to vector (lda2vec)?

A

lda2vec builds document representations on top of word embeddings.

3
Q

What is a topic modeling?

A

Topic models often assume that word usage is correlated with topic occurence. It divides the documents into clusters according to word usage.

4
Q

What is Bag-of-words?

A

Document as a vector with dim = vocabulary size. Each dimension of this vector corresponds to the count or occurrence of a word in a document.

5
Q

LDA is a general Machine Learning (ML) technique, How?

A

Used in unsupervised ML problems where the input is a collection of fixed-length vectors and the goal is to explore the structure of this data.

6
Q

What is the biggest disadvantage of LDA?

A

The LDA model learns a document vector that predicts words inside of that document while disregarding any structure or how these words interact on a local level.

7
Q

When LDA produce value?

A

[1] A good estimate of the number of topics
[2] Manually assign a distinct nominator/‘topic’ to the different topic vector.
[3] The topic vectors will be interpretable

8
Q

What is the problem in bag-of-words representation?

A

Figuring out which dimensions in the document vectors are semantically related. (solution: word embeddings)

9
Q

What is word embeddings?

A

Words that occur in the same context are represented by vectors in close proximity to each other.

10
Q

How to visulize word embedding space?

A

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a dimensionality reduction method that you can use to visualise high-dimensional data in 2D.

11
Q

What is CBOW in word2vec embedding?

A

In the bag-of-words architecture (CBOW) the pivot word is predicted based on a set of surrounding context words (i.e. given ‘thank’, ‘such’, ‘you’, ‘top’ the model has to predict ‘awesome’)

12
Q

What is skip-gram in word2vec embedding?

A

In the skip-gram architecture, the pivot word is used to predict the surrounding context words (i.e. given ‘awesome’ predict ‘thank’, ‘such’, ‘you’, ‘top’ ).

13
Q

A

The word2vec model learns a word vector that predicts context words across different documents. As a result, document-specific information is mixed together in the word embeddings.

14
Q

what is lda2vec?

A

Inspired by Latent Dirichlet Allocation (LDA), the word2vec model is expanded to simultaneously learn word, document and topic vectors.

15
Q

what is Lda2vec process?

A

The pivot word vector and a document vector are used to obtain a context vector. This context vector is then used to predict context words.

16
Q

What are the components of a document vector?

A

A document vector is decomposed into a document weight vector and a topic matrix.

17
Q

The document weight vector represents:

A

the percentage of the different topics.

18
Q

The topic matrix consists

A

of the different topic vectors

19
Q

A context vector is

A

constructed by combining the different topic vectors that occur in a document.

20
Q

What is the end-result of the lda2vec?

A

a set of sparse document weight vectors, as well as easily interpretable topic vectors.

21
Q

A

https: //github.com/meereeum/lda2vec-tf
https: //github.com/cemoody/lda2vec

22
Q

Best lda2vec tutorial:

A

https://www.datacamp.com/community/tutorials/lda2vec-topic-model