NLP with Deep Learning (Stanford CS224N)

20 flashcards · Computer Science · by charlie42

  1. Visually imagine WordNet?

  2. How do similar words end up together in the vector space?

    They are close in vector space if the words often used around them are similar

  3. Visualize how to calculate cosine of an angle in a right triangle

    cos⁡(θ)=adjhyp\cos(\theta) = \frac{{\text{adj}}}{{\text{hyp}}}cos(θ)=hypadj​

  4. Visualize how to get the formula for adjacent from the formula of cosine

    cos⁡(θ)=adjhyp\cos(\theta) = \frac{{\text{adj}}}{{\text{hyp}}}cos(θ)=hypadj​ adj=hyp⋅cos⁡(θ)\text{adj} = \text{hyp} \cdot \cos(\theta)adj=hyp⋅cos(θ)

  5. Visualize dot product of two vectors at an acute angle

  6. Visualize how to calculate dot product algebraically?

  7. Word2vec notation: what do c and o stand for?

    c = central word o = context word ("outside")

  8. Word2vec notation: what does t stands for?

    t = position of a word in text

  9. Word2vec notation: what does this equation mean in words?

    Probability of the word wt+j being in position t+j given that the word wt is in position wt+j (probability that the word banking is at t+1 when the word into is at t)

  10. Word2vec notation: what m stands for (in the second product)?

    Context window -- how many words around t we care about

  11. Word2vec notation: what is θ\thetaθ ?

    The vectors we're learning

  12. What is the optimization objective of word2vec in words?

    Find the most likely (=maximize likelihood) vectors ( θ\thetaθ ) that produce the probabilities of each word being surrounded by certain words in the given text (T)

  13. Word2vec notation: what vw and uw represent?

    Word vectors:

  14. Word2vec: main part of the equation for calculating the probability that a word is often next to another word in a certain corpus (text)?

    Dot product of their two vectors The rest of the equation just makes the dot product look like a probability: exponent makes it positive but also maximizes probability of large similarities dividing by sum of dot products of all words in the vocabulary makes it between 0 and 1

  15. Visually imagine the softmax function

    Produces a probability distribution from any set of numbers, where higher numbers are given very high probabilities and lower numbers are given a very low probabilities

  16. Visually imagine the θ\thetaθ that we're trying to find?

    V (length of vocabulary) d-dimensional vectors Each word appears twice -- for when it's the center word or context word

  17. How can you tell on the first glace if these two vectors are similar?

    They have the same sign for values in the same dimensions

  18. Visually imagine the difference between CBOW (continuous bag of words) and skip-gram Word2Vec models?

  19. Visually imagine an n-gram

  20. Visually imagine how two different senses of a word apple are encoded in Word2Vec

    Apple = Apple (technology) + Apple (fruit) Apple (fruit) = Apple - Technology