Coursera Stanford ML: Neural Networks

50 flashcards · Computer Science · by charlie42

1. Visually imagine a plot that explains why neural networks are chosen over traditional models for large datasets

2. What's the point of the hidden layers, why is this better than just running one logistic regression?

   Automatic feature engineering -- discovering feature combinations that work together

3. Visually imagine plot of sigmoid function

4. Visualize how to calculate this dot product

5. Imagine what this looks like in vectorized form:

6. Imagine formula for z (x-axis)

7. Notation: what the superscript square bracket numbers refer to

   Number of layer

8. Imagine the relationship between neural network and logistic regression

   Each activation value is calculated with a logistic regression formula, where the input is the activation values from the previous layet

9. Write the general equation for the j-th activation value of the l-th layer

10. Write the equation for the second activation value of the third layer

11. What's the difference between these two structures:

    First one is a 2D matrix with 1 column, the second one is a 1D array.

12. In a neural network with 2 layers, what steps model.predict(NN) performs?

    Calculate activation values of the first layer, with x as input Calculate activation values of the last layer, with a[1] as input

13. Which parameters each neuron in a NN layer has?

    w coefficient for each input value one value of b (intercept)

14. Visualize the relation between the dot product of two vectors are matrix multiplication

    Dot product of two vectors is equivalent to multiplying the transpose of the first vector by the second vector

15. Visualize how to multiply a horizontal vector by a matrix

    Each value of the result vector is the result of multiplication of the vector by each column of the matrix

16. Visualize how to find dimensions of the resulting matrix

    Number of rows come from the first matrix, number of columns come from the second matrix

17. Visualize why the inner dimensions need to be the same

    Because you can only get dot product of vectors of the same length

18. What values do we need to calculate A[1]1?

    Input vector a[0] or x Vector of weights w[1]1 b[1]1

19. How do you write weights of the first layer in a matrix?

    It's reversed: each column is a neuron each row is a weight (as many rows as there are inputs)

20. Imagine vectorized formula for calculating activation values for layer 1

21. Visually imagine why this works:

    For each new activation value (columns), each input activation values are multiplied by corresponding weights and added up

22. g(100)? (g is the sigmoid function)

    1

23. Imagine a neural network created by this code

24. What would be the dimensions of the matrix of weights for the layer 1?

    3 columns, number of rows = number of activation values in the previous layer

25. What's the difference between loss and cost functions?

    Loss function is for one training example Cost function is for the whole dataset

26. Visually imagine the plot of the ReLU activation function

27. ReLU(10) = ?

    10

28. ReLU(-100) = ?

    0

29. Which activation function to use for the last layer in case of a regression problem?

    Linear, or ReLU if positive values are not possible (rent price):

30. Which activation function is the most common for hidden layers?

    ReLU

31. Why using activation functions at all in case of regression?

    Combination of linear models is a linear model, so a NN like this won't be able to learn non-linear relationships. Activation functions are non-linear, so they introduce non-linearity.

32. Think of the phrase that helps to calculate log21

    2 to which power gives 1?

33. log22 = ?

    1

34. log24 = ?

    2

35. log21 = ?

    0

36. log216 = ?

    4

37. Imagine graph of log2x

38. Explain why the log2x stagnates like this

    x increases fast y is the exponent, and x is the result of raising 2 to this exponent. The result will increase faster than the exponent Inverse of this graph:

39. Explain why log10 is less steep than log2?

    When y=log2x increases by 1, x =2y becomes x = 2y+1, i.e. x (2y) is multiplied by 2 When y=log10x increases by 1, x = 10y becomes x = 10y+1, i.e. x (10y) is multiplied by 10

40. What's the relation between logistic regression and softmax regression?

    Softmax regression is a generalization of logistic regression. Logistic regression works only with 2 classes, softmax works with any number of classes.

41. Keeping in mind that softmax is a generalization of logistic regression, what does the loss function for logistic regression look like?

42. How is softmax activation function different from other activation functions?

    Other activation functions only need the z (wx+b) value of the current node. Softmax needs all z values of the layer

43. What's a logit?

    z -- value of the node (wx+b) before applying activation function

44. What did this code look like before the optimization?

    activation = 'softmax' loss = SparseCategoricalCrossentropy()

45. Why do we want to do this instead of using the sigmoid activation function:

    We include the activation function formula inside the loss function calculation, to avoid extra steps in calculation (rounding errors)

46. How do we need to change the prediction part of the code with this optimization?

    Our neural network outputs the logits directly, without applying softmax. We need to apply it to predicted values.

47. How does the Adam algorithm improve on gradient descent?

    It has separate learning rate for each parameter. If parameter moves slowly in one direction, the learning rate is increased; if it oscillates, the learning rate is too high and needs to be decreased.

48. Imagine a dense neural network layer

49. Imagine a convolutional NN layer

    Each neuron output is a function of a subset of the activation outputs of the previous layer

50. Imagine a simplified explanation of how to find a derivative of a function at a certain point?

    Divide difference in f(x) when increasing x a little bit by the difference in x