Machine Learning (Coursera): Part 1/2

232 flashcards · Computer Science · by charlie42

1. What h stands for?

   Hypothesis

2. What is h? Example of h

   Function that maps x to y

3. How do we represent the training set?

   A list of m training examples

4. What does this hypothesis function looks like plotted?

5. What does this hypothesis function looks like plotted?

6. What's a cost function? What do we do with it?

   Function that measures the accuracy of the hypothesis function. We want to minimize it.

7. Reasonable cost function for most linear regression functions?

   Mean squared error

8. Cost function for linear regression formula

9. What does each symbol stand for?

10. Equation for slope and what does it mean on the plot

    The higher the slope value, the more vertical the line is.

11. What does a cost function J of one parameter looks like plotted for linear regression?

12. What does a cost function J for linear regression of 2 parameters look like?

13. Alternative way to represent the cost function for linear regression of 2 parameters in 2D?

    Contour plot:

14. The values on one contour line of a contour plot of a function of two variables are

    constant

15. Metaphor for gradient descend

    Looking around and taking a baby step in the direction that is the most downhill

16. Gradient descend algorithm

17. What does α stand for, what does it represent in the gradient descend metaphor

    Learning rate, how large is the baby step

18. Important consideration about the order of calculations in the gradient descend algorithm

    Parameters should be updated simultaneously, don't use the updates first parameter for calculating the second one

19. Two definitions of a derivative

    Instantaneous rate of change Slope of the tangent line

20. Gradient descent formula with one parameter

21. Why the cost function J converges to its minimal value during gradient descend with one parameter?

    When the curve goes down its derivative slope is negative, so the parameter value goes up, where J is lower.

22. What does the derivative part of the gradient descend function contribute?

    It gives the direction of the steepest descend to move towards (the learning rate determines the size of the step)

23. What happens if we set the learning rate too high?

    Parameter may change too much, to a point where the function changed its direction and is further from the minimum

24. Why does the gradient descend for linear regression always converge to the global optimum?

    The cost function for the linear regression is always a convex function (bowl shaped), and doesn't have local optima

25. What does "batch" gradient descend means

    We look at all the training examples to calculate the cost function

26. Final algorithm for linear regression gradient descend

27. One drawback of gradient descend

    May converge to a local minimum, sensitive to small differences in initial values

28. What happens if we set the initial parameters at the local minimum of the cost function?

    Since the slope of the derivative is 0, the values won't change

29. What does this stand for

    Set of all matrices 4 by 2 (4 rows, 2 columns)

30. What is this

    A four-dimensional vector

31. "Scalar" means that an object is a...

    single value, not a vector or matrix

32. Matlab: initialize a matrix

    A = [1,2,3; 4,5,6]

33. Matlab: get an element on the first row second column of a matrix

    A(1,2)

34. Matlab: two ways to store dimensions of a vector/matrix

    [m,n] = size(v) dims = size(v)

35. To add or subtract two matrices, their dimensions must be ...

    the same

36. In scalar multiplication, we simply...

    multiply every element by the scalar value

37. Constraint on dimensions when multiplying a matrix by a vect or

    Number of columns in the matrix must be equal to number of rows in the vector

38. Dimensions of a resulting matrix from multiplying a 3x2 matrix and a 2-dimensional vector

    3-dimensional vector

39. How to get the first value of the resulting vector

    1*1 + 2*3 + 1*2 + 5*1

40. How to solve h for every house using matrix operations

41. What will the code for this look like? Why is it better to use matrices for this operation than a for loop?

    It's computationally faster

42. How to get first two values of the resulting matrix (first row)

    1*1 + 3*0 + 2*5 | 1*3 + 3*1 + 2*2

43. Constraint on dimensions when multiplying two matrices

    Number of columns of the first matrix must be equal to the number of rows of the second matrix

44. Dimensions of a resulting matrix from multiplying a 3x2 matrix and a 2x3 matrix

    3x3

45. Two steps to go from matrix x vector multiplication of matrix x matrix multiplication

    Multiply the first matrix by the vectors from the second matrix one by one Write down the results as vectors

46. How to solve this using matrix operations?

47. Commutative property of multiplication of real numbers

    3x5 = 5x3

48. Is matrix multiplication commutative?

    No

49. Associative property of multiplication of real numbers

    1 x (2 x 3) = (1 x 2) x 3

50. Is matrix multiplication associative?

    Yes

51. Identity for real numbers

    1 (1 x z = z x 1 = z)

52. Identity matrix

53. Matlab: Initialize an identity matrix 2x2

    eye(2)

54. Inverse of a real number

    i * (i-1) = 1

55. Inverse of 0

    Undefined (0-1 = 1/0)

56. Equation for a matrix inverse

    A * (A-1) = (A-1) * A = I

57. Constraint on matrix inverse dimensions

    Only a square matrix can have an inverse

58. How to compute a transpose of a matrix

    Take each row of a matrix and write it as columns of the resulting matrix

59. Formal definition of a transpose

60. Notation: what do m and n stand for?

    m - number of example n - number of features

61. Notation: what does this stand for?

    Value of the feature j of the i-th training example

62. Notation: what does this stand for?

    Input vector of the i-th training example

63. How many dimensions are there in this vector?

    n+1

64. Hypothesis function for multivariate linear regression

65. Vectorized way to write the hypothesis function for multivariate linear regression

66. What makes it possible to write the hypothesis function for the multivariate linear regression as a matrix operation?

    To match the length of features of the training examples and the length of linear regression parameters, we create a "fake" feature x0 = 1

67. Full algorithm for gradient descend for multivariate linear regression

68. Calculate the standard deviation of

69. Why we need to scale features for gradient descend

    If features have very different ranges, the cost function will be unbalanced, the gradient descend can take a long time to converge

70. Example of perfect range for all features

71. Two tricks on features to speed up gradient descend

    feature scaling mean normalization

72. Formula to apply both mean normalization and feature scaling, explain each term

    μ = mean s = range or standard deviation

73. How to tell if the gradient descend is working?

    For each iteration plot the cost function with the current Θ, see if it decreases

74. Fix for cost function increasing with iterations instead of decreasing

    Smaller α

75. Fix for cost function decreasing too slowly with iterations

    Higher α

76. Automatic convergence test

    Declare convergence if J(θ) decreases by less than E in one iteration, where E is some small value such as 10−3. In practice the value is hard to choose.

77. Two things we can do to features to create a better linear regression model

    Combine two features into one (size = length x width) Create additional features from x to make the hypothesis function polynomial

78. Important consideration about features in polynomial linear regression

    You need to scale the features (if x has range 1 - 1000 then range of x2 becomes 1 - 1000000)

79. How to minimize a function?

    Take it's derivative Set it to 0 Solve the equation

80. How to minimize a function with multiple parameters?

    Take partial derivatives of each parameter Set all of the equations to 0 Solve the equations for every parameter

81. Alternative to gradient descend to find Θ at the minimum of the cost function

    Analytically minimize the cost function

82. The equation for minimizing the cost function for linear regression analytically and it's name

    The normal equation

83. What does X look like in the normal equation? What is it called?

    Design matrix. Matrix where each row is a training example. First column is always 1 (x0)

84. Do we need feature scaling when using the normal equation?

    No

85. A advantage of the gradient descend over the normal equation?

    Gradient descend scales well with high n. In normal equation need to compute (XTX)-1, and the cost of computing inverse of a matrix is O(n3)

86. Approximate n that is too large for the normal equation (using modern computers)

    n > 10'000

87. Why matrix inversion function in octave is called pinv?

    There are two functions to invert matrices in octave: inv and pinv. pinv is pseudo-inversion, it will invert a matrix even if it's non-invertable (singular/degenerate)

88. Why would XTX be non-invertible?

    Redundant features (linearly dependent on each other) Too many features (more features than training examples)

89. Octave: comment

    %

90. Octave: not equals

    ~=

91. Octave: and

    &&

92. Octave: how to suppress output?

    End line with a ;

93. Octave: print a string

    disp(string_name)

94. Octave: elements from 1 to 2 with interval 0.1

    (middle value is 1 by default)

95. Octave: matrix filled with 1s, 3 by 2

    ones(3,2)

96. Octave: rand(1,3)

    3x1 matrix of random numbers, uniformly distributed between 0 and 1

97. Octave: randn(1,3)

    Matrix 3x1 of random numbers from the gaussian distribution around 0

98. Octave: histogram with 50 bins

    hist(a, 50)

99. Octave: documentation for the rand function

    help rand

100. Octave: size(A,2)

     number of columns of the matrix A