Machine Learning Supervised Techniques JKU

67 flashcards · Computer Science · by AZAD KARA

1. Which Statements are true? a. The parameters in logistic regression are always positive. b. The maximum-likelihood approach assumes that the data samples all originate from the same distribution. c. Logistic regression also works for multi-dimensional and non-binary inputs. d. Any number between -1 and 1 can be an output of the sigmoid function. e. Stochastic gradient descent provides a cheaper estimate of a gradient by using only a fraction of the training samples. f. Logistic regression ca…

   b. The maximum-likelihood approach assumes that the data samples all originate from the same distribution. c. Logistic regression also works for multi-dimensional and non-binary inputs. E.(FULL)

2. Which statements about supervised machine learning, datasets and crossvalidation are true? b. It is common practice to train a model with the training data set and then check the model's performance on the test set. This is repeated until the test performance can not be improved anymore c. Cross-validation can be used without defining an explicit test set. d. The error on the test set is not the empirical risk. e. We assume all datapoints come from an underlying distribution. In Machine-Learn…

   a,c,d(full)

3. Logistic regression Which statements are true? Multiple answers are possible! a. Logistic regression is a type of linear regression. b. The negative logarithm turns the maximum-likelihood method into a minimization problem and is numerically more stable c. Logistic regression can be used for both classification and regression problems d. Logistic regression can be used for multi-class classification problems. e. Since logistic regression is a convex problem, we need to apply second order method…

   b,d,f(FULL)

4. Which of the following statements are true? (Multiple answers may be correct.) a. RNNs can be trained using backpropagation through time. b. We denote with r(t) the recurrent weight matrix at time t, with f as the activation function s(t−1) as the preactivation at time t−1. For stable gradients,we need c. RNNs are not Turing complete. d.The constant error carousel In LSTMs, must use a non-linear activation function in order to avoid/mitigate the vanishing gradient problem. e. RNNs use weight sh…

   A,B,E,F(FULL)

5. Which statements about Artificial Neural Networks are true? a. Multi-layer perceptrons are equivalent to fully connected feed-forward neural networks. b. Dropout and weight decay are regularization techniques which aim at avoiding underfitting. c. Without activation functions, neural networks are linear functions with respect to their weights. d. The Universal Approximation Theorem guarantees that neural networks can in principle approximate arbitrarily well any continuous function f E [0, 1].…

   a,c,D,f,G(FULL)

6. Which statements about Logistic Regression (and related topics) are true? a. Logistic regression does not have a closed-from solution. b. The logistic (or sigmoid) function reads: o(z) = (1 - exp(-z))-1. c. The momentum term in gradient descent is particularly helpful in steep regions of the loss function. d. The likelihood function for a Bernoulli distribution reads: e. Logistic Regression is a non-convex problem. f. In Logistic Regression, minimization of the loss function can be achieved by…

   a,f,h(FULL)

7. Which statements about ensemble methods, in particular random forests and (gradient) boosting, are true? Multiple answers are possible! a. A common measure of variable importance in random forersts is the mean accuracy decrease. b. In AdaBoost, the more accurate the weak classifiers are the smaller the influence they are given for the classification output. c. A random forest has smaller variance than the individual trees if the correlation between the trees is large. x d. Forward stagewise add…

   A,F,G,H(FULL)

8. Convolutional Neural Networks (CNNs): Which statements are true? Multiple answers are possible! a. Transfer Learning allows to use a kernel from one convolutional layer also in the next (neighbouring) layer. b. With input dimension A, kernel size R, padding P, and stride S, the feature map dimension is (A + R+ 2P)/S + 1. c. ResNet is a famous CNN, whose good learning performance is due to skip connections. d. A convolutional layer can use multiple kernels. e. Re-using a kernel weight matrix at…

   C,D,E,F,H(FULL)

9. Which of the following statements about Logistic Regression are correct? Wählen Sie eine oder mehrere Antworten: a. The optimal weights for a Logistic Regression model can be calculated by solving the normal equation. b. Due to the non-linearity of the sigmoid function, a Logistic Regression model is able to perfectly solve the XOR problem. c. The optimal weights of a Logistic Regression model cannot be directly calculated; instead, optimization algorithms are typically used for this purpose. d…

   C,E(FULL)

10. Which of the following statements about Random Forests are correct? Wählen Sie eine oder mehrere Antworten: a. The bootstrap subsample from the training data set is drawn at the beginning of the training of a Random Forest and kept constant throughout the training procedure of the whole Random Forest. b. The bootstrap subsample from the training data set is drawn at the beginning of the training of a new tree and kept constant throughout the training procedure of this tree. c. A new subsample o…

    B,C(FULL)

11. Tick the correct statements related to the vanishing gradient problem of feed-forward neural networks (NNs). As usual, the numbering of the layers starts at 1 for the input layer. Wählen Sie eine oder mehrere Antworten: a. One reason why several well-known activation functions different from sigmoid can lead to more stable training can be the fact that their derivatives are always larger than 1. b. The sigmoid function is known to induce vanishing gradients because its derivative is strictly sm…

    B,D(FULL)

12. Which of the following statements about Logistic Regression are correct? Wählen Sie eine oder mehrere Antworten: a. The optimal weights for a Logistic Regression model can be calculated by solving the normal equation. b. Due to the non-linearity of the sigmoid function a Logistic Regression model is able to perfectly solve the XOR problem. c. The optimal weights of a Logistic Regression model can not be directly calculated, instead optimization algorithms are typically used for this purpose. d.…

    C,E(FULL)

13. Tick the correct statements related to the vanishing gradient problem of feed forward neural networks (NNs). As usual, the numbering of the layers starts at 1 for the input layer. Wählen Sie eine oder mehrere Antworten: a. One reason why several well-known activation functions different from sigmoid can lead to more stable training can be the fact that their derivatives are always larger than 1. b. Assume that you have a three-layer feed-forward NN (i.e. one hidden layer), where the average abs…

    E,F(FULL)

14. Feed-forward networks Which statements are true? Multiple answers are possible! a. The perceptron learning algorithm tries to find the optimal weights for a linear threshold unit. b. Dropout is a regularization technique where random data features are set to zero during training. c. Backpropagation is a recursive procedure so that knowing gradients of weights in lower layers allows for computing the gradients in upper layers. d. For weight decay, a regularization term that measures the overall…

    A,D,E(FULL)

15. Logistic regression Which statements are true? Multiple answers are possible! a. The parameters in logistic regression are always positive. b. The maximum-likelihood approach assumes that the data samples all originate from the same distribution. c. Logistic regression also works for multi-dimensional and non-binary inputs. d. Any number between -1 and 1 can be an output of the sigmoid function. e. Stochastic gradient descent provides a cheaper estimate of a gradient by using only a fraction…

    B,C,E(FULL)

16. Which statements about ensemble methods, in particular random forests and (gradient) boosting, are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. A random forest in general has smaller bias than the individual trees. b. A common measure of variable importance in random forersts is the mean decrease in Gini impurity.  c. In AdaBoost, the less accurate the weak classifiers are, the higher the influence they are given for the classification output. d. In a random…

    B,D,F(Tüm cevaplar tamam)

17. Which statements about Logistic Regression (and related topics) are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. In Logistic Regression, minimization of the loss function can be achieved by gradient descent.  b. Logistic Regression is a non-convex problem as can be seen by the positive-semidefiniteness of the Hesse matrix. c. The logistic function reads:σ(z) = (1 − exp(z))−1 . It is also known as the sigmoid function d.Assume that data are generated around a…

    A,F,H (SORUNUN CEVAPLARI TAMAMEN DOĞRU)

18. Which statements about Perceptrons and Perceptron Learning are true? Notation as in the lecture: Weight vector: w Threshold: θ Data samples: xi Learning rate: η > 0 Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. In the learning algorithm: If g(xi; w, θ) = 0 and yi = 1 , then the uptdate rule for the weight vector reads: w := w + η xi.  b. If the data set is linearly separable, then there exists a stopping criterion such that with probability 1 the perceptron learning…

    A,B,F (TAMAMI DOĞRU)

19. Which statements about Artificial Neural Networks are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Double descent is the phenomenon that the test risk first decreases with increasing number of parameters, then increases up to the interpolation threshold, and then decreases again. The final decrease takes place in the so-called over-parametrized regime, where there are more data points than parameters. b. Multi-layer perceptrons are equivalent to fully connecte…

    B,D,E (TAMAMI DOĞRU)

20. Convolutional Neural Networks (CNNs): Which statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Re-using a kernel weight matrix multiple positions of an image is called weight sharing. b. With kernel weight matrix , kernel size , and input image , the feature map preactivation is caluclated by . c. Dataset augmentation can help to overcome the problem of limited quantity and/or diversity of given data.  d. AlexNet is a famous CNN, whose good learning…

    A,B,C,D (TAMAMI DOĞRU)

21. Recurrent Neural Networks (RNNs): Which statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. The constant error carousel in LSTMs uses a non-linear activation function in order to avoid/mitigate the vanishing gradient problem. b. We denote with R the recurrent weight matrix, with f the activation function, and with s(t-1) the preactivation at time t-1 . For stable gradients, we need ∥R diag(f (s(t−1))∥ ≈ 1.  c. RNNs can be trained via backprogation thr…

    B,C,G,H (TAMAMI DOĞRU)

22. Which statements about the generalization e pirical risk are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. The generalization error is the expected loss on future data for a given model. b. The generalization error has the form c.The empirical risk on a dataset with n samples has the form d. A common assumption for estimation of the generalization error via the empirical risk is that data are independent and identically distributed (i.i.d.). A good estimation o…

    A,D,F (TAMAMI DOĞRU)

23. Which statements about supervised machine learning are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ✅ a. Having a small training error but a large test error indicates overfitting. ⬜ b. Bias increases with increasing model class complexity. ✅ c. ROC curves allow to evaluate a classifier independent of class distribution and independent of misclassification cost. ⬜ d. Model variance decreases with increasing model class complexity. ✅ e. Supervised machine learni…

    A,C,E,G(TAMAMI DOĞUR)

24. Which statements about the probabilistic framework and the Bayes optimal classifier are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ⬜ a. Using the zero-one loss and binary classification, the minimal risk is given by ⬜ b. Bayes Theorem reads: ​ ✅ c. The Bayes Optimal Classifier minimizes the misclassification probability, i.e., it minimizes the risk. ⬜ d. Given two variables, the marginal distribution of one is obtained by integrating (or summing) the conditio…

    C

25. Which statements about support vector machines (SVMs) are true? Multiple answers are possible! a.Kernels allow to transform data into a lower-dimensional space where separability can easier be achieved. b. A sample xi, is a support vector if the corresponding Lagrange parameter ai is larger than 0. c. SVMs work effectively if the number of dimensions (features) d is much larger than the number of samples l. d. For I data points with features xi, labels yi, and Lagrange parameters ai, the linear…

    B,C,E,F,G(HEPSİ DOĞRU)

26. Consider multi-class classification with Support Vector Machines. Let there be MM M classes. Which of the following statements are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ✅ a. In the one-versus-one approach, there are M(M−1)/2M(M - 1)/2 M(M−1)/2 classifiers. ✅ b. In the one-versus-all approach, there are MM M classifiers. ⬜ c. In the one-versus-one approach, the final classification is done by choosing the class which produces the largest value of all clas…

    A VE B (HEPSİ DOĞRU)

27. Which statements about decision trees are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ✅ a. Variance reduction is a common splitting criterion for regression. ✅ b. Shallow trees tend to underfit, deep trees tend to overfit. ⬜ c. Decision trees can only be applied to categorical attributes. ✅ d. For classification tasks, a new input sample can be assigned to a class value or to a conditional probability. ⬜ e. The number of leaf nodes cannot be larger than twice…

    A,B,D,H (HEPSİ DOĞRU)

28. Which statements about ensemble methods, in particular random forests and (gradient) boosting, are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ✅ a. In AdaBoost, the more accurate the weak classifiers are, the larger the influence they are given for the classification output. b. Random Forests are generally harder to tune compared to Gradient Boosted Trees. ⬜ c. Forward stepwise additive modeling is a robust boosting method that specifically depends on the quad…

    A,B,D,G,H

29. Which statements about AdaBoost are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ✅ a. In every step n, compute a misclassification error and classifier weight If classification for a data point was correct, leave the corresponding weight unchanged. If classification was wrong, increase weight. ✅ b. Final output: ✅ c. Initialization: Weights are all initialized to the same value and sum up to 1. ✅ d. In every step, with given weights, find the optimal classifier…

    A,B,C,D(TAMAMI DOĞRU)

30. Which statements about Logistic Regression (and related topics) are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ✅ a. In Logistic Regression, minimization of the loss function can be achieved by numerical methods such as gradient descent and conjugate gradient. ✅ b. Assume that y data are generated around a function g(x) with Gaussian noise. Minimizing the negative log-likelihood of observing the data is equivalent to minimizing the squared error between y and…

    A,B,D(HEPSİ DOĞRU)

31. Which statements about Artificial Neural Networks are true? (Multiple answers are possible!) Wählen Sie eine oder mehrere Antworten: ⬜ a. Dropout and weight decay are regularization techniques which aim at avoiding underfitting. b.Let us denote the learning rate with n> 0, the weights with w, and the empirical risk with R. Then the gradient descent update rule reads: c.Using the matrix-vector notation from the lecture (Unit 6),the recursive formula for delta errors is given by: . Here, Jl is th…

    C,D,E,H(TAMAMI DOĞRU)

32. Which statements about Perceptrons and Perceptron Learning are true? Notation as in the lecture: Weight vector: w Threshold: θ Data samples: xi Learning rate: η > 0 Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ⬜ a. If the data set is linearly separable, then there exists a stopping criterion such that with probability 1 the perceptron learning algorithm will terminate and solve the learning task. b.In the learning algorithm: lf g(xi; w, 0) = 1 and yi= 0, then the uptda…

    D

33. Convolutional Neural Networks (CNNs): Which statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ✅ a. Dataset augmentation can help to overcome the problem of limited quantity and/or diversity of given data. b.With kernel weight matrix the feature map preactivation is caluclated by c. With input dimension A, kernel size R, padding P, and stride S, the feature map dimension is (A+ R+ 2P)/S+1. d. Re-using a kernel weight matrix at all positions of an image i…

    A,D,E,F,H

34. Recurrent Neural Networks (RNNs): Which statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ⬜ a. In an RNN, the units in a hidden layer are all connected with each other but cannot be connected to themselves. ⬜ b. The constant error carousel in LSTMs must use a non-linear activation function in order to avoid the vanishing gradient problem. ✅ c. RNNs can be trained via Backpropagation Through Time. d.We denote with R the recurrent weight matrix at time t,…

    B,F,G,H(TAMAMI DOĞRU)

35. Choose the correct statements involving FNNs, CNNs, and RNNs (several may be correct): Select one or more: ⬜ a. Sliding a kernel with three channels over an input image of three channels produces an output image that also consists of three channels. ⬜ b. The weight matrices that process the inputs in an RNN are the same for each sequence. ⬜ c. In an LSTM block, the current cell state is computed by multiplying together the previous cell state with the current cell input. This procedure prevents…

    D,E,G (7.5 OUT OF 10)

36. Tick the correct statements related to ensemble methods (several may be correct): Select one or more: ⬜ a. To construct the out-of-bag random forest predictor corresponding to a specific data sample z, you only average trees which are trained on data that contain z. ⬜ b. In AdaBoost, models with larger error rates get assigned the larger model weights. ⬜ c. In each iteration step, forward stagewise additive modeling tries to compute the optimal weights and basis functions to add to the model fr…

    F,G(5 OUT OF 10)

37. Which statements about support vector machines (SVMs) are true? Multiple answers are possible! a. SVMs work effectively if the number of dimensions (features) d is much smaller than the number of samples l. b. SVMs allow for a probabilistic explanation for classification. c. The Gaussian kernel has the form: d. One of the KKT conditions has the form: e. A sample is a support vector if and only if the corresponding Lagrange parameter is strictly larger than 0. f. Support vector regression uses t…

    C,E,F,G

38. Consider multi-class classification with Support Vector Machines. Let there be classes. Which of the following statementsare true? Multiple answers are requested! Wählen Sie eine oder mehrere Antworten: ⬜ a. In the one-versus-one approach, there are M - 1 classifiers. ⬜ b. In the one-versus-one approach, there are M classifiers. ✅ c. In the one-versus-one approach, there are M(M - 1)/2 classifiers. ⬜ d. In the one-versus-one approach, there are M(M + 1)/2 classifiers. ⬜ e. In the one-versus-all…

    C,F,J,K(4OUT OF 4)

39. Which statements about decision trees are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Variance reduction is a common splitting criterion for regression. b. The number of leaf nodes cannot be larger than twice the number of internal nodes. c. All decision tree learning algorithms are non-recursive. d. For classification tasks, a new input sample can be assigned to a class value or to a conditional probability. e. A split maximizes the information gain if the p…

    A,D,E,H (4 OUT OF 4)

40. Which statements about Logistic Regression (and related topics) are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Logistic Regression is a convex problem. b. The logistic function reads: . It is also known as the sigmoid function. c. The likelihood function for a Bernoulli distribution reads: d. The softmax function is a generalization of the sigmoid function and is suitable for multi-class classification. e. In Logistic Regression, minimization of the loss fun…

    A,B,D,E (4 OUT OF 4)

41. Which statements about Artificial Neural Networks are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Without activation functions, neural networks are linear functions with respect to their weights. b. Dropout and weight decay are regularization techniques which aim at avoiding underfitting. c. Multi-layer perceptrons are equivalent to fully connected feed-forward neural networks. d. The vanishing gradient problem can be mitigated/avoided by the use of suitable…

    A,C,D,G(4 OUT OF 4)

42. Convolutional Neural Networks (CNNs): Which statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Re-using a kernel weight matrix multiple positions of an image is called weight sharing. b. Transfer Learning allows to use pretrained kernels from other networks which were trained on different data sets. c. With input dimension A, kernel size R, padding P, and stride S, the feature map dimension is d. With kernel weight matrix wi,j, kernel size R, and inpu…

    A,B,D,G(4 OUT OF 4)

43. Recurrent Neural Networks (RNNs): Which statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. RNNs use weight sharing as the network is slid over the input sequence and the hidden-layer weight matrix isreused for all timesteps. b. In an RNN, the units in a hidden layer are all conneted with each other but cannot be connected to themselves. c.The operations complexity of backpropagation through time for an RNN with N hidden units and sequence length T is…

    A,C,D,H(4 OUT OF 4)

44. Which of the following statements about Logistic Regression are correct? Wählen Sie eine oder mehrere Antworten: a. Due to the non-linearity of the sigmoid function a Logistic Regression model is able to perfectly solve the XOR problem. b. The optimal weights for a Logistic Regression model can be calculated by solving the normal equation. c. The loss function used in Logistic Regression is convex and therefore every minimum of the loss function is a global minimum. d.When training a Logistic R…

    C,E(50 OUT OF 50)

45. Tick the correct statements related to the vanishing gradient problem of feed forward neural networks (NNs). As usual, the numbering of the layers starts at 1 for the input layer. Wählen Sie eine oder mehrere Antworten: a. One reason why several well-known activation functions different from sigmoid can lead to more stable training can be the fact that their derivatives are always larger than 1. b. The sigmoid function is known to induce vanishing gradients, because its derivative is strictly s…

    B,D(50 OUT OF 50)

46. Which of the following statements are correct? Select one or more: a. The generalization error is also called empirical risk. b. The training data set and the test data set should be i.i.d.,i.e. they should be distributed as if theywere drawn independently from the same distribution. c. The training data set and the test data set have to be disjoint, i.e. they must have no data samples in common. d. Any two data sets that are independent are useful as test and training data sets. e. Underfittin…

    b,c,f(50 out of 50)

47. Which of the following statements about Logistic Regression are correct? Select one or more: a.The optimal weights of a Logistic Regression model can not be directly calculated, instead optimization algorithms are typically used for this purpose. b.Due to the non-linearity of the sigmoid function a Logistic Regression model is able to perfectly solve the XOR problem. c.Logistic regression models the linear relationship between a categorical label and some features. d. When training a Logistic R…

    A,E(50 OUT OF 50)

48. Which of the following statements about Random Forests are correct? Select one or more: a. A new bootstrap subsample of the training data set is drawn at each individual split during training of a tree. b. A new subsample of considered features from the training data set is drawn at each individual split during the training of a tree. c. For out of bag estimates for each observation 2 one constructs the RF predictor by averaging only trees corresponding to bootstrap samples in which the observa…

    B,D(FULL)

49. Which of the following statements about the generalization error is true (tick for true, leave empty for false)? Select one or more: a. Cross validation is a method to estimate the generalization error.  b. Increasing the model class complexity always decreases the generalization error. c. Too small model class complexity leads to underfitting.  d. Regularization increases the gap between generalization error and training error. e. The generalization error can be calculated exactly using the…

    A,C(50 OUT OF 50)

50. Which of the following changes would commonly cause an SVM’s margin to shrink? a. C-SVM: increasing the value of C.  b. Hard margin SVM: adding a sample point that violates the margin. c. Hard margin SVM: adding a new feature to each sample point. d. C-SVM: decreasing the value of C

    A,B (50OUT OF 25)

51. Which of the following statements about Random Forests are correct? Select one or more: a. The bootstrap subsample from the training data set is drawn at the beginning of the training of a new tree and kept constant throughout the training procedure of this tree.  b. A new bootstrap subsample of the training data set is drawn at each individual split during training of a tree. c. A new subsample of considered features from the training data set is drawn at each individual split during the trai…

    A,C(50 OUT OF 50)

52. Which of the following statements about Logistic Regression are correct? Select one or more: a. The optimal weights for a Logistic Regression model can be calculated by solving the normal equation. b. Due to the non-linearity of the sigmoid function a Logistic Regression model is able to perfectly solve the XOR problem. c. The optimal weights of a Logistic Regression model cannot be directly calculated, instead optimization algorithms are typically used for this purpose.  d. The loss function…

    C,D(50 OUT OF 50)

53. Which statements about the generalization error and the empirical risk are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ✅ a. The generalization error is the expected loss on future data for a given model. ⬜ b. Estimating the risk via samples from a test set, which have not been used in training, avoids the problem of underfitting. ⬜ c. The generalization error has the form ✅ d. The empirical risk on a dataset with n samples has the form ✅ e. A common assumption f…

    A,D,E,G (4 OUT OF 4)

54. Which statements about supervised machine learning are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. ROC curves allow to evaluate a classifier independently of class distribution and independently of misclassification cost. They plotTPR versus TNR. b. Cross validation can help to opimize model hyperparameters. c. Having a small training error but a large test error indicates the problem of underfitting. d. Model variance typically increases with increasing mode…

    B,D(2 OUT OF 4)

55. Which statements about support vector machines (SVMs) are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. One of the KKT conditions has the form: b. In the dual problem, the dual function L is maximized with respect to w and b. c. Support vector regression uses the magnitude (and not only the sign) of the disrciminant function. d. A sample xi is a support vector if the corresponding Lagrange parameter αi is strictly larger than 0. e. Kernels allow to transform da…

    C,D,G,F(4 OUT OF 4)

56. Consider multi-class classification with Support Vector Machines. Let there be M classes, Which of the following statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Training pairwise classifiers is usually less time costly due to smaller training sets. b. In the one-versus-one approach, the final classification is done by choosing the class which produces the largest value of all classifiers. c. In the one-versus-all approach, the final classification…

    A,E,F(4 OUT OF 4)

57. Which statements about decision trees are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. The number of leaf nodes in a tree cannot be larger than 2s with S the number of splits. b. A split maximizes the information gain if the produced subsets are completely homogeneous. c. Decision tree learning algorithms recursively split the training set into subsets. d. The maximal Gini impurity of a dataset with M classes is 1/M. e. The maximal entropy of a dataset with MM…

    B,C,E,H(4 OUT OF 4)

58. Which statements about the generalization error and the empirical risk are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ✅ a. The generalization error is the expected loss on future data for a given model. ✅ b. The generalization error has the form: ✅ c. The empirical risk on a dataset with nn n samples has the form: ✅ d. A common assumption for estimation of the generalization error via the empirical risk is that data are independent and identically distributed (…

    A,B,C,D,F(4 OUT OF 4)

59. Which statements about supervised machine learning are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ⬜ a. Model variance decreases with increasing model class complexity. ⬜ b. The more degrees of freedom, the lower the risk to fit to noise. ⬜ c. Supervised machine learning uses explicit knowledge to design models deductively. ✅ d. The Bayes Optimal Classifier is a probabilistic model that makes the most probable prediction for a new sample. It requires the use of…

    F,G,H(4 OUT OF 4)

60. Which statements about support vector machines (SVMs) are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: a. Kernels allow transforming data into a lower-dimensional space where separability can be easier achieved. b. The Gaussian kernel has the form: c. The mathematical problem (constrained convex optimization) has a global solution. d. SVMs allow for a probabilistic explanation for classification. e. A sample x_i is a support vector if and only if the correspondin…

    B,C,E,F,G,H(4 OUT OF 4)

61. Consider multi-class classification with Support Vector Machines. Let there be M classes. Which of the following statements are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ✅ a. In the one-versus-one approach, there are M(M - 1)/2 classifiers. ✅ b. In the one-versus-all approach, there are M classifiers. ⬜ c. In the one-versus-one approach, the final classification is done by choosing the class which produces the largest value of all classifiers. ⬜ d. In the one-…

    A,B(4 0UT OF 4)

62. Which statements about decision trees are true? Multiple answers are possible! Wählen Sie eine oder mehrere Antworten: ✅ a. Decision trees perform well on large data sets. ⬜ b. Decision trees can be applied to categorical, but not to numerical features. ✅ c. A split maximizes the information gain if the produced subsets are completely homogenous. ⬜ d. Shallow trees tend to overfit, deep trees tend to underfit. ✅ e. Decision tree learning algorithms recursively split the training set into subset…

    A,C,E,H(4 OUT OF 4)

63. Which statements about ensemble methods, in particular random forests and (gradient) boosting, are true? Multiple answers are possible! Select one or more: a. A common measure of variable importance in random forests is the mean accuracy decrease. b. In Boosting, the main idea is that those samples which led to a small error (i.e., were classified well) in an iteration get a larger weight in the next iteration. c. In a random forest, each individual decision tree is built upon a random bootstra…

    A,C,G,H(4 OUT OF 4)

64. Which statements about Logistic Regression (and related topics) are true? Multiple answers are possible! Select one or more: ❌ a. The softmax function is a generalization of the sigmoid function and is suitable for regression. ✅ b. The momentum term in gradient descent is particularly helpful in flat regions of the loss function. ❌ c. The logistic (or sigmoid) function reads: ✅ d. Assume that y data are generated around a function g(x) with Gaussian noise. Minimizing the negative log-likelihood…

    B,D,E,G(4 OUT OF 4)

65. Which statements about Artificial Neural Networks are true? Multiple answers are possible! Select one or more: ✅ a. Dropout and weight decay are regularization techniques which aim at avoiding overfitting. ✅ b. Multi-layer perceptrons are equivalent to fully connected feed-forward neural networks. ✅ c. Without activation functions, neural networks are linear functions with respect to their weights. ❌ d. Let us denote the learning rate with η>0, the weights with w, and the empirical risk with R.…

    A,B,C,G(4 OUT OF 4)

66. Convolutional Neural Networks (CNNs): Which statements are true? Multiple answers are possible! Select one or more: a. Transfer Learning allows to use a kernel from one convolutional layer also in the next (neighbouring) layer. b. AlexNet is a famous CNN, whose good learning performance is due to skip connections. c. With kernel weight matrix wi,j​, kernel size R, and input image xa,b​, the feature map preactivation is calculated by; d. Pooling decreases the receptive field through the depth of…

    C,E,G(4 out of 4)

67. Recurrent Neural Networks (RNNs) Which statements are true? Multiple answers are possible! Select one or more: a. The constant error carousel in LSTMs must use a non-linear activation function to avoid or mitigate the vanishing gradient problem. b. We denote with R(t)R(t) R(t) the recurrent weight matrix at time t, with f as the activation function, and with s(t−1) as the preactivation at time t−1. For stable gradients, we need: c. RNNs use weight sharing in the sense that the network is slid o…

    B,C,E,F,G(4 OUT OF 4)