Scikit-learn MOOC

https://inria.github.io/scikit-learn-mooc/index.html

57 flashcards · Computer Science · by charlie42

1. What variable called "_" (underscore) is for?

   For storing things we're not interested in, garbage

2. A tool to quantify fairness issue in your model

   Fairlearn.org

3. Tool used to explore relationship between two categorical variables

   Contingency table

4. Pandas function to print a contingency table

   pd.crosstab

5. What is this plot called?

   pairplot

6. What pairplot displays?

   Relationships between each two variables, plus distribution of the class for each pair

7. What pairplot is used for? Example.

   To explore relationships between input and output variables. Here we could even manually predict class from some pairs

8. Explain this code:

   We divide the number of correct predictions by the total number of predictions. It just happens to be the same formula as mean.

9. Arguments of the model.score method

   X_test, y_test

10. What's generalization performance of a model?

    Score obtained on the test set

11. What's computational performance of a model?

    Computational cost of training a model or to use it to make predictions

12. How to get a baseline score for a useless model with sklearn?

    Use a Dummy Classifier, it always predict the same class from sklearn.dummy import DummyClassifier DummyClassifier(strategy="most_frequent")

13. Why scaling data is useful for logistic regression?

    It uses gradient descent which converges much faster if ranges are the same. It probably won't affect predictive (generalization) performance

14. Which class of models doesn't need data scaling?

    Tree models (but they will not suffer from it)

15. In sklearn, which attributes end in "_", like scaler.mean_?

    Attributes that are learned from data

16. Formula used in StandardScaler

17. What would make_pipeline helper function name StandardScaler step of the pipeline?

    standardscaler

18. Function to get steps of a pipeline

    pipe.named_steps()

19. What does this return: pipe.named_steps()

    Dictionary where keys are names of steps and values are the objects

20. What does cross_validate function do to the trained models? Explain.

    It discards them. The goal of cross-validation is to get an idea on generalizability of a model (and variability of this value), not to train a model.

21. Another useful information you can get from cross-validation besides the average performance?

    Standard deviation of the performance, to estimate uncertainty of the performance score. If uncertainty is low, we can even use validation set directly in the future.

22. What conclusion can we make about uncertainty of the score in this case:

23. How to get all newly created features from the OneHotEncoder from one original feature

    encoder.get_feature_names_out(input_features=["education"])

24. What happens if an error occures during cross-validation (cross_validate)?

    It will print a warning and return NaN as a score. To force it to raise an exception you need to provide error_score="raise" argument

25. Sklearn functionality that helps applying different transformers to different columns

    ColumnTransformer It applies different transformers to different sets of columns Define two transformers Tell which columns should use which Here we apply OneHotEncoder to categorical variables and StandardScaler to numerical variables: Then just use the ColumnTranformer in a pipeline: You can use make_column_selector to select columns of a specific type:

26. When should you use OneHotEncoding (vs OrdinalEncoding)?

    When the data is not ordinal AND you're using non-tree based models

27. When is it okay to use mean imputation?

    When data is normally distributed. If data is skewed, median imputation is better.

28. Two steps of the model.score method

    It computes the predicted values using .predict method It uses scoring formula, comparing predicted values to provided target values

29. Another term for data standardization

    Z-score normalization

30. What does training error refer to?

    Error on the train set

31. Another name for training error.

    Empirical error

32. What does test error refer to?

    Error on test set

33. Another name for test error.

    Generalization error

34. What's a score?

    It's a metric for which higher values mean better results The opposite is error - a metric for which lower values mean better results

35. The parameter scoring in cross_validate always expect a function that is a score (as opposed to an error). What can you do to pass an error metric to the scoring parameter instead?

    Add neg_ to the string:

36. Does this model underfit? Why?

    No, the train error is very low, the model is flexible enough to capture all the variation in the training set.

37. Does this model overfit? Why?

    Yes, the test error is much higher than the train error. The model doesn't generalize well.

38. In decision tree model, which parameter is used to control the tradeoff between overfitting and underfitting?

    max_depth

39. How to time a cell in jupyter notebook?

    %%time

40. What does validation curve show?

    Train and test error (y axis) on different complexities of a model (x axis)

41. Visualize validation curve of a decision tree model with max_depth varying from 1 to 25.

42. On which values the model overfits?

    >10, training error goes down, which test error goes up

43. What's between overfitting and underfitting?

    Generalizing

44. What's plot of model performance over difference number of training examples called?

    Learning curve

45. What learning curve is used for?

    To check if adding more training examples is useful

46. If we see that the learning curve plateaued, how can we increase performance?

    Using a more complex model

47. How to build a validation curve with sklearn?

    from sklearn.model_selection import validation_curve train_scores, test_scores = validation_curve( ...)

48. How to build a learning curve with sklearn?

    from sklearn.model_selection import learning_curve

49. Statistical word to describe a problem an overfitting model causes, explain

    Variance The models trained of different samples from the population vary a lot (unstable models).

50. Statistical word to describe a problem an underfitting model causes, explain

    Bias The models trained of different samples from the population are consistently biased (systematic errors).

51. If model is not overfitting or underfitting, why would we still get bad performance?

    If we don't have access to important features

52. Example of a model with low number of parameters but high variance

    The model has only two parameters (a, b) but it can interpolate any number of points by oscillating with a high enough frequency.

53. When Randomized Search is generally recommended over Grid Search?

    When 3 or more hyperparameters are being optimized

54. Good number of iterations for Randomized Search

    500

55. What kind of plot is useful to explore importance of multiple hyperparameters

    Parallel coordinates plot

56. What we can do with parallel coordinates plot to explore hyperparameter importances?

    We can select ranges of several hyperparameters and see if there are ranges on some hyperparameters that prevent the highest scores which hyperparameters don't affect the score

57. After you defined a GridSearch, what do you do next?

    You need to fit it: train the model with each combination of parameters