Machine Learning with R (Rhys)

Hefin I. Rhys "Rhys: Machine Learning with R, the tidyverse and mlr"

21 flashcards · Computer Science · by lakesare

1. What does PCA stand for?

   Principal Component Analysis

2. What's the point of performing a Principal Component Analysis (PCA)?

   To present our high-dimensional data in smaller n of dimensions.

3. In simple words, what's the curse of dimensionality?

   It's the fact that it's hard to identify patterns in high-dimensional data.

4. Visualise why: as the n of dimensions increases, data becomes more sparse.

   Data becomes more sparse as the number of dimensions increases. Two classes are shown in one-, two-, and three-dimensional feature spaces. The dotted lines in the three-dimensional representation are to clarify the position of the points along the z-axis. Note the increasing empty space with increased dimensions.

5. Give an example of collinear variables in a data set. Variables in a dataset often have varying degrees of correlation with each other.

   height leg length

6. What does "Naive" in Naive Bayes refer to?

   It assumes features are perfectly independent.

7. Name 2 widespread reasons we might want to reduce the dimensionality of the dataset?

   to get rid of collinear variables to make distances meaningful again (counteract curse of dimensionality)

8. Name 2 techniques to reduce the dimensionality of the dataset.

   PCA (Principal Component Analysis) SOM (Self-Organizing Maps) LLE (Locally Linear Embdessing) t-SNE UMAP

9. What's a good physical analogy for Self-Organizing Maps (SOMs)?

   Putting each bead in a bowl. Placing beads into bowls based on their characteristics. Similar beads are placed in the same or nearby bowls, while dissimilar beads are placed in bowls far away from each other. One bowl didn’t have any beads placed in it, but that’s okay. This is what self-organizing maps try to do. The “map” of a self-organizing map is equivalent to the grid of bowls, where each bowl is called a node.

10. [Self-Organizing Maps] What n of weights does each node on a map have?

    n of weights per node == n of dimensions in our data.

11. [Self-Organizing Maps] Name 2 examples of neighborhood functions for SOMs. Our choice of neighborhood function is a hyperparameter, as it will affect the way our map updates its nodes but cannot be estimated from the data itself.

    Bubble function - when we update the units in 3 direct connections to the BMU. With the bubble function, we simply define a radius (or bubble) around the BMU, and all nodes inside that radius have their weights updated to the same degree. Any nodes outside the radius are not updated at all. For the bubble function, a radius of 3 would include any node within three direct connections of the BMU.Gaussian function - update the neighbours proportionally to their distance from the BMU. You can pictu…

12. [Self-Organizing Maps] To what n of dimensions does SOM reduce the data?

    To 2 dimensions.

13. [How good is our clustering] Name 2 methods to assess our clustering. K-means clustering For supervised learning problems, telling how good our model is is easy. There are already labels for every example, so the practitioner can test the model’s performance on a reserved evaluation set.We don’t have that luxury when we’re dealing with unlabeled data in unsupervised learning contexts. There’s nothing to even test since there’s no ground truth — the idea of testing in this arena is a flawed prem…

    DB Index (Davies-Bouldin Index) Silhouette Coefficient Dunn Index Link: opendatascience.com/assessment-metrics-clustering-algorithms.

14. [How good is our clustering] Name 2 criteria by which we define whether our clustering is good. K-means clustering

    each cluster is compact distances between clusters are large

15. [How good is our clustering] What do we divide by what in the Davies-Bouldin Index? K-means clustering

    Intracluster variance ______________________________ Distance between centroids

16. [Unsupervised Learning: Kohonen networks] Name 3 steps in the SOM (Self-Origanizing Map) algorithm. Kohonen network

    For x1: Competition - which neuron has the value closest to x1 Winning neuron = Best Matching Unit (BMU)Cooperation - the winner chooses neighbours Adaptation - the winner and its neighbours update Nodes closer to the winning point learn more than those farther away.

17. [Unsupervised Learning: Kohonen networks] What does SOM stand for?

    Self-ogranizing maps.

18. What are these about? Holdout ____________ K-fold ____________ Leave-one-out ____________

    These are common cross-validation approaches.

19. Name 2 ways to adjust k in k-nearest neighbours algorithm? k is a hyperparameter determining how many neighbours can vote for the label of the vector.

    manually try a few different values hyperparameter tuning You can always just pick a sensible default value too, of course.

20. What's cross-validation?

    It's when we divide the data into: training set test set to avoid overfitting to noise in the training data. There are three common cross-validation approaches: Holdout cross-validation K-fold cross-validation Leave-one-out cross-validation

21. What does CV stand for?

    Cross-validation.