Coursera Stanford ML: Decision Trees

20 flashcards · Computer Science · by charlie42

  1. How does the decision tree choose which feature to split on?

    Feature that maximises purity of the resulting nodes

  2. Two main measures of impurity for classification?

    Entropy Gini

  3. Visually imagine the plot of the Entropy function

  4. What is information gain? (in 2.5 words)

    Reduction in entropy

  5. Visually imagine information gain for a decision tree split

    (Entropy before split) - (weighted average entropy after split)

  6. What's is the value of entropy of a sample with half positive and half negative values?

    p1 = 1/2 H(p1) = H(1/2) = 1

  7. What's the value of entropy of a sample where all examples are of the same class?

    H = 0

  8. What the average entropy between the nodes after splitting is weighted for?

    Proportion of examples that went into each node

  9. What are regression trees?

    Trees that predict a continuous value

  10. How is impurity measured in regression trees?

    Variance of the final samples instead of entropy

  11. Visually imagine formulas for population and sample variance

  12. Visually imagine the explanation why changing the dataset a tiny bit might result in a completely different tree

    Changing one value in the dataset can change which feature to split by first - root node. If root node is different, all consecutive splits are different.

  13. What simpler algorithm random forest is based on?

    Bagged decision trees

  14. Visually imagine bagged decision tree algorithm

  15. Visually imagine how subsets for each decision tree in bagged decision tree are generated?

    Random sampling with replacement N times: take a random example record it put it back

  16. How the final prediction value is aggregated between the trees in bagged decision tree algorithm?

    Value which majority of trees predicted

  17. What is the improvement of random forest over bagged decision tree?

    It forces trees to be more different from each other. At each split: take random k<n features -- can choose only among those

  18. Rule of thumb number of trees for bagged decision tree

    >100 is marginal improvement

  19. Very generally, how do boosted trees work?

    Sequentially. Examples misclassified by the first tree are given higher weights, and the consequent classifier focuses on them more

  20. When neural networks usually outperform ensemble decision tree models?

    On unstructured data (image, voice, ...)