MLAT Bayes Techniques & Latent Variable models

19 flashcards · Computer Science · by Mathias777-crypto

1. How can the likelihood be declared, too? How can that be further specified in supervised machine learning? Remembering the definition of the likelihood, how does the specified form look like then?

   p(z;ω)p(z;\omega)p(z;ω) =p(x)p(y∣x;ω)= p(x) p(y|x;\omega)=p(x)p(y∣x;ω) =∏i=1lp(xi)∏i=1lp(yi∣xi;ω)= \prod_{i=1}^l p(x_i) \prod_{i=1}^l p(y_i|x_i; \omega)=∏i=1l​p(xi​)∏i=1l​p(yi​∣xi​;ω)

2. What the supervised machine learning setting expressed mathematically? What does it mean and what is often left out for simplicity? What's this approach called?

   p({z}∣ω,M)=∏i=1lp(zi∣w,M)p \left( \{ z \} \vert \omega, \Mu \right) = \prod_{i=1}^l p \left( z_i \vert w, \Mu \right)p({z}∣ω,M)=∏i=1l​p(zi​∣w,M) In machine learning, we try to find model parameters and an architecture that maximizes the likelihood. M\MuM The frequentist approach

3. What's another factor that can lead to overfitting?

   A tiny dataset

4. What's one way to prevent overfitting when it comes to maximizing the likelihood? In what form is it applied?

   Weight decay There's a weight decay term added that act against maximizing the likelihood.

5. What's a basic principle of the Bayesian approach in constrast to the Frequentist approach?

   Possible parameters in the likelihood estimation have different probabilities; in the Frequentist approach, there is no a priori assessment of parameter values.

6. Which part of training do Bayesian Techniques affect and is the probability assessment of parameters only done of parameter values?

   Initialization No, it also covers the model choice.

7. What's the prior distribution? How is it denoted formally?

   It denotes the likeliness of models which is gotten from prior knowledge about a problem. p(ω)p(\omega)p(ω)

8. What's a crucial property of the Bayesian approach?

   Realistically quantify uncertainty

9. Which type of estimation do you get by the Frequentist and which by the Bayesian approach?

   Frequentist: point estimate Bayesian: posterior distribution

10. What does the posterior represent about the value of each parameter (setting)?

    Uncertainty

11. What can happen if one intitialzes a model's parameters wrongly?

    Everything goes through the roof.

12. How do prior and likelihood interplay? Concretely?

    The prior belief (about the model parameters before seeing any data) is updated with the likelihood. Backward pass

13. What's the evil component of Bayes' Theorem? And why?

    Evidence Because one needs the marginal of the data which is obtained by integrating over all possible parameter combinations. This is tedious because the distribution of parameters is huge (ChatGPT: ~1.3 trillion parameters).

14. The higher the evidence, the better the model. This quantity provides evidence on how good the model is.

15. What's a famous joke about marginalization?

    The more people get into Bayesian Techniques, the longer the 'a' gets in 'marginalization'.

16. What is meant by Conjugate distribution?

    A pair of distributions where the prior and the posterior come from the same distribution. The product of prior and likelihood is of the same type as the posterior. It's the case for many distributions.

17. The prior is a conjugate for the likelihood because it produces the same posterior.

18. The Gamma distribution is a conjugate prior of the Poisson distribution.

19. How does the example of a conjugate pair with a Gamma and a Poisson distribution go?

    A Gamma distribution is assumed as a prior and a Poisson distribution as a basis for the likelihood. Multiplying both distributions (and leaving out the normalizing evidence) another Gamma distribution is obtained.