Ch.4: Distributions

Chapter 4 of the book "Probability for the Enthusiastic Beginner" by David Morin (2016) Discrete distributions Continuous distributions Uniform distribution Bernoulli distribution Binomial distribution Exponential distribution Poisson distribution Gaussian distribution Problems

141 flashcards · Mathematics · by lakesare

Part of Textbooks › Probability for the Enthusiastic Beginner (David Morin) · 7 courses

[Distributions] What is a random variable?
A variable that takes numerical values, each with a certain probability. Example: the number of Heads in five coin flips takes the values 0,1,…,50, 1, \dots, 50,1,…,5 .
[Distributions] What is the probability distribution of a random variable?
The collection of the probabilities of all the values the random variable can take. It says how the total probability, which is always 1, is distributed among the possible outcomes.Example: for one fair die roll, each of the values 1,…,61, \dots, 61,…,6 has probability 1/61/61/6 .
[Distributions] Which six common distributions does Morin treat, and which of them are discrete?
Uniform, Bernoulli, binomial, exponential, Poisson, Gaussian (normal). Discrete: Bernoulli, binomial, Poisson. Uniform, exponential and Gaussian are continuous. Only about a hundred distributions come up often enough to have names.
[Discrete distributions] nnn is the number of Heads in five fair coin flips. What are P(0),P(1),…,P(5)P(0), P(1), \dots, P(5)P(0),P(1),…,P(5) ?
132,532,1032,1032,532,132\boldsymbol{\frac{1}{32}, \frac{5}{32}, \frac{10}{32}, \frac{10}{32}, \frac{5}{32}, \frac{1}{32}}321,325,3210,3210,325,321 P(n)=(5n)/25P(n) = \binom{5}{n} / 2^5P(n)=(n5)/25 , and the six values add up to 1. The probability distribution of the number of Heads in five flips.
[Discrete distributions] When plotting a probability distribution, what goes on which axis?
The random variable on the horizontal axis, the probability on the vertical axis.
[Discrete distributions] Why is the distribution of the number of Heads in five coin flips called discrete?
P(n)P(n)P(n) is nonzero only for the discrete values n=0,1,2,3,4,5n = 0, 1, 2, 3, 4, 5n=0,1,2,3,4,5 . Example: the probability of getting 4.27 Heads is 0, since the number of Heads is an integer.
[Discrete distributions] A die has a small piece of lead embedded just below the center of its "1" face. How does its distribution differ from that of a fair die (all P(n)=1/6P(n) = 1/6P(n)=1/6 )?
P(6)P(6)P(6) is larger than 1/61/61/6 ; P(1)P(1)P(1) through P(5)P(5)P(5) are smaller, and P(2)P(2)P(2) through P(5)P(5)P(5) are equal by symmetry. The "1" face is heavier, so it tends to point down, and the "6" face is opposite to it. The sum is still 1. The exact values depend on the lead's mass and on the die and the table (friction, bounciness).
[Discrete distributions] Is the outcome (Heads or Tails) of a single coin flip a random variable?
Technically no: a random variable takes numerical values by definition. The number of Heads is a random variable: Heads becomes 1, Tails becomes 0. Morin still calls whatever is on the horizontal axis of a distribution plot a random variable.
[Continuous distributions] What is wrong with asking for the probability that the temperature at noon tomorrow will be 70°?
The answer is always 0, so the question gives no information. The temperature takes a continuous set of values, so it is exactly 70° with probability 0: 70.01° or 70.00000001° doesn't count.
[Continuous distributions] A digital thermometer shows the temperature to the nearest tenth of a degree. Is the probability that it reads 70.0 also 0?
No, it is generally nonzero. The reading is a discrete random variable; the actual temperature is a continuous one.
[Continuous distributions] For a continuous random variable, which kind of question has a nontrivial (nonzero) probability as its answer?
The probability that the variable lies in a given range. Example: the probability that the temperature at noon is between 69° and 71° might be 20%.
[Continuous distributions] The temperature at noon lies between 69° and 71° with probability 20%, and inside that range it is about equally likely to be anywhere. What is the probability that it lies between 69.5° and 70.5°?
About 10% The range is half as large.
[Continuous distributions] For a continuous random variable, which three things determine the probability that it lies in a given range?
Location of the range The same size of range at a different place gives a different probability. Example: on a late-summer day in Boston, the temperature is much more likely to be between 69° and 71° than between 11° and 13°. Size of the range The larger the range, the larger the probability. Example: the temperature is between −100° and 200° with probability 1. Specifics of the situation The data of the setup, which determine the probability density. Example: temperatures need meteorological d…
[Continuous distributions] What notation does Morin use for probability densities, and what for actual probabilities?
Lowercase letters for densities ( ρ(T)\rho(T)ρ(T) , f(x)f(x)f(x) ), uppercase for probabilities ( P(n)P(n)P(n) ).
[Continuous distributions] Definition of the probability density ρ(T)\rho(T)ρ(T) of a continuous random variable TTT (e.g. a temperature)?
ρ(T)\rho(T)ρ(T) is the function with P(value lies between T and T+ΔT)=ρ(T)⋅ΔTP(\text{value lies between } T \text{ and } T + \Delta T) = \rho(T) \cdot \Delta TP(value lies between T and T+ΔT)=ρ(T)⋅ΔT for a small interval ΔT\Delta TΔT . ρ(T)\rho(T)ρ(T) is not a probability; it must be multiplied by a range of TTT to give one. Example: a hypothetical temperature density with ρ(70∘)≈0.07\rho(70^\circ) \approx 0.07ρ(70∘)≈0.07 gives P(70∘ to 71∘)≈(0.07)(1)=7%P(70^\circ \text{ to } 71^\circ) \approx…
[Continuous distributions] P(temperature lies between T and T+ΔT)=ρ(T)⋅ΔTP(\text{temperature lies between } T \text{ and } T + \Delta T) = \rho(T) \cdot \Delta TP(temperature lies between T and T+ΔT)=ρ(T)⋅ΔT for small ΔT\Delta TΔT . ρ(70∘)≈0.07\rho(70^\circ) \approx 0.07ρ(70∘)≈0.07 and ρ(60∘)≈0.02\rho(60^\circ) \approx 0.02ρ(60∘)≈0.02 (per degree). What are the probabilities that the temperature lies between 70° and 70.5°, between 70° and 70.1°, and between 60° and 61°?
About 3.5%, 0.7% and 2%. (0.07)(0.5)=0.035(0.07)(0.5) = 0.035(0.07)(0.5)=0.035 , (0.07)(0.1)=0.007\ (0.07)(0.1) = 0.007 (0.07)(0.1)=0.007 , (0.02)(1)=0.02\ (0.02)(1) = 0.02 (0.02)(1)=0.02 .
[Continuous distributions] Definition: P(value lies between T and T+ΔT)=ρ(T)⋅ΔTP(\text{value lies between } T \text{ and } T + \Delta T) = \rho(T) \cdot \Delta TP(value lies between T and T+ΔT)=ρ(T)⋅ΔT for a small interval ΔT\Delta TΔT .Why does the definition require ΔT\Delta TΔT to be small?
So that ρ(T)\rho(T)ρ(T) is roughly constant over the interval. If ρ\rhoρ varied a lot between TTT and T+ΔTT + \Delta TT+ΔT , it would be unclear which value of ρ\rhoρ to multiply by ΔT\Delta TΔT .
[Continuous distributions] Definition: P(value lies between T and T+ΔT)=ρ(T)⋅ΔTP(\text{value lies between } T \text{ and } T + \Delta T) = \rho(T) \cdot \Delta TP(value lies between T and T+ΔT)=ρ(T)⋅ΔT for a small interval ΔT\Delta TΔT .Could one use ρ\rhoρ at the midpoint or at the right end of the interval instead?
Yes: for small ΔT\Delta TΔT all give essentially the same result. Midpoint version: P(between T−ΔT/2 and T+ΔT/2)=ρ(T)⋅ΔTP(\text{between } T - \Delta T/2 \text{ and } T + \Delta T/2) = \rho(T) \cdot \Delta TP(between T−ΔT/2 and T+ΔT/2)=ρ(T)⋅ΔT . Right end: ρ(T+ΔT)⋅ΔT\rho(T + \Delta T) \cdot \Delta Tρ(T+ΔT)⋅ΔT .
[Continuous distributions] In the definition P(between T and T+ΔT)≈ρ(T)⋅ΔTP(\text{between } T \text{ and } T + \Delta T) \approx \rho(T) \cdot \Delta TP(between T and T+ΔT)≈ρ(T)⋅ΔT , how small must ΔT\Delta TΔT be?
The change of ρ(T)\rho(T)ρ(T) over the span ΔT\Delta TΔT must be small compared with the values of ρ(T)\rho(T)ρ(T) in that span.
[Continuous distributions] Probability density ρ(T)\rho(T)ρ(T) : three inputs give the probability that TTT lies in a range. Which ones, and which two are multiplied?
The value TTT , the range ΔT\Delta TΔT , and the density function ρ\rhoρ . The product is ρ(T)⋅ΔT\rho(T) \cdot \Delta Tρ(T)⋅ΔT . Knowing only ρ(T)\rho(T)ρ(T) or only ΔT\Delta TΔT doesn't give a probability.
[Continuous distributions] Why do we plot a probability density, not probabilities, for a continuous random variable?
The probability of each exact value is 0, so a plot of probabilities would be zero everywhere. There are infinitely many possible values.
[Continuous distributions] Why does a probability density make no sense for a discrete random variable?
The density would be a collection of infinite spikes. Example: a die gives a number between 4.9999999 and 5.0000001 with probability 1/61/61/6 , so the density at 5 would be (1/6)/(0.0000002)≈833,000(1/6)/(0.0000002) \approx 833{,}000(1/6)/(0.0000002)≈833,000 , and arbitrarily large for smaller intervals.
[Continuous distributions] The term "probability distribution" applies to both discrete and continuous random variables; the term "probability density" applies only to continuous ones.
[Continuous distributions] Does the probability density ρ(T)\rho(T)ρ(T) of a temperature TTT depend on the unit of temperature?
Yes: ρ(T)\rho(T)ρ(T) is a function of TTT , and ρ(T)⋅ΔT\rho(T) \cdot \Delta Tρ(T)⋅ΔT must be a probability whatever the unit of ΔT\Delta TΔT . Example: 1 °C is 1.8 °F, so a density of 0.07 per °F at 70 °F is a density of 0.07⋅1.8=0.1260.07 \cdot 1.8 = 0.1260.07⋅1.8=0.126 per °C at 21.1 °C.
[Continuous distributions] Does a probability density ρ(T)\rho(T)ρ(T) depend on the interval ΔT\Delta TΔT chosen?
No. It depends only on TTT and on the specifics of the situation (e.g. meteorological data). This holds for every kind of density, e.g. the mass density of a material.
[Continuous distributions] How does a probability density ρ(T)\rho(T)ρ(T) correspond to the mass density ρ(r)\rho(\mathbf{r})ρ(r) of an object?
Mass in a small volume ΔV\Delta VΔV around r\mathbf{r}r is ρ(r)⋅ΔV\rho(\mathbf{r}) \cdot \Delta Vρ(r)⋅ΔV , just as the probability that TTT lies in ΔT\Delta TΔT is ρ(T)⋅ΔT\rho(T) \cdot \Delta Tρ(T)⋅ΔT . mass in ΔV\Delta VΔV ⟺\Longleftrightarrow⟺ probability in ΔT\Delta TΔT ; ρ(r)⟺ρ(T)\ \rho(\mathbf{r}) \Longleftrightarrow \rho(T) ρ(r)⟺ρ(T) ; ΔV⟺ΔT\ \Delta V \Longleftrightarrow \Delta T ΔV⟺ΔT Example: gold with impurities has a mass density that varies with the position r\mathbf{r}r , as ρ(T)\rh…
[Continuous distributions] What is the graphical meaning of ρ(T)⋅ΔT\rho(T) \cdot \Delta Tρ(T)⋅ΔT , for a probability density ρ\rhoρ ?
The area of a rectangle with base ΔT\Delta TΔT and height ρ(T)\rho(T)ρ(T) . ΔT=2∘\Delta T = 2^\circΔT=2∘ : the rectangle's area ρ(70∘)⋅(2∘)\rho(70^\circ) \cdot (2^\circ)ρ(70∘)⋅(2∘) approximates the probability that the temperature lies between 70° and 72°.
[Continuous distributions] The rectangle has height ρ(70∘)\rho(70^\circ)ρ(70∘) and base 70° to 72°. Is its area larger or smaller than the true probability that the temperature lies between 70° and 72°, and by how much?
Larger, by the small roughly triangular region between the curve and the rectangle's upper right corner. ρ(T)\rho(T)ρ(T) decreases between 70° and 72°, so it is below ρ(70∘)\rho(70^\circ)ρ(70∘) there.
[Continuous distributions] How do you get a better approximation, and then the exact value, of the probability that a temperature lies between 70° and 72° from its density ρ(T)\rho(T)ρ(T) ?
Split the interval into ever smaller intervals and add up the rectangle areas; in the limit, the sum is the area under the ρ(T)\rho(T)ρ(T) curve between 70° and 72°. Two rectangles: ρ(70∘)⋅(1∘)+ρ(71∘)⋅(1∘)\rho(70^\circ) \cdot (1^\circ) + \rho(71^\circ) \cdot (1^\circ)ρ(70∘)⋅(1∘)+ρ(71∘)⋅(1∘) . The exact probability: the area under the curve.
[Continuous distributions] What is the exact (improved) definition of the probability density ρ(T)\rho(T)ρ(T) ?
The area under the ρ(T)\rho(T)ρ(T) curve between TTT and T+ΔTT + \Delta TT+ΔT is the probability that the value lies between TTT and T+ΔTT + \Delta TT+ΔT . No need for ΔT\Delta TΔT to be small. Equivalently, P=ρ(T)⋅ΔTP = \rho(T) \cdot \Delta TP=ρ(T)⋅ΔT in the limit ΔT→0\Delta T \to 0ΔT→0 .
[Continuous distributions] What is the total area under any probability density curve, and why?
1\boldsymbol{1}1 The area is the probability that the value lies between −∞-\infty−∞ and +∞+\infty+∞ , which includes every possible result. In realistic cases the density is essentially 0 outside a finite region.
[Continuous distributions] Given a probability density f(x)f(x)f(x) of a random variable XXX , what is the cumulative distribution function F(x)F(x)F(x) ?
F(x)=P(X≤x)\boldsymbol{F(x) = P(X \le x)}F(x)=P(X≤x) For a continuous distribution, F(x)F(x)F(x) is the area under the fff curve from −∞-\infty−∞ up to xxx .Example: for XXX uniform between 0 and 10, F(x)=x/10F(x) = x/10F(x)=x/10 for 0≤x≤100 \le x \le 100≤x≤10 .
[Continuous distributions] FFF is the cumulative distribution function of a random variable XXX . What is P(a<X≤b)P(a < X \le b)P(a<X≤b) ?
F(b)−F(a)\boldsymbol{F(b) - F(a)}F(b)−F(a) Example: for XXX uniform between 0 and 10, P(6<X≤8)=0.8−0.6=0.2P(6 < X \le 8) = 0.8 - 0.6 = 0.2P(6<X≤8)=0.8−0.6=0.2 .
[Continuous distributions] How is the cumulative distribution function F(x)=P(X≤x)F(x) = P(X \le x)F(x)=P(X≤x) computed for a discrete random variable XXX ?
As a discrete sum of the probabilities of the values ≤x\le x≤x , instead of an area. Example: for one fair die roll, F(2)=P(1)+P(2)=1/3F(2) = P(1) + P(2) = 1/3F(2)=P(1)+P(2)=1/3 .
[Uniform distribution] A uniform distribution is nonzero exactly between x1x_1x1 and x2x_2x2 . What is its density ρ(x)\rho(x)ρ(x) there?
ρ(x)=1x2−x1\boldsymbol{\rho(x) = \frac{1}{x_2 - x_1}}ρ(x)=x2−x11 So that the total area under ρ\rhoρ is 1.
[Uniform distribution] Give a physical setup with a (roughly) uniform distribution.
A rubber ball thrown hard in an empty rectangular room: its distance xxx from one wall when it comes to rest. xxx is about equally likely to take any value between 0 and LLL , the length of the room in that direction.
[Uniform distribution] A ball's resting distance xxx from a wall is uniform between 0 and L=10L = 10L=10 feet. What is the density ρ(x)\rho(x)ρ(x) , and the probability that 6<x<86 < x < 86<x<8 ?
ρ(x)=\rho(x) = ρ(x)= 1/10 per foot; P(6<x<8)=P(6 < x < 8) = P(6<x<8)= 1/5 (1/10 per foot)⋅(2 feet)=2/10(1/10 \text{ per foot}) \cdot (2 \text{ feet}) = 2/10(1/10 per foot)⋅(2 feet)=2/10 : the 2-foot interval is 1/51/51/5 of the total distance.
[Uniform distribution] Why is a uniform density easy to work with?
The area under a part of the curve is a rectangle: interval length times density. For a nonuniform density, the area under a part of the curve can be hard to find.
[Uniform distribution] In a uniform distribution, what happens to the value of ρ(x)\rho(x)ρ(x) when the region where ρ\rhoρ is nonzero gets longer, and why?
ρ(x)\rho(x)ρ(x) gets smaller, because the total area (base times height) must stay 1. Example: uniform on a 2-unit interval: ρ=1/2\rho = 1/2ρ=1/2 ; on a 10-unit interval: ρ=1/10\rho = 1/10ρ=1/10 .
[Bernoulli distribution] What is the Bernoulli distribution?
Only two outcomes, 1 and 0, with probabilities ppp and 1−p1 - p1−p . Example: a fair coin toss with Heads = 1, Tails = 0 is Bernoulli with p=1/2p = 1/2p=1/2 .
[Bernoulli distribution] In a Bernoulli distribution, the outcome 1 is called a success and the outcome 0 a failure.
[Bernoulli distribution] Which distribution is the simplest of all, apart from the trivial one with a single outcome of probability 1?
The Bernoulli distribution (outcomes 1 and 0 with probabilities ppp and 1−p1 - p1−p ).
[Binomial distribution] What is the binomial distribution?
The distribution of the total number of successes in nnn independent and identically distributed Bernoulli processes. (A Bernoulli process gives success with probability ppp , failure with 1−p1 - p1−p .)Example: the number of Heads in five coin tosses, with n=5n = 5n=5 and p=1/2p = 1/2p=1/2 .
[Binomial distribution] Binomial distribution. nnn independent Bernoulli trials each succeed with probability ppp . Probability of exactly kkk successes?
P(k)=(nk)pk(1−p)n−k\boldsymbol{P(k) = \binom{n}{k} p^k (1 - p)^{n - k}}P(k)=(kn)pk(1−p)n−k , k=0,1,…,n\ k = 0, 1, \dots, n k=0,1,…,n Example: n=3n = 3n=3 die rolls, success = rolling a 6: P(1)=3⋅16⋅(56)2=75216P(1) = 3 \cdot \frac{1}{6} \cdot \left(\frac{5}{6}\right)^2 = \frac{75}{216}P(1)=3⋅61⋅(65)2=21675 .
[Binomial distribution] How does the derivation of the binomial distribution P(k)=(nk)pk(1−p)n−kP(k) = \binom{n}{k} p^k (1 - p)^{n - k}P(k)=(kn)pk(1−p)n−k go ( nnn independent trials, success probability ppp )?
A specific set of kkk trials all succeeds and the other n−kn - kn−k all fail with probability pk(1−p)n−kp^k (1 - p)^{n - k}pk(1−p)n−k ; there are (nk)\binom{n}{k}(kn) such sets. Exactly kkk successes means the remaining n−kn - kn−k trials must not succeed, each with probability 1−p1 - p1−p .
[Binomial distribution] What are the parameters of the binomial distribution, and how is it written with them made explicit?
nnn , the number of Bernoulli trials, and ppp , the probability of success in each: Bn,p(k)=(nk)pk(1−p)n−kB_{n,p}(k) = \binom{n}{k} p^k (1 - p)^{n - k}Bn,p(k)=(kn)pk(1−p)n−k
[Binomial distribution] What is the probability of exactly kkk Heads in nnn fair coin tosses?
P(k)=12n(nk)\boldsymbol{P(k) = \frac{1}{2^n} \binom{n}{k}}P(k)=2n1(kn) The binomial distribution with p=1/2p = 1/2p=1/2 .Example: 2 Heads in 4 tosses: 6/16=3/86/16 = 3/86/16=3/8 .
[Binomial distribution] Binomial distribution with n=30n = 30n=30 trials: where is the peak of P(k)P(k)P(k) for p=1/2p = 1/2p=1/2 (30 coin tosses), and where for p=1/6p = 1/6p=1/6 (30 die rolls, one number counts as success)?
Near k=15k = 15k=15 and near k=5k = 5k=5 , i.e. near pnpnpn . P(15)≈0.14P(15) \approx 0.14P(15)≈0.14 for p=1/2p = 1/2p=1/2 , P(5)≈0.19P(5) \approx 0.19P(5)≈0.19 for p=1/6p = 1/6p=1/6 .
[Binomial distribution] Binomial distribution with nnn trials and success probability ppp : for which ppp is P(0)=P(1)P(0) = P(1)P(0)=P(1) ?
p=1n+1\boldsymbol{p = \frac{1}{n + 1}}p=n+11 (1−p)n=np(1−p)n−1(1 - p)^n = n p (1 - p)^{n - 1}(1−p)n=np(1−p)n−1 gives 1−p=np1 - p = np1−p=np . Example: n=10n = 10n=10 , p=1/11p = 1/11p=1/11 : P(0)=P(1)=(10/11)10≈0.39P(0) = P(1) = (10/11)^{10} \approx 0.39P(0)=P(1)=(10/11)10≈0.39 .
[Binomial distribution] Binomial: P(k)=(nk)pk(1−p)n−kP(k) = \binom{n}{k} p^k (1 - p)^{n - k}P(k)=(kn)pk(1−p)n−k . P(0)=P(1)P(0) = P(1)P(0)=P(1) exactly when p=1/(n+1)p = 1/(n + 1)p=1/(n+1) .Which competing effects cancel at p=1/(n+1)p = 1/(n + 1)p=1/(n+1) ?
P(1)P(1)P(1) has an extra factor nnn (the nnn ways one success can happen), but also a factor ppp (one success does happen). The factor nnn makes P(1)P(1)P(1) larger than P(0)P(0)P(0) , the factor ppp makes it smaller. ( P(1)P(1)P(1) also has one fewer factor 1−p1 - p1−p , a minor effect when ppp is small.)
[Binomial distribution] Binomial distribution with nnn trials and success probability ppp : when is the maximum of P(k)P(k)P(k) at k=0k = 0k=0 , and when at a nonzero kkk ?
p<1n+1p < \frac{1}{n + 1}p<n+11 : maximum at k=0k = 0k=0 ; p>1n+1\ p > \frac{1}{n + 1} p>n+11 : maximum at a nonzero kkk (the distribution has a bump). P(1)/P(0)=np/(1−p)P(1)/P(0) = np/(1 - p)P(1)/P(0)=np/(1−p) , which exceeds 1 exactly when p>1/(n+1)p > 1/(n + 1)p>1/(n+1) .Example: n=10n = 10n=10 , p=1/6>1/11p = 1/6 > 1/11p=1/6>1/11 : P(0)=(5/6)10≈0.16P(0) = (5/6)^{10} \approx 0.16P(0)=(5/6)10≈0.16 , P(1)≈0.32P(1) \approx 0.32P(1)≈0.32 .
[Binomial distribution] Why do the binomial probabilities P(k)=(nk)pk(1−p)n−kP(k) = \binom{n}{k} p^k (1 - p)^{n - k}P(k)=(kn)pk(1−p)n−k , k=0,…,nk = 0, \dots, nk=0,…,n , add up to 1?
Binomial expansion: ∑k=0n(nk)pk(1−p)n−k=(p+(1−p))n=1n=1\displaystyle\sum_{k=0}^{n} \binom{n}{k} p^k (1 - p)^{n - k} = \big(p + (1 - p)\big)^n = \boldsymbol{1^n = 1}k=0∑n(kn)pk(1−p)n−k=(p+(1−p))n=1n=1 Example: n=2n = 2n=2 : (1−p)2+2p(1−p)+p2=1(1 - p)^2 + 2p(1 - p) + p^2 = 1(1−p)2+2p(1−p)+p2=1 .
[Binomial distribution] What is the expectation value of the binomial distribution ( nnn trials, success probability ppp ), and why does it make sense?
pn\boldsymbol{pn}pn On average a fraction ppp of the nnn trials succeeds, by the definition of ppp .Example: 30 die rolls, success = a 6: on average 5 successes.
[Binomial distribution] How would you check experimentally that the binomial distribution correctly gives the number of Heads in 30 coin flips?
Do many sets of 30 flips, record the number of Heads in each set, and compare the fraction of sets with each number to P(k)P(k)P(k) . The longer the record, the better the agreement in general. A distribution says little about one particular set of 30 flips; it gives the expected spread of outcomes over many sets.
[Exponential distribution] What does the exponential distribution describe?
The waiting time until the next event, for an event that repeats completely randomly in time. "Completely randomly": in every small time interval of a given length the event has the same probability, independent of what has already happened (the process has no memory).Example: the time until the next click of a Geiger counter.
[Exponential distribution] A 10-sided die is rolled once every second, and a "1" counts as a success. What is the average waiting time between successes?
10 seconds. In an hour (3600 rolls) about 3600/10=3603600/10 = 3603600/10=360 rolls show a 1, so the average gap is 3600/360=103600/360 = 103600/360=10 seconds.
[Exponential distribution] Trials happen once per second, each a success with probability ppp . What is the average waiting time between successes, and why?
1/p\boldsymbol{1/p}1/p In nnn trials there are about pnpnpn successes, so the average gap is npn=1p\dfrac{n}{pn} = \dfrac{1}{p}pnn=p1 . This needs no knowledge of the distribution of the waiting times.Example: p=1/10p = 1/10p=1/10 gives 10 seconds.
[Exponential distribution] Trials happen once per second, each a success with probability ppp . What is the probability P(k)P(k)P(k) that the next success comes on exactly the kkk -th trial?
P(k)=(1−p)k−1p\boldsymbol{P(k) = (1 - p)^{k-1} p}P(k)=(1−p)k−1p (the geometric distribution) The first k−1k - 1k−1 trials must fail (probability 1−p1 - p1−p each) and the kkk -th must succeed.Example: p=1/2p = 1/2p=1/2 (a coin) gives P(1),P(2),P(3),P(4)=1/2,1/4,1/8,1/16P(1), P(2), P(3), P(4) = 1/2, 1/4, 1/8, 1/16P(1),P(2),P(3),P(4)=1/2,1/4,1/8,1/16 .
[Exponential distribution] The distribution P(k)=(1−p)k−1pP(k) = (1 - p)^{k-1} pP(k)=(1−p)k−1p of the number of trials until the next success is called the geometric distribution.
Its probabilities form a geometric progression with ratio 1−p1 - p1−p . It is the discrete version of the exponential distribution.
[Exponential distribution] In the geometric distribution P(k)=(1−p)k−1pP(k) = (1 - p)^{k-1} pP(k)=(1−p)k−1p with p=1/10p = 1/10p=1/10 , the average waiting time is 10 trials. What is the probability that the waiting time is exactly 10?
Only about 4%: P(10)=(0.9)9(0.1)≈0.04P(10) = (0.9)^9 (0.1) \approx 0.04P(10)=(0.9)9(0.1)≈0.04 . P(k)P(k)P(k) is largest at k=1k = 1k=1 and decreases by the factor 0.90.90.9 with each step. P(k)P(k)P(k) for p=1/10p = 1/10p=1/10 .
[Exponential distribution] How does the plot of the geometric distribution P(k)=(1−p)k−1pP(k) = (1 - p)^{k-1} pP(k)=(1−p)k−1p depend on ppp ?
Large ppp : it starts high and decreases quickly. Small ppp : it starts low and decreases slowly. It starts at P(1)=pP(1) = pP(1)=p , and each next value is (1−p)(1 - p)(1−p) times the previous one.
[Exponential distribution] How do you check that the geometric distribution P(k)=(1−p)k−1pP(k) = (1 - p)^{k-1} pP(k)=(1−p)k−1p has total probability 1?
Sum the geometric series: p+p(1−p)+p(1−p)2+⋯=p1−(1−p)=1\displaystyle p + p(1 - p) + p(1 - p)^2 + \dots = \frac{p}{1 - (1 - p)} = 1p+p(1−p)+p(1−p)2+⋯=1−(1−p)p=1 A geometric series with first term aaa and ratio rrr sums to a/(1−r)a/(1 - r)a/(1−r) . The sum must be 1 because the next success happens at some time.
[Exponential distribution] For a process with events at random times, what is the rate λ\lambdaλ , and how is it measured?
The average number of events per unit time: count the events in a long time interval and divide by its length. A completely random process (same probability at every instant) is fully described by this one number.Example: 300 events in 100 minutes gives λ=3\lambda = 3λ=3 per minute, which is also 1 per 20 seconds or 1/201/201/20 per second. "The rate is 3" alone means nothing.
[Exponential distribution] Events happen randomly at rate λ\lambdaλ . What is the expected number of events in a time ttt , and why?
λt\boldsymbol{\lambda t}λt Take nnn intervals of length ttt : in the total time ntntnt there are about ntλnt\lambdantλ events, so ntλ/n=λtnt\lambda / n = \lambda tntλ/n=λt per interval.Example: λ=3\lambda = 3λ=3 per minute and t=5t = 5t=5 minutes give 15 events on average, not exactly 15 every time.
[Exponential distribution] Events happen randomly at rate λ\lambdaλ . Is λt\lambda tλt the probability that exactly one event happens in a time ttt ?
Only when ttt is very small. In general λt\lambda tλt is only the expected number of events. Example: with λ=3\lambda = 3λ=3 per minute, t=5t = 5t=5 minutes gives λt=15\lambda t = 15λt=15 , which can't be a probability; t=20t = 20t=20 seconds gives λt=1\lambda t = 1λt=1 , but exactly one event isn't certain.
[Exponential distribution] Events happen randomly at rate λ\lambdaλ . What is the probability Pϵ(1)P_\epsilon(1)Pϵ(1) that exactly one event happens in a very small time interval ϵ\epsilonϵ ?
Pϵ(1)≈λϵ\boldsymbol{P_\epsilon(1) \approx \lambda \epsilon}Pϵ(1)≈λϵ , exact in the limit ϵ→0\epsilon \to 0ϵ→0 . Strictly, the condition is that λϵ\lambda\epsilonλϵ is small.Example: with λ=1/20\lambda = 1/20λ=1/20 per second, λt\lambda tλt as the probability of exactly one event is a poor approximation for t=20t = 20t=20 s, decent for t=2t = 2t=2 s, very good for t=0.2t = 0.2t=0.2 s.
[Exponential distribution] Events happen randomly at rate λ\lambdaλ ; Pt(k)P_t(k)Pt(k) is the probability of exactly kkk events in a time ttt .Why is Pt(1)≈λtP_t(1) \approx \lambda tPt(1)≈λt only for small ttt ?
Pt(1)=λt−2Pt(2)−3Pt(3)−…P_t(1) = \lambda t - 2P_t(2) - 3P_t(3) - \dotsPt(1)=λt−2Pt(2)−3Pt(3)−… , and the multiple-event terms are negligible only for small ttt . This comes from the expectation value λt=0⋅Pt(0)+1⋅Pt(1)+2Pt(2)+…\lambda t = 0 \cdot P_t(0) + 1 \cdot P_t(1) + 2 P_t(2) + \dotsλt=0⋅Pt(0)+1⋅Pt(1)+2Pt(2)+… . Pt(2)P_t(2)Pt(2) is proportional to (λt)2(\lambda t)^2(λt)2 and Pt(3)P_t(3)Pt(3) to (λt)3(\lambda t)^3(λt)3 , which are much smaller than λt\lambda tλt when λt\lambda tλt i…
[Exponential distribution] Events happen randomly at a constant rate λ\lambdaλ . Why is the graph of λ\lambdaλ against ttt not a probability distribution?
Its area λΔt\lambda \Delta tλΔt equals a probability only for very small intervals Δt\Delta tΔt , and its total area is infinite instead of 1. A probability density must give the probability for every interval, and must have total area 1.
[Exponential distribution] Events happen randomly at rate λ\lambdaλ in continuous time. What is the average waiting time τ\tauτ between events, and why?
τ=1/λ\boldsymbol{\tau = 1/\lambda}τ=1/λ In a long time ttt there are about λt\lambda tλt events, so τ=tλt=1λ\tau = \dfrac{t}{\lambda t} = \dfrac{1}{\lambda}τ=λtt=λ1 . This holds for the random process too, where the gaps are unequal.Example: 5 events per second give τ=1/5\tau = 1/5τ=1/5 second.
[Exponential distribution] Events happen randomly at rate λ\lambdaλ : in a small interval ϵ\epsilonϵ an event happens with probability λϵ\lambda\epsilonλϵ .How does the derivation get the probability P(t,Δt)P(t, \Delta t)P(t,Δt) that the next event happens between ttt and t+Δtt + \Delta tt+Δt (with Δt\Delta tΔt small)?
Split [0,t][0, t][0,t] into t/ϵt/\epsilont/ϵ tiny intervals: no event in each of them, then an event in Δt\Delta tΔt : P(t,Δt)=(1−λϵ)t/ϵ λΔtP(t, \Delta t) = (1 - \lambda\epsilon)^{t/\epsilon} \, \lambda \Delta tP(t,Δt)=(1−λϵ)t/ϵλΔt Then ϵ→0\epsilon \to 0ϵ→0 makes time continuous. ϵ\epsilonϵ is only a tool of the calculation; Δt\Delta tΔt is the interval whose probability is asked for.
[Exponential distribution] For small aaa (positive or negative), (1+a)n≈ ?(1 + a)^n \approx \; ?(1+a)n≈?
ena\boldsymbol{e^{na}}ena Example: a=0.01a = 0.01a=0.01 , n=200n = 200n=200 : 1.01200≈7.321.01^{200} \approx 7.321.01200≈7.32 and e2≈7.39e^{2} \approx 7.39e2≈7.39 .
[Exponential distribution] For events at rate λ\lambdaλ , the probability that the next event happens between ttt and t+Δtt + \Delta tt+Δt is P(t,Δt)=(1−λϵ)t/ϵ λΔtP(t, \Delta t) = (1 - \lambda\epsilon)^{t/\epsilon} \, \lambda \Delta tP(t,Δt)=(1−λϵ)t/ϵλΔt in the limit ϵ→0\epsilon \to 0ϵ→0 .How does P(t,Δt)P(t, \Delta t)P(t,Δt) simplify, and how do you get the density ρ(t)\rho(t)ρ(t) from P(t,Δt)P(t, \Delta t)P(t,Δt) ?
P(t,Δt)=e−λtλΔt\boldsymbol{P(t, \Delta t) = e^{-\lambda t} \lambda \Delta t}P(t,Δt)=e−λtλΔt , and dividing by Δt\Delta tΔt gives ρ(t)=λe−λt\boldsymbol{\rho(t) = \lambda e^{-\lambda t}}ρ(t)=λe−λt . (1+a)n≈ena(1 + a)^n \approx e^{na}(1+a)n≈ena with a=−λϵa = -\lambda\epsilona=−λϵ and n=t/ϵn = t/\epsilonn=t/ϵ gives na=−λtna = -\lambda tna=−λt . A density is a probability divided by the interval length.
[Exponential distribution] What is the exponential distribution ρ(t)\rho(t)ρ(t) of the waiting time, in terms of the rate λ\lambdaλ and in terms of the average waiting time τ\tauτ ?
ρ(t)=λe−λt=e−t/ττ\boldsymbol{\rho(t) = \lambda e^{-\lambda t} = \dfrac{e^{-t/\tau}}{\tau}}ρ(t)=λe−λt=τe−t/τ τ=1/λ\tau = 1/\lambdaτ=1/λ . In the general form ρ(t)=Ae−bt\rho(t) = Ae^{-bt}ρ(t)=Ae−bt , both AAA and bbb equal λ\lambdaλ .Example: τ=10\tau = 10τ=10 s gives ρ(t)=e−t/10/10\rho(t) = e^{-t/10}/10ρ(t)=e−t/10/10 , so ρ(0)=0.1\rho(0) = 0.1ρ(0)=0.1 per second.
[Exponential distribution] Remember the geometric distribution P(k)=(1−p)k−1pP(k) = (1 - p)^{k-1} pP(k)=(1−p)k−1p for the trial of the next success.Which substitutions turn P(k)P(k)P(k) into P(t,Δt)=(1−λϵ)t/ϵλΔtP(t, \Delta t) = (1 - \lambda\epsilon)^{t/\epsilon} \lambda \Delta tP(t,Δt)=(1−λϵ)t/ϵλΔt , the probability that the next random event (rate λ\lambdaλ ) happens between ttt and t+Δtt + \Delta tt+Δt ?
The ppp in 1−p1 - p1−p becomes λϵ\lambda\epsilonλϵ , the last ppp becomes λΔt\lambda\Delta tλΔt , and k−1k - 1k−1 becomes t/ϵt/\epsilont/ϵ . λϵ\lambda\epsilonλϵ is the success probability in each tiny interval, λΔt\lambda\Delta tλΔt in the last interval, and t/ϵt/\epsilont/ϵ is the number of tiny intervals before it.
[Exponential distribution] For the exponential distribution ρ(t)=e−t/τ/τ\rho(t) = e^{-t/\tau}/\tauρ(t)=e−t/τ/τ , what is the value at t=0t = 0t=0 , and why does it make sense?
ρ(0)=1/τ\boldsymbol{\rho(0) = 1/\tau}ρ(0)=1/τ : the larger τ\tauτ , the lower the start. With a large average waiting time, the next event is unlikely to happen right away. τ=5,10,20\tau = 5, 10, 20τ=5,10,20 : start values 0.2,0.1,0.050.2, 0.1, 0.050.2,0.1,0.05 .
[Exponential distribution] How fast does the exponential distribution ρ(t)=e−t/τ/τ\rho(t) = e^{-t/\tau}/\tauρ(t)=e−t/τ/τ decrease?
By a factor 1/e1/e1/e every time ttt increases by τ\tauτ : ρ(t+τ)=1eρ(t)\rho(t + \tau) = \frac{1}{e}\rho(t)ρ(t+τ)=e1ρ(t) for every ttt . ρ(0)=1τ, ρ(τ)=1eτ, ρ(2τ)=1e2τ,…\rho(0) = \frac{1}{\tau}, \ \rho(\tau) = \frac{1}{e\tau}, \ \rho(2\tau) = \frac{1}{e^2\tau}, \dotsρ(0)=τ1, ρ(τ)=eτ1, ρ(2τ)=e2τ1,… So a large τ\tauτ gives a curve that starts low and decreases slowly; a small τ\tauτ , one that starts high and decreases quickly.
[Exponential distribution] For the exponential distribution ρ(t)=e−t/τ/τ\rho(t) = e^{-t/\tau}/\tauρ(t)=e−t/τ/τ of waiting times, what are the total area and the expectation value?
Area 1, expectation value τ\tauτ . The waiting time has some value, so the total probability is 1; and τ\tauτ was defined as the average waiting time. Both need an integral to verify.
[Exponential distribution] Person A measures many waiting times of a random process with average waiting time τ=5\tau = 5τ=5 , person B of one with τ=20\tau = 20τ=20 . For which ttt do both see the same number of waiting times near ttt ?
t=203ln⁡4≈9.24\boldsymbol{t = \frac{20}{3}\ln 4 \approx 9.24}t=320ln4≈9.24 Set e−t/55=e−t/2020\dfrac{e^{-t/5}}{5} = \dfrac{e^{-t/20}}{20}5e−t/5=20e−t/20 : then ln⁡4=t(15−120)=3t20\ln 4 = t\left(\frac{1}{5} - \frac{1}{20}\right) = \frac{3t}{20}ln4=t(51−201)=203t .Such a ttt must exist by continuity: for small ttt the τ=5\tau = 5τ=5 density is larger (events closer together), for large ttt the τ=20\tau = 20τ=20 density is larger.
[Exponential distribution] ρ(t)=e−t/τ/τ\rho(t) = e^{-t/\tau}/\tauρ(t)=e−t/τ/τ is the distribution of waiting times from one event to the next. Does it also hold when the stopwatch starts at an arbitrary time?
Yes: ρ(t)\rho(t)ρ(t) is the distribution of the waiting time from any moment to the next event. The derivation never uses that an event happened at t=0t = 0t=0 .
[Exponential distribution] For a random process with average waiting time τ\tauτ , the waiting time from any moment to the next event has average τ\tauτ .An event happens at t=0t = 0t=0 . From t=1t = 1t=1 the next event comes on average τ\tauτ later. Why is the average waiting time from t=0t = 0t=0 still τ\tauτ and not 1+τ1 + \tau1+τ ?
The next event after t=0t = 0t=0 may already happen between t=0t = 0t=0 and t=1t = 1t=1 . Including these cases brings the average back down to τ\tauτ .
[Exponential distribution] In what sense are all exponential distributions ρ(t)=e−t/τ/τ\rho(t) = e^{-t/\tau}/\tauρ(t)=e−t/τ/τ the same curve?
With axes scaled so that τ\tauτ and 1/τ1/\tau1/τ are at fixed positions, all of them look identical. Stretching the ttt -axis by a factor fff and compressing the ρ\rhoρ -axis by the same factor fff turns one into another, and keeps the area (1): each thin vertical rectangle under the curve keeps its area.Example: at t=τt = \taut=τ the value is always 1/e1/e1/e times the value at t=0t = 0t=0 .
[Exponential distribution] Is the distribution of the waiting time until the next random event discrete or continuous?
Discrete in discrete time (geometric), continuous in continuous time (exponential). The random variable is the time itself. Morin warns against generalising: the Poisson distribution is discrete even in continuous time.
[Poisson distribution] What does the Poisson distribution describe, and what is its formula?
The number kkk of events in a given time interval, for a completely random process: P(k)=ake−ak!\boldsymbol{P(k) = \dfrac{a^k e^{-a}}{k!}}P(k)=k!ake−a aaa is the expected number of events in the interval. The exponential distribution describes the waiting time instead.Example: a=1a = 1a=1 : P(0)=e−1≈0.37P(0) = e^{-1} \approx 0.37P(0)=e−1≈0.37 , P(1)=e−1≈0.37P(1) = e^{-1} \approx 0.37P(1)=e−1≈0.37 , P(2)=e−1/2≈0.18P(2) = e^{-1}/2 \approx 0.18P(2)=e−1/2≈0.18 .
[Poisson distribution] nnn balls are thrown randomly into bbb boxes. What is the probability P(k)P(k)P(k) that a given box contains exactly kkk balls?
P(k)=(nk)(1b)k(1−1b)n−k\boldsymbol{P(k) = \binom{n}{k} \left(\frac{1}{b}\right)^{k} \left(1 - \frac{1}{b}\right)^{n-k}}P(k)=(kn)(b1)k(1−b1)n−k Each ball is a trial with success probability 1/b1/b1/b (it lands in the given box), so this is the binomial distribution with p=1/bp = 1/bp=1/b . n=1000n = 1000n=1000 , b=100b = 100b=100 : peaked at n/b=10n/b = 10n/b=10 ; P(2)≈0.2%P(2) \approx 0.2\%P(2)≈0.2% , P(21)≈0.1%P(21) \approx 0.1\%P(21)≈0.1% , P(1000)=10−2000P(1000) = 10^{-2000}P(1000)=10−200…
[Poisson distribution] Events happen randomly at rate λ\lambdaλ : in a tiny interval ϵ\epsilonϵ exactly one event happens with probability λϵ\lambda\epsilonλϵ .What is the first step of deriving the probability P(k)P(k)P(k) of exactly kkk events in a time ttt ?
Split ttt into n=t/ϵn = t/\epsilonn=t/ϵ tiny intervals, each one a trial with success probability λϵ\lambda\epsilonλϵ : P(k)=(nk)(λϵ)k(1−λϵ)n−kP(k) = \binom{n}{k} (\lambda\epsilon)^k (1 - \lambda\epsilon)^{n-k}P(k)=(kn)(λϵ)k(1−λϵ)n−k This is the binomial distribution with p=λϵp = \lambda\epsilonp=λϵ . Next comes the limit ϵ→0\epsilon \to 0ϵ→0 , n→∞n \to \inftyn→∞ .
[Poisson distribution] Deriving the Poisson distribution: n=t/ϵn = t/\epsilonn=t/ϵ tiny intervals, each with an event with probability λϵ\lambda\epsilonλϵ , so P(k)=n!(n−k)! k!(λϵ)k(1−λϵ)n(1−λϵ)−kP(k) = \dfrac{n!}{(n - k)! \, k!} (\lambda\epsilon)^k (1 - \lambda\epsilon)^{n} (1 - \lambda\epsilon)^{-k}P(k)=(n−k)!k!n!(λϵ)k(1−λϵ)n(1−λϵ)−k .Which three approximations are made as ϵ→0\epsilon \to 0ϵ→0 ( n→∞n \to \inftyn→∞ )?
Falling factorial n!(n−k)!≈nk\dfrac{n!}{(n - k)!} \approx n^k(n−k)!n!≈nk Example: the product of the 10 numbers from 1,000,000 down to 999,991 equals 1,000,000101{,}000{,}000^{10}1,000,00010 within 0.005%. The nnn -th power (1−λϵ)n≈e−nλϵ(1 - \lambda\epsilon)^n \approx e^{-n\lambda\epsilon}(1−λϵ)n≈e−nλϵ , by (1+a)n≈ena(1 + a)^n \approx e^{na}(1+a)n≈ena . Example: λ=2\lambda = 2λ=2 per second, t=1t = 1t=1 s: (1−2ϵ)1/ϵ→e−2(1 - 2\epsilon)^{1/\epsilon} \to e^{-2}(1−2ϵ)1/ϵ→e−2 . The (−k)(-k)(−k) -th…
[Poisson distribution] Why is n!(n−k)!≈nk\dfrac{n!}{(n - k)!} \approx n^k(n−k)!n!≈nk when nnn is much larger than kkk ?
n!(n−k)!\dfrac{n!}{(n - k)!}(n−k)!n! is the product of the kkk numbers n,n−1,…,n−k+1n, n - 1, \dots, n - k + 1n,n−1,…,n−k+1 , each almost equal to nnn . The approximation holds multiplicatively (the ratio tends to 1), not additively.Example: n=1000n = 1000n=1000 , k=2k = 2k=2 : 1000⋅999=999,0001000 \cdot 999 = 999{,}0001000⋅999=999,000 versus 10610^6106 , a 0.1% difference.
[Poisson distribution] As ϵ→0\epsilon \to 0ϵ→0 with n=t/ϵn = t/\epsilonn=t/ϵ , why does (1−λϵ)−k→1(1 - \lambda\epsilon)^{-k} \to 1(1−λϵ)−k→1 but (1−λϵ)n(1 - \lambda\epsilon)^{n}(1−λϵ)n does not?
kkk is fixed, but nnn grows as ϵ\epsilonϵ shrinks. A fixed power of a number close to 1 is close to 1. In (1−λϵ)n≈e−λnϵ=e−λt(1 - \lambda\epsilon)^n \approx e^{-\lambda n\epsilon} = e^{-\lambda t}(1−λϵ)n≈e−λnϵ=e−λt the exponent stays finite.
[Poisson distribution] In the Poisson distribution P(k)=ake−ak!P(k) = \dfrac{a^k e^{-a}}{k!}P(k)=k!ake−a for events at rate λ\lambdaλ during a time ttt , what is aaa ?
a=λt\boldsymbol{a = \lambda t}a=λt , the expected number of events in the time ttt . P(k)P(k)P(k) is sometimes written Pa(k)P_a(k)Pa(k) . aaa need not be an integer, but kkk is.
[Poisson distribution] Events happen randomly at rate λ\lambdaλ . If you double the time interval ttt and halve λ\lambdaλ , what happens to the distribution P(k)P(k)P(k) of the number of events?
Nothing: P(k)=ake−ak!P(k) = \dfrac{a^k e^{-a}}{k!}P(k)=k!ake−a depends only on the product a=λta = \lambda ta=λt . That the average stays the same is clear; that the whole distribution stays the same is what the formula shows.Example: one shopper per 15 seconds over a minute, and one shopper per 15 minutes over an hour, both have a=4a = 4a=4 and the same P(k)P(k)P(k) .
[Poisson distribution] Is the Poisson distribution discrete or continuous, given that it comes from a process in continuous time?
Discrete: the random variable is the number of events kkk , an integer. Whether time is continuous doesn't matter for this. For the exponential distribution, time itself is the random variable, so continuous time gives a continuous distribution.
[Poisson distribution] On average one shopper enters a store every 15 seconds. What is the probability that in a given minute 0, 4, or 8 shoppers enter?
P(0)≈2%\boldsymbol{P(0) \approx 2\%}P(0)≈2% , P(4)≈20%\boldsymbol{P(4) \approx 20\%}P(4)≈20% , P(8)≈3%\boldsymbol{P(8) \approx 3\%}P(8)≈3% Poisson with a=4a = 4a=4 : P(0)=e−4≈0.018P(0) = e^{-4} \approx 0.018P(0)=e−4≈0.018 , P(4)=444!e−4=323e−4≈0.195P(4) = \frac{4^4}{4!}e^{-4} = \frac{32}{3}e^{-4} \approx 0.195P(4)=4!44e−4=332e−4≈0.195 , P(8)=488!e−4=512315e−4≈0.030P(8) = \frac{4^8}{8!}e^{-4} = \frac{512}{315}e^{-4} \approx 0.030P(8)=8!48e−4=315512e−4≈0.030 . P(k)P(k)P(k) for a=4a = 4a=4 .
[Poisson distribution] For the Poisson distribution P(k)=ake−ak!P(k) = \dfrac{a^k e^{-a}}{k!}P(k)=k!ake−a with a=4a = 4a=4 , why is P(3)=P(4)P(3) = P(4)P(3)=P(4) ?
444!=4⋅434⋅3!=433!\dfrac{4^4}{4!} = \dfrac{4 \cdot 4^3}{4 \cdot 3!} = \dfrac{4^3}{3!}4!44=4⋅3!4⋅43=3!43 In general P(a−1)=P(a)P(a - 1) = P(a)P(a−1)=P(a) for an integer aaa .
[Poisson distribution] The Poisson distribution P(k)=ake−ak!P(k) = \dfrac{a^k e^{-a}}{k!}P(k)=k!ake−a with a=npa = npa=np can approximate the binomial distribution (nk)pk(1−p)n−k\binom{n}{k}p^k(1 - p)^{n - k}(kn)pk(1−p)n−k .When is the approximation good?
When p≪1p \ll 1p≪1 . Dots: binomial; curves: Poisson with a=10a = 10a=10 . Left p=1/10p = 1/10p=1/10 (slightly off), right p=1/100p = 1/100p=1/100 (essentially equal).
[Poisson distribution] In the Poisson distribution P(k)=ake−ak!P(k) = \dfrac{a^k e^{-a}}{k!}P(k)=k!ake−a , what is P(0)P(0)P(0) , and how does it behave for small and large aaa ?
P(0)=e−a\boldsymbol{P(0) = e^{-a}}P(0)=e−a : close to 1 for small aaa , close to 0 for large aaa . With a large expected number of events, zero events is unlikely; with a small one, likely.Example: a=0.5a = 0.5a=0.5 : e−0.5≈0.61e^{-0.5} \approx 0.61e−0.5≈0.61 ; a=1a = 1a=1 : e−1≈0.37e^{-1} \approx 0.37e−1≈0.37 .
[Poisson distribution] How does the shape of the Poisson distribution P(k)=ake−ak!P(k) = \dfrac{a^k e^{-a}}{k!}P(k)=k!ake−a change as aaa increases?
Small aaa (such as a<1a < 1a<1 ) No peak: P(k)P(k)P(k) is largest at k=0k = 0k=0 and decreases. The peak moves right It is located near k=ak = ak=a . The peak gets lower The peak gets wider Lower and wider together keep the total probability 1. a=0.5,1,2,5,10,15a = 0.5, 1, 2, 5, 10, 15a=0.5,1,2,5,10,15 , drawn as curves although only integer kkk count.
[Gaussian distribution] What are the two usual forms of the Gaussian distribution?
f(x)=bπ e−b(x−μ)2=12πσ2 e−(x−μ)2/2σ2\boldsymbol{f(x) = \sqrt{\dfrac{b}{\pi}}\, e^{-b(x - \mu)^2} = \sqrt{\dfrac{1}{2\pi\sigma^2}}\, e^{-(x - \mu)^2/2\sigma^2}}f(x)=πbe−b(x−μ)2=2πσ21e−(x−μ)2/2σ2 The second form uses b=1/2σ2b = 1/2\sigma^2b=1/2σ2 ; it is more common because σ\sigmaσ is the standard deviation. It is a continuous distribution, so fff is a probability density.Example: μ=0\mu = 0μ=0 , σ=1\sigma = 1σ=1 : f(x)=12πe−x2/2f(x) = \frac{1}{\sqrt{2\pi}} e^{-x^2/2}f(x)=2π1e−x2/2 , with…
[Gaussian distribution] The Gaussian distribution is also called the normal distribution or the bell curve.
It is the most important distribution: in the limit of large numbers, many other distributions become Gaussian.
[Gaussian distribution] What naming convention distinguishes probability densities from probabilities?
Lowercase letters for densities, uppercase for probabilities. Example: the Gaussian density f(x)f(x)f(x) versus the Poisson probability P(k)P(k)P(k) .
[Gaussian distribution] Where is the maximum of the Gaussian f(x)=b/π e−b(x−μ)2f(x) = \sqrt{b/\pi}\, e^{-b(x - \mu)^2}f(x)=b/πe−b(x−μ)2 , and why?
At x=μx = \mux=μ , where the exponent −b(x−μ)2-b(x - \mu)^2−b(x−μ)2 , which is never positive, equals 0. Increasing μ\muμ (same bbb ) moves the curve to the right without changing its shape. The curve is symmetric around μ\muμ , so μ\muμ is also the mean.Example: 2/π e−2(x−10)2\sqrt{2/\pi}\, e^{-2(x - 10)^2}2/πe−2(x−10)2 has its peak at x=10x = 10x=10 .