Suppose we care about a population of $N$ people and one number for each of them, $X$. Below, $X$ is (variable description). The population has a mean $\mu$ and a variance $\sigma^2$. In real life we never see the whole population. We see a random sample of $n$ people, $X_1, \dots, X_n$, each drawn with equal probability and independently of the others, and we compute the sample mean $$\overline{X} = \frac{1}{n}\sum_{i=1}^n X_i .$$ In this (contrived) example, we do have the whole population, so we can draw sample after sample and watch what $\overline{X}$ does. In practice this is infeasible: we will typically have one random sample, and so one value, $\overline{x}$. But this simulation lets us compare the distribution of $\overline{X}$ over many samples (which we could never see in practice) with the theoretical distribution implied by the CLT.
population histogram $\mu$
Pick a sample size $n$, then press Draw 1 sample. The $n$ draws appear as dots on the population axis and their mean $\overline{x}$ drops into the lower histogram. Press it a few times: the sample mean comes out different every time. A statistic computed from a random sample is itself a random variable. Then press Draw 1,000 to see its distribution (the sampling distribution of $\overline{X}$) take shape. (With real data this step is infeasible: we would have only one sample.)
Changing $n$ starts a new experiment: the lower histogram is cleared, because $\overline{X}$ has a different distribution for each $n$. Draws are with replacement (the same person can appear twice), which is what makes the $X_i$ independent.
Latest sample:
histogram of sample means so far normal curve with mean $\mu$ and variance $\sigma^2/n$ $\mu$
| Theory | Simulation | |
|---|---|---|
| Number of samples drawn | 0 | |
| Mean of $\overline{X}$: $E[\overline{X}] = \mu$ | ||
| Variance of $\overline{X}$: $\sigma^2_{\overline{X}} = \sigma^2 / n$ | ||
| Standard deviation of $\overline{X}$ (the standard error): $\sigma_{\overline{X}} = \sigma/\sqrt{n}$ |
Samples are drawn with replacement, so the $X_i$ are independent and identically distributed, as the theory assumes.
The Central Limit Theorem describes the distribution of the sample mean for a random sample from any population (with a finite variance), as long as the draws are i.i.d. and $n$ is large. As you can see in the simulation, the shape becomes approximately normal even though the population is not. At $n = 1$ the histogram of sample means is the population. At $n = 5$ or $10$ it is still lopsided on the right. By $n = 30$ or so the normal curve fits. The Central Limit Theorem says that for large $n$, $$Z = \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} \;\overset{\text{approx}}{\sim}\; \normal{0}{1}, \qquad\text{equivalently}\qquad \overline{X} \;\overset{\text{approx}}{\sim}\; \normal{\mu}{\sigma^2/n}.$$ Switch to the standardized $z$ view to watch the histogram approach the $\normal{0}{1}$ curve as $n$ grows.
Two facts about the centre and spread hold for every $n$, not just large $n$:
When the population is itself normal, no approximation is needed: $\overline{X}$ is exactly normal for every $n$, because sums of independent normals are normal (NCT Chapter 5, Oct 20).
Once $\overline{X} \overset{\text{approx}}{\sim} \normal{\mu}{\sigma^2/n}$, we can compute probabilities about the sample mean using $\Phi$, as in NCT Chapter 5, with $\sigma/\sqrt{n}$ in the denominator instead of $\sigma$: $$Pr(\overline{X} \le c) \;\approx\; \Phi\!\left(\frac{c - \mu}{\sigma/\sqrt{n}}\right).$$
Draw some samples first.