Personal notes on hypothesis testing in statistics.
Based on notes taken during a course [1].
Hypotheses
- null hypothesis
- Hypothesis assumed to be true. Often denoted by \(H_0\).
- alternative hypothesis
- Hypothesis for which we are looking for evidence. Often denoted by \(H_1\) or \(H_a\).
- simple hypothesis
- Hypothesis which completely specifies the distribution.
- composite hypothesis
- Hypothesis which is not simple. I.e., aspects of the distribution are left unspecified.
Hypothesis testing either rejects or fails to reject the null hypothesis.
Types of error
- Type I error
- Rejecting the null hypothesis when it is true.
- Type II error
- Failing to reject the null hypothesis when it is false.
Power functions
A power function \(\gamma(\theta)\) for some parameter is the probability that the null hypothesis is rejected when the parameter is \(\theta\).
P-values
Given a sample mean and a distribution for a null hypothesis, the probability that samples from the distribution will fall outside the range of the sample mean.
QQ plot
Quantile-quantile plot. Useful for checking for normality by plotting the quantile for the sample in one axis and the quantile for a normal distribution for the same sample in another axis; plot will be linear if the sample follows a normal distribution.
Sample variance
The naive estimator of the variance is biased.
\[ E\left[ \frac{1}{n} \sum_i (X_i - \overline{X})^2 \right] = \frac{n-1}{n} \sigma^2 \]
The following definition of the sample variance is unbiased.
\[ S^2 = \frac{\sum_i (X_i - \overline{X})^2}{n-1} \]
See also properties of the \(\chi^2\) distribution.
\(t\)-test
When \(X\) has a normal distribution,
\[ \frac{\overline{X} - \mu}{S / \sqrt{n}} \sim t(n-1) \]
\(t\)-tests are usually only used with small sample sizes. The \(t\)-distribution is virtually indistinguishable from normal distribution for large sample sizes.