Hypothesis tests

The next step toward statistical inference is the more formal development and testing of specific hypotheses (as opposed to the rather informal inspection of descriptive plots, confidence intervals, etc.)

"Hypothesis" is a word used in several contexts in data analysis or statistics:

• the research hypothesis is the general scientific issue that is being explored by a data analysis. It may take the form of quite specific statements, or just general speculations.
• the null hypothesis (H_o) is a specific statement whose truthfulness can be evaluated by a particular statistical test. An example of a null hypothesis is that the means of two groups of observations are identical.
• the alternative hypothesis (H_a) is, as its name suggests an alternative statement of what situation is true, in the event that the null hypothesis is rejected. An example of an alternative hypothesis to a null hypothesis that the means of two groups of observations are identical is that the means are not identical.

A null hypothesis is never "proven" by a statistical test. Tests may only reject, or fail to reject, a null hypothesis.

There are two general approaches toward setting up and testing specific hypotheses: the "classical approach" and the "p-value" approach.

The steps in the classical approach:

1. define or state the null and alternative hypotheses.
2. select a test statistic.
3. select a significance level, or a specific probability level, which if exceeded, signals that the test statistic is large enough to consider significant.
4. delineate the "rejection region" under the pdf of the appropriate distribution for the test statistic, (i.e. determine the specific value of the test statistic that if exceeded would be grounds to consider it significant.
5. compute the test statistic.
6. depending on the particular value of the test statistics either a) reject the null hypothesis (H_o) and accept the alternative hypothesis (H_a), or b) fail to reject the null hypothesis.

The steps in the "p-value" approach are:

1. define or state the null and alternative hypotheses.
2. select and compute the test statistic.
3. refer the test statistic to its appropriate reference distribution.
4. calculate the probability that a value of the test statistic as large as that observed would occur by chance if the null hypothesis were true (this probability, or p-value, is called the significance level).
5. if the significance level is small, the tested hypothesis (H_o) is discredited, and we assert that a "significant result" or "significant difference" has been observed.

A short guide to interpreting test statistics, p-values, and significance