Interpreting test statistics, p-values, and significance
Analysis Test statistic Null hypothesis Alternative hypothesis Results p-value significance decision
Difference-of- means test  t (two-tailed) (see note 1)  m1 = m2  m1 ne m2 big t (> +2.0 
or < -2.0)		 small p
(< 0.05)		 yes
(significant difference of means)		 reject Ho,
accept Ha
  small t (< +2.0 
and > -2.0)		 big p
( > 0.05)		 no don't reject Ho
   t (one-tailed) (see note 2)  m1 > m2  m1 ≤ m2 big t (> +2.0 
or < -2.0)		 small p
( < 0.05)		 yes
(significant difference of means)		 reject Ho,
accept Ha
small t (< +2.0 
and > -2.0)		 big p
( > 0.05)		 no don't reject Ho
 Analysis of variance (ANOVA)  F
(see note 3)		 m1 = m2 = m3 = ... = mk  m1 ≠ m2 ≠ m3 ≠ ... ≠ mk big F  small p
( < 0.05)		 yes
(significant difference among means)		 reject Ho,
accept Ha
        small F big p
( > 0.05)		 no don't reject Ho
Homogeneity of variance (Bartlett) K2
(see note 4)		 s21 = s22 = s23 = ... = s2k s21 ≠ s22 ≠ s23 ≠ ... ≠ s2k big K2 small p
( < 0.05)		 yes
(sig. difference among variances)		 reject Ho,
accept Ha
        small K2 big p
( > 0.05)		 no don't reject Ho
 Regression analysis F
(see note 5)

no relationship between response and predictor vars.

relationship between response and predictor vars.

  big F small p (<0.05) yes
(there is a relationship)		 reject Ho,
accept Ha 
small F big p
(>0.05)		 no (there is not a relationship) don't reject Ho
t
 (see note 6)		 bp = 0 bp ≠ 0 big t (> +2.0 
or < -2.0)		 small p
( < 0.05)		 yes (xp is an important predictor) reject Ho,
accept Ha
small t (< +2.0 
and > -2.0)		 big p
( > 0.05)		 no  (xp is not an important predictor) don't reject Ho

Notes:

1) The null hypothesis here is that the means are equal, and the alternative hypothesis is that they are not.  A big t, with a small p-value, means that the null hypothesis is discredited, and we would assert that the means are significantly different (while a small t, with a big p-value indicates that they are not significantly different).

2) The null hypothesis here is that one mean is greater than the other, and the alternative hypothesis is that it isn't.  A big t, with a small p-value, means that the null hypothesis is discredited, and we would assert that the means are significantly different in the way specified by the null hypothesis (and a small t, with a big p-value means they are not significantly different in the way specified by the null hypothesis).

3)  The null hypothsis here is that the group means are all equal, and the alternative hypothesis is that they are not.  A big F, with a small p-value, means that the null hypothesis is discredited, and we would assert that the means are significantly different (while a small F, with a big p-value indicates that they are not significantly different).

4)  The null hypothsis here is that the group variances are all equal, and the alternative hypothesis is that they are not.  A big K2, with a small p-value, means that the null hypothesis is discredited, and we would assert that the group variances are significantly different (while a small K2, with a big p-value indicates that they are not significantly different).

5)  The null hypothesis here is that there is not a general relationship between the response (dependent) variable and one or more of the predictor (independent) variables, and the alternative hypothesis is that there is one.  A big F, with a small p-value, means that the null hypothesis is discredited, and we would assert that there is a general relationship between the response and predictors (while a small F, with a big p-value indicates that there is no relationship).

6)  The null hypothesis is that the value of the p-th regression coefficient is 0, and the alternative hypothesis is that it isn't.  A big t, with a small p-value, means that the null hypothesis is discredited, and we would assert that the regression coefficient is not 0 (and a small t, with a big p-value indicates that it is not significantly different from 0).