Hypothesis Testing

Key Questions

  • In short: If the other side is not important or not possible.

    You want to examine whether "brain gym" (a mixture of small mental and physical exercises) will improve your pupils' scores. They cannot harm their performances, as they're not tiring or confusing at all.

    In that case you may set up a control group or do a before/after test, whatever.

    • Your zero-hypothesis is that there will be no difference.
    • Your alternative hypothesis is that there will be improvement.

    Then you do all your testing and measuring and, following all the statistical rules, you decide whether the difference is significant or not.

  • One of the most useful features of statistics is the ability to make inferences about a population based on a sample. The term 'population' is to be understood in context. We may want to make an inference about all men over 18 in France by taking a sample of men over 18 in France. (In this case the 'population' of interest is all men over the age of 18 in France). Or we may want to make an inference about all eucalyptus trees in a national park in Australia, by taking a sample of such trees from the same national park. (In this case the 'population' of interest is all eucalyptus trees in the national park.)

    So, what we are interested in is understanding some feature of a population. This is called a 'hypothesis'. However, we rarely have enough resources (time and money) to sample the entire population. So, what we need to do is to take a sample, and then test whether we have enough evidence to support our hypothesis. This is what is called 'hypothesis testing'.

    Let's say we were interesting in learning whether the average height of men over 18 in France is at least 6"0' (six feet). It is not feasible, normally, to go and measure every man in France over 18. So we take a sample of men over 18 and measure their heights. Let's say we find that their average height is 5"8'. A hypothesis test would then tell us the probability of observing our sample average if the actual average height of all men in France over 18 is 6"0'.

    Typically, we would reject our hypothesis if there is a less than 95% chance of the hypothesis being true given the observed sample.