A number of significance

Significance has a special meaning in stats

If you want to accept your alternative hypothesis, you must first reject your null hypothesis. There is a chance, however, of rejecting the null hypothesis when it is actually true. It is usually possible to calculate the probability (p-value) that what you observed in an experiment was due just to chance.

You use a significance level to decide whether you will reject the null hypothesis, and this is often set at the 0.05 or 5 per cent level. If your measured p-value equals 0.04, for example, then this is less than 5 per cent, so you can reject the null hypothesis and accept your alternative. Still, this doesn’t mean that you have proved the alternative hypothesis: if the null hypothesis were true, there would still be a 4 per cent chance of getting your result.

If an unscrupulous investigator keeps on doing experiments on useless treatments, they will still get results ‘significant at the 5 per cent level’ on 1 in 20 occasions. If only those ‘positive’ trials are reported, we will get a very misleading impression. This is why it is essential to have access to all the evidence, whether positive or negative. ‘All trials registered, all results reported’ is a campaign started by researchers, doctors and others to try and get all clinical trials past and present to be registered and to have their results reported.

About this resource

This resource was first published in ‘Number Crunching’ in June 2013.

Statistics and maths, Medicine
Number Crunching
Education levels:
16–19, Continuing professional development, Undergraduate