# A number of significance

Signiﬁcance has a special meaning in stats

If you want to accept your alternative hypothesis, you must ﬁrst reject your null hypothesis. There is a chance, however, of rejecting the null hypothesis when it is actually true. It is usually possible to calculate the probability (p-value) that what you observed in an experiment was due just to chance.

You use a signiﬁcance level to decide whether you will reject the null hypothesis, and this is often set at the 0.05 or 5 per cent level. If your measured p-value equals 0.04, for example, then this is less than 5 per cent, so you can reject the null hypothesis and accept your alternative. Still, this doesn’t mean that you have proved the alternative hypothesis: if the null hypothesis were true, there would still be a 4 per cent chance of getting your result.

If an unscrupulous investigator keeps on doing experiments on useless treatments, they will still get results ‘signiﬁcant at the 5 per cent level’ on 1 in 20 occasions. If only those ‘positive’ trials are reported, we will get a very misleading impression. This is why it is essential to have access to all the evidence, whether positive or negative. ‘All trials registered, all results reported’ is a campaign started by researchers, doctors and others to try and get all clinical trials past and present to be registered and to have their results reported.