In most experiments, researchers are interested to know whether there is an effect or difference between groups they are testing. There is always the possibility that there is no effect! This lack of an effect or a difference is called the Null hypothesis.
Let's assume we study the effectiveness of a new drug. Despite the null being true, it’s entirely possible that there will be an effect in the sample data due to random sampling error. The devil’s advocate position is that the observed difference in the sample does not reflect a true difference between populations. This is why we need P Values. They evaluate how well the sample data support the devil’s advocate argument that the null hypothesis is true.
- High P values: your data are likely with a true null
- Low P values: your data are unlikely with a true null
A low P Value suggests that your sample provides enough evidence that you can reject the null hypothesis for the entire population.