Definition
A p-value (probability value) is a number between 0 and 1 that measures how likely you would see results as extreme as what you observed, assuming there is no real effect. A very small p-value means your result would be extremely unlikely under the assumption of sheer chance. It does not tell you the probability that your hypothesis is right.
Let's Start with a Coin Toss Bet
Imagine you make a bet with a friend on a coin flip, and the coin lands on heads 10 times in a row. Naturally, you would start to wonder, 'Is this coin rigged?'
With a fair coin, the chance of getting heads 10 times straight is only about 0.1%. The p-value is simply the calculated probability of seeing an outcome this extreme, assuming the coin is completely fair.
If the p-value of this toss is 0.001 (0.1%), it means that 'assuming no trickery, the odds of seeing 10 straight heads just by chance' is about 1 in 1,000 (specifically, 1 in 1,024). When the odds are that ridiculously low, we reject mere coincidence and strongly suspect that our friend is cheating.
A Plant Fertilizer Test and the 0.05 Threshold
Suppose you give a new eco-friendly fertilizer to a houseplant, and it grows noticeably faster than a plant given only plain water. Now you must figure out: did the plant thrive because of the fertilizer, or did you just happen to pick an unusually hardy seed?
This is where scientists test statistical hypotheses. By convention, researchers treat results as significant when the chance of seeing such a difference by pure fluke is under 5%—meaning a p-value below 0.05. This cutoff is called the significance level.
If your fertilizer experiment yields a p-value of 0.02 (2%), that means 'assuming the fertilizer has zero effect, the chance of seeing a difference this large or larger purely by luck' is just 2%. Because that is lower than the 5% threshold, researchers conclude the result is statistically significant and cite it as evidence that the fertilizer works. Still, there is always a slim chance that luck fooled them.
Looking a Little Closer
To be precise, a p-value does not tell you 'the probability that a new discovery is actually true.' It strictly measures the conditional odds of stumbling onto this data or something more extreme, assuming there is zero real effect in the first place.
Also, getting a p-value below 0.05 does not automatically mean a breakthrough is huge or groundbreaking. If you blow up your sample size to tens of thousands of plants, even a tiny, meaningless difference can produce a p-value under 0.05 and be flagged as significant.
In the end, the p-value is a great traffic light for filtering out wild coincidences, but it cannot vouch for the practical importance or the absolute truth of a claim. That is why modern researchers look at p-values alongside effect size and confidence intervals.
Common misconceptions
A p-value of 0.01 means there is a 99% chance my hypothesis is true.
A p-value is not the probability that your hypothesis is right. It only measures how rarely you would see data like this purely by chance if there were zero real effect.
Where you meet it
Checking whether improved test scores after introducing a new teaching method are too large to be explained by pure luck.
Running an A/B test to see if a bump in website button clicks is a genuine user preference or just random noise.
In one sentence
The p-value measures how likely your results would be if there were no real effect, and results below 0.05 are typically considered too unusual to chalk up to chance.