What does a p-value actually mean?

Q: What is a p-value in simple terms?

A p-value is the probability of observing a result at least as extreme as yours if the null hypothesis (no effect) were true. A small p-value means your data would be surprising under 'no effect', which is taken as evidence against the null.

Q: Does p < 0.05 mean my hypothesis is true?

No. The p-value is not the probability that the null is true or that your hypothesis is true, and it does not tell you the size or importance of an effect. p < 0.05 only means the data are unlikely under the null at a conventional threshold.

Q: What should I report instead of just a p-value?

Report the effect size and a confidence interval alongside the p-value. The effect size tells you how large the effect is, and the confidence interval shows the range of plausible values — together they convey both significance and magnitude.

Q: Is a smaller p-value a bigger effect?

No. A very small p-value can come from a large sample even when the effect is trivial. Statistical significance and practical importance are different things — always read the p-value next to the effect size.

By Dr. Rafiq Muhammad, MD, PhD · Updated June 2026

A p-value is the probability of getting a result at least as extreme as yours if the null hypothesis — “there is no real effect” — were true. A small p-value means your data would be surprising in a world with no effect, which we treat as evidence against that “no effect” assumption. That’s all it is — and several tempting readings of it are simply wrong.

What a p-value is not

Not the probability that the null hypothesis is true.
Not the probability that your hypothesis is true.
Not the probability your result was due to chance.
Not a measure of how large or important the effect is.
Not a guarantee the finding will replicate.

The big one: p = 0.04 does not mean “96% chance the effect is real.” It means “if there were no effect, data like mine would occur about 4% of the time.” Those are very different statements.

Why “p < 0.05” isn’t the finish line

The 0.05 threshold is a convention, not a law of nature. Two studies can both report “p < 0.05” while one found a life-changing effect and the other a trivial one — because significance depends on sample size. With a large enough sample, a meaningless difference becomes “significant.” That’s why a p-value alone never tells the whole story.

What to report instead

Report the effect size and a confidence interval alongside the p-value. The effect size says how big; the confidence interval shows the range of plausible values; the p-value says how surprising under the null. Together they let a reader judge both whether an effect is real and whether it matters.

Have a confidence interval but need the p-value (or vice versa)? The free p-value ↔ Confidence Interval converter moves between them using the Altman–Bland method.

Get the free Statistics toolkit

A reporting checklist, p-value and confidence-interval worked examples, and test-selection flowcharts from the Statistical Test Selection Workbook. We’ll email you the download link.

Get the Statistical Test Selection Workbook

Frequently asked questions

What is a p-value in simple terms?

The probability of a result at least as extreme as yours if there were no real effect. Small = your data would be surprising under “no effect.”

Does p < 0.05 mean my hypothesis is true?

No — it isn’t the probability the null (or your hypothesis) is true, and it says nothing about effect size.

What should I report instead of just a p-value?

The effect size and a confidence interval alongside it — magnitude and plausible range, not just significance.

Is a smaller p-value a bigger effect?

No. A tiny p-value can come from a large sample with a trivial effect. Read it next to the effect size.

Effect size explained → Open the p↔CI converter →