Hypothesis Testing Explained

Hypothesis testing is how statistics decides whether the evidence in a sample is strong enough to believe a claim about a population. You start by assuming the claim is false, then ask how surprising your data would be if that assumption were true — and let the p-value give the answer.

This guide explains the five steps of a hypothesis test in plain language, then works through three complete examples — a z-test, a t-test and a chi-square test — using the site’s own live calculators, starting with the One-Proportion Z-Test Calculator, so you can see every test statistic, p-value and decision for yourself.

Short answer: Hypothesis testing is a procedure that uses sample evidence to test a claim about a population. You assume the null hypothesis is true, measure how surprising the data are with a p-value, and reject the null hypothesis only if the p-value is smaller than the significance level you chose. A test never proves a claim true or false.

What Is Hypothesis Testing?

A hypothesis test is a formal procedure for deciding whether the data in a sample support a claim about a population. The claim might be about a proportion (“half of all voters support the measure”), a mean (“the average battery lasts 100 hours”) or a relationship (“smoking and lung cancer are associated”). Because you cannot measure the whole population, you take a sample and ask: is what we observed consistent with the claim, or is it too far away to be explained by chance alone?

The logic is the same as a courtroom trial. The defendant is presumed innocent until the evidence proves guilt beyond reasonable doubt. In a hypothesis test the null hypothesis plays the defendant: it is assumed true until the sample evidence is strong enough to convict it. The verdict is never “innocent” or “guilty” — it is “reject the null hypothesis” or “fail to reject the null hypothesis”, and the strength of the evidence is measured by the p-value, which you will meet in Step 4.

Every test starts with two competing statements. The null hypothesis (H0, “H-naught”) says there is no effect, no difference or no association — the status quo. The alternative hypothesis (H1 or Ha) is the claim the test is designed to detect: an effect exists, a difference exists, an association exists. The test never proves either one; it only weighs the evidence against H0. Step 1 below looks at both hypotheses in detail.

The 5-Step Procedure

Every hypothesis test follows the same five steps, whatever the question and whatever the test statistic. The rest of this guide walks through them one by one, then runs three complete examples through them.

1. State the null and alternative hypotheses, and choose the significance level α.
2. Collect the data and check the conditions the test needs.
3. Compute the test statistic (z, t or χ²).
4. Find the p-value.
5. Decide: reject H0 if p < α, otherwise fail to reject H0 — then state the conclusion in plain language.

Step 1: The Null and Alternative Hypotheses

The null hypothesis H0 is the conservative claim: no difference, no effect, no change. The alternative hypothesis Ha (also written H1) is the complement of H0 — the claim the test is designed to detect. In the worked examples below, H0 is “the coin is fair, p = 0.5”, “the mean is 100” and “the two variables are independent”. Ha is the opposite in each case.

Ha comes in two shapes, and the shape decides where the rejection region sits. In a two-tailed test, Ha says the value is different from H0 (p ≠ 0.5 or μ ≠ 100). The rejection region is split between both tails, so each tail gets α/2. This is the default choice when you do not know the direction of the effect in advance. In a one-tailed test, Ha says the value is bigger (or smaller) than H0. All of α sits in one tail, so the test is more sensitive in that direction — but it cannot detect an effect in the other direction at all. Choose one-tailed only if you have a clear, pre-registered reason to look in one direction; choosing it after seeing the data is cheating (see Common Mistakes).

For a two-tailed test at α = 0.05 the critical z-values are ±1.96; for a one-tailed test the critical value is 1.645 (upper tail) or −1.645 (lower tail). The critical t-values depend on the degrees of freedom — with 24 degrees of freedom, for example, the two-tailed critical t at α = 0.05 is 2.064, and you will see it in Worked Example 2. If the exact value matters for your test, use the Critical Z-Value Calculator, or read how to find a critical value for the step-by-step method.

Step 2: Check the Conditions

A test statistic is only trustworthy if the conditions behind it hold. Three conditions appear in almost every test.

  • Random sample: the data come from a random sample (or a randomised experiment), so they represent the population.
  • Independence: the observations do not influence each other. For a proportion test, the sample should be no more than 10% of the population.
  • Normality, or a large enough sample: for a z-test on a proportion, np0 and n(1 − p0) should both be at least 10. For a t-test on a mean, the sample should come from a roughly normal population, or n should be large enough (usually n ≥ 30) for the central limit theorem to apply. For a chi-square test, every expected count should be at least 5.

The three worked examples below each name the condition they rely on, so you can see what to check before you trust a p-value.

Step 3: The Test Statistic

The test statistic measures how far the sample result sits from what H0 predicts, in standard-error units. Three test statistics do almost all the work.

  • z: used for proportions, and for means when the population standard deviation is known. A z of 2.83 means the sample proportion is 2.83 standard errors above the claimed value.
  • t: used for means when the population standard deviation is unknown and must be estimated from the sample. It behaves like z, but with heavier tails, and its shape depends on the degrees of freedom. If you are unsure which one your data needs, see t-distribution vs normal distribution.
  • χ² (chi-square): used for counts in categories. It adds up (observed − expected)² / expected over every cell of a table; large values mean the observed counts are far from what H0 predicts.

The bigger the test statistic (in absolute value), the more surprising the data are under H0 — and the smaller the p-value will be.

Step 4: The P-Value

The p-value is the probability of getting data at least as extreme as the ones you observed, if the null hypothesis were true: P(data this extreme | H0 true). It is not the probability that H0 is true — that is the most common mistake in statistics, and Step 5 and Common Mistakes come back to it.

Take z = 2.10. The P-Value Calculator gives a two-tailed p-value of 0.0357 and a one-tailed p-value of 0.0179, with Φ(2.10) = 0.9821. In words: if H0 were true, a z-score as far from zero as 2.10 would appear about 3.6% of the time in a two-tailed test — rare enough to reject H0 at α = 0.05, but not by much. Run the reverse calculation too: a two-tailed p of 0.05 corresponds to a z of ±1.96, the most famous number in statistics.

If you want the full treatment of what a p-value means (and does not mean), read What Is a P-Value?.

How to read a p-value at a glance:

p-valueWhat it means
p < 0.01Very strong evidence against H0
0.01 to 0.05Strong evidence against H0
0.05 to 0.10Weak evidence against H0 — worth a second look, not a conclusion
p ≥ 0.10Little or no evidence against H0

What a p-value is NOT:

  • Not the probability that H0 is true. A p-value of 0.0047 does not mean there is a 0.47% chance the null hypothesis is true.
  • Not the size of the effect. A tiny p-value can come from a trivial difference if the sample is huge; a large p-value can hide a big difference if the sample is small. Always report the effect size as well — for a mean, Cohen’s d; for a 2×2 table, Cramér’s V.
  • Not a verdict on importance. Statistical significance and practical importance are different questions.

Step 5: Decide and Conclude

Compare the p-value with the significance level α you chose in Step 1. If p < α, the data are too surprising under H0: reject H0. If p ≥ α, the data are not surprising enough: fail to reject H0. Then translate the decision into a plain-language conclusion about the original claim.

The language matters. “Fail to reject H0” is not the same as “accept H0”. A test that finds no evidence against the null has not proved the null true — it has only failed to find enough evidence to overturn it. Absence of evidence is not evidence of absence. The NIST/SEMATECH e-Handbook states the same rule in its chapter on statistical tests: a test can reject the null hypothesis or fail to reject it — the language is deliberate, and “accept” never appears.

And α = 0.05 is a convention, not a law. It comes from R. A. Fisher’s suggestion in the 1920s that 1 in 20 is a convenient line for “rare”. There is nothing magical about 0.05: a p-value of 0.049 and a p-value of 0.051 are nearly identical evidence, even though one crosses the line and the other does not. Choose α before you collect the data (Step 1), and report the exact p-value so readers can judge for themselves.

Worked Example 1: One-Proportion Z-Test

The claim: a coin is fair, so the long-run proportion of heads is p0 = 0.5. You toss it 200 times and get 120 heads. Is that evidence the coin is biased?

Step 1: H0: p = 0.5; Ha: p ≠ 0.5 (two-tailed), α = 0.05. Step 2: the tosses are random and independent, and np0 = 100 and n(1 − p0) = 100 are both well above 10, so the normal approximation is fine.

Steps 3–4, computed with the one-proportion z-test calculator:

StepResult
Sample proportion p̂120 / 200 = 0.60
Standard error0.0354
Test statisticz = 2.83
p-value (two-tailed)0.0047
95% confidence interval[0.532, 0.668]
Decision at α = 0.05Reject H0

Step 5: the p-value 0.0047 is well below 0.05, so reject H0. In plain language: 120 heads in 200 tosses is strong evidence that the coin is not fair — and the confidence interval agrees, since the whole interval [0.532, 0.668] sits above 0.5.

Contrast: keep the same claim, but toss the coin 100 times and get 52 heads. The calculator gives z = 0.40 and p = 0.689, with a 95% confidence interval of [0.422, 0.618]. The same procedure, the opposite decision: 52 heads in 100 tosses is exactly the sort of result a fair coin produces, so we fail to reject H0. The p-value — not the headline number — decides.

Worked Example 2: One-Sample T-Test

The claim: a machine fills bottles to a mean of μ0 = 100 ml. A sample of 25 bottles has a mean of 105 ml with a standard deviation of 15 ml. Is the machine overfilling?

Step 1: H0: μ = 100; Ha: μ ≠ 100 (two-tailed), α = 0.05. Step 2: assume the sample is random and the fill volumes are roughly normal — with n = 25 the t-test needs that normality condition. The population standard deviation is unknown, which is why this is a t-test rather than a z-test.

Steps 3–4, computed with the One-Sample T-Test Calculator:

StepResult
Test statistict = 1.667
Degrees of freedom24
p-value (two-tailed)0.1086
Critical t (α = 0.05)2.064
Cohen’s d0.333
95% confidence interval[98.81, 111.19]
Decision at α = 0.05Fail to reject H0

Step 5: p = 0.1086 is above 0.05, and |t| = 1.667 is below the critical value 2.064, so fail to reject H0. Five ml above the target is not enough evidence — with a standard deviation of 15 ml and only 25 bottles, a sample mean of 105 ml is the kind of result a correctly set machine produces by chance. The confidence interval agrees: it covers 100.

Contrast: same machine, same spread (s = 15, n = 25), but a sample mean of 108.8 ml. Now the calculator gives t = 2.933, p = 0.0073, Cohen’s d = 0.587 and a 95% confidence interval of [102.61, 114.99]. The evidence crosses the line: reject H0 — the machine is overfilling. Same sample size, different sample mean; the evidence, not the procedure, decides.

Worked Example 3: Chi-Square Test

The chi-square test works on counts in categories, and the same χ² machinery answers two different questions: is there an association between two variables, and does a sample fit an expected distribution?

Test of independence: is there an association?

A survey asks 40 people whether they prefer tea or coffee, split by time of day: morning drinkers choose tea 10 times and coffee 20 times; evening drinkers choose tea 20 times and coffee 10 times — a 2×2 table of [[10, 20], [20, 10]]. H0: preference is independent of time of day; Ha: preference is associated with time of day. α = 0.05.

Steps 3–4, with the Chi-Square Test Calculator:

StepResult
χ²6.667
Degrees of freedom1
p-value0.0098
Yates’ corrected χ²5.400 (p = 0.0201)
Cramér’s V0.333
Decision at α = 0.05Reject H0

Step 5: p = 0.0098 is well below 0.05 — and even with Yates’ continuity correction (p = 0.0201) the result still rejects H0. There is evidence of an association between time of day and drink preference. Cramér’s V = 0.333 describes a moderate association.

Goodness of fit: does the sample match the expected counts?

A four-sided die is rolled 60 times and lands on 1, 2, 3 and 4 with counts [16, 18, 16, 10]. H0: each face is equally likely (15 expected per face); Ha: at least one face is not equally likely. α = 0.05.

The calculator gives χ² = 2.400 with 3 degrees of freedom and p = 0.4936 — so fail to reject H0. The observed counts are close to the 15-each expected counts, and a fair die produces a χ² of 2.400 or larger about 49% of the time. Same χ² machinery, different question, opposite verdict.

Type I and Type II Errors — and Power

Every decision in a hypothesis test can go wrong in two ways, and their names matter. A Type I error is a false positive: rejecting a true null hypothesis — convicting an innocent defendant. A Type II error is a false negative: failing to reject a false null hypothesis — letting a guilty defendant go free. The table shows all four combinations.

DecisionH0 is trueH0 is false
Reject H0Type I error (probability α)Correct decision (probability = power)
Fail to reject H0Correct decisionType II error (probability β)

The probability of a Type I error is α, which you set yourself in Step 1. The probability of a Type II error is β. Power = 1 − β is the probability of detecting a real effect when it exists. Bigger samples and bigger effects both raise power; a small sample with a small effect is the classic recipe for a Type II error. To make Type I errors rarer you lower α — but that makes Type II errors more likely. The two errors trade off against each other, and the only way to reduce both at once is more data.

Common Mistakes

  • Saying “accept H0”. A test can only reject or fail to reject the null hypothesis. “Accept” implies a proof the test cannot give.
  • Reading p as P(H0 is true). A p-value of 0.05 does not mean there is a 5% chance the null hypothesis is true; it is the probability of data this extreme if H0 were true.
  • Choosing one-tailed after seeing the data. Picking the tail that gives the smaller p-value after the results are in is p-hacking, not testing.
  • Treating α = 0.05 as a magic line. A p-value of 0.049 and one of 0.051 carry almost the same evidence; report the exact p-value.
  • Confusing statistical significance with practical importance. A tiny p-value from a huge sample can describe a trivial difference — report the effect size (Cohen’s d, Cramér’s V) alongside the p-value.

What is the difference between the null and alternative hypothesis?

The null hypothesis H0 is the “no effect” claim — no difference, no change, no association. The alternative hypothesis Ha is its complement: the claim that an effect, difference or association exists. The test assumes H0 is true and asks whether the data are surprising enough under that assumption to reject it; Ha is what you conclude when the evidence is strong enough.

What does it mean to fail to reject the null hypothesis?

It means the sample evidence is not strong enough to overturn H0 at the significance level you chose — not that H0 is true. The data were compatible with the null hypothesis, so the test does not convict. A larger sample, a bigger effect or a different test might reach a different verdict, which is why “fail to reject” is the careful wording and “accept” is not.

What is a Type I error?

A Type I error is rejecting a null hypothesis that is actually true — a false positive. Its probability is the significance level α: if you test at α = 0.05, then up to 5% of tests run on true null hypotheses will reject them by chance. To make Type I errors rarer you lower α, but that makes Type II errors (missing real effects) more likely — the two errors trade off against each other.

Frequently Asked Questions

Tap a question to see the answer.

Hypothesis testing is a statistical procedure that uses sample data to decide whether a claim about a population is believable. You assume the null hypothesis is true, then check how surprising your data would be under that assumption: if the p-value is smaller than your significance level, you reject the null hypothesis.

The null hypothesis (H0) says there is no effect, no difference or no association. The alternative hypothesis (Ha) is the opposite claim: that an effect, difference or association exists. You test the data against H0, and only conclude Ha when the evidence is strong enough.

A p-value of 0.05 means that if the null hypothesis were true, you would see data at least this extreme about 5% of the time. At a significance level of 0.05 it sits exactly on the decision line, so most researchers report the exact p-value instead of relying on the line.

It means the sample evidence was not strong enough to overturn the null hypothesis at the significance level you chose. It does not mean the null hypothesis is true — only that the test found no good reason to reject it.

In a two-tailed test the alternative hypothesis says the value is different from the null, in either direction, and the rejection region is split across both tails. In a one-tailed test the alternative says the value is bigger (or smaller), and all of the significance level sits in one tail. Choose the direction before you see the data, and use a two-tailed test if you are not sure.

Use a z-test for a proportion, or for a mean when the population standard deviation is known. Use a t-test for a mean when the standard deviation is estimated from the sample. With large samples the two give almost the same answer.

A Type I error is rejecting a null hypothesis that is actually true — a false positive. Its probability is the significance level α, so testing at α = 0.05 means up to 5 in 100 true null hypotheses will be rejected by chance.

No. A hypothesis test can reject the null hypothesis or fail to reject it, but it cannot prove either one true. Evidence can be strong enough to act on, but a test never turns a claim into a certainty.

Conclusion

Hypothesis testing turns a claim about a population into a decision with a known error rate. State the two hypotheses and your significance level first, check the conditions, compute the test statistic, read the p-value, and then decide — reject the null hypothesis only when the data are surprising enough, and never say “accept”. The three examples above show the same five steps producing three different decisions: strong evidence against a fair coin (p = 0.0047), not enough evidence against a machine set to 100 ml (p = 0.1086), and strong evidence of an association in a 2×2 table (p = 0.0098). The procedure is always the same; only the evidence changes.

Check Tools

One-Proportion Z-Test Calculator

Test a claimed proportion against sample data, with the z-score, p-value and confidence interval. Run the one-proportion z-test

One-Sample T-Test Calculator

Test a sample mean against a claimed value, with the t-statistic, p-value, Cohen’s d and confidence interval. Run the one-sample t-test

Chi-Square Test Calculator

Test independence in a contingency table or goodness of fit, with chi-square, p-value and Cramér’s V. Run the chi-square test

Explore All Statistical Tools

Browse the full set of calculators and visualisers on the site. Browse all the statistical tools