Which Statistical Test Should I Use?
Choosing the wrong test is the most common statistics mistake — a t-test on skewed data, a paired test on independent groups. This guide asks four questions about your data and points you to the right test, with a link to the free calculator for it, plus a quick-reference table for every common design.
This guide answers ONE question: given YOUR data, which test do you run? It does not teach the tests themselves — each result links to the tool and the explainer for that test. If you want the general framework first, read Hypothesis Testing Explained.
Which test should I use? The decision tree
Answer each question. The tree shows the test for your data — and a link to the free calculator for it.
1. What kind of data are you testing?
2. How many groups?
3. Are the two groups paired or independent?
4. Is the data roughly normal?
Not sure? Run the Shapiro-Wilk test — it returns W and a p-value.
4. Are the differences roughly normal?
Not sure? Run the Shapiro-Wilk test on the differences — it returns W and a p-value.
4. Are the groups roughly normal?
Not sure? Run the Shapiro-Wilk test on each group — it returns W and a p-value.
4. Are the groups roughly normal?
Not sure? Run the Shapiro-Wilk test on each group — it returns W and a p-value.
2. What does the table look like?
One-sample t-test
Compare one group’s mean with a known value. Use it when the data are roughly normal.
Sign test
Compare one group’s median with a known value when the data are clearly not normal. The sign test is coming in a later wave.
Paired t-test
Compare two paired means (same subjects twice). Use it when the differences are roughly normal.
Wilcoxon signed-rank test
Compare two paired groups by rank. Use it when the differences are clearly not normal.
Two-sample t-test
Compare two independent means. Use it when both groups are roughly normal.
Mann-Whitney U test
Compare two independent groups by rank. Use it when the groups are clearly not normal.
One-way ANOVA
Compare three or more independent means when the groups are roughly normal. One-way ANOVA is coming in Batch 3.
Kruskal-Wallis test
Compare three or more independent groups by rank when the groups are clearly not normal. The Kruskal-Wallis test is coming in a later wave.
Chi-square test
Test independence in a table. Use it when the expected counts are large enough (every expected count ≥ 5).
Fisher’s exact test
Test a 2×2 table exactly. Use it when any expected count is below 5.
Odds ratio
Measure how strong the association in a 2×2 table is — the odds ratio, with a confidence interval.
Correlation and linear regression
For two continuous variables: use correlation to measure the association, and linear regression to predict one from the other.
Try the correlation calculator → · Try the linear regression calculator →
Quick-reference table
Bookmark this table. Find your design in the first column, read the test in the second, and follow the link in the third to the free calculator.
| Your design | Test to run | Calculator |
|---|---|---|
| One sample, mean vs a known value, normal | One-sample t-test | One-sample t-test calculator |
| One sample — is it normal? | Shapiro-Wilk test | Shapiro-Wilk test calculator |
| One sample, median vs a known value / non-normal | Sign test | Coming in a later wave |
| Two groups, paired, differences normal | Paired t-test | Paired t-test calculator |
| Two groups, paired, differences non-normal | Wilcoxon signed-rank test | Wilcoxon signed-rank test calculator |
| Two groups, independent, both normal | Two-sample t-test | Two-sample t-test calculator |
| Two groups, independent, non-normal | Mann-Whitney U test | Mann-Whitney U test calculator |
| Three+ groups, normal | One-way ANOVA | Coming in Batch 3 |
| Three+ groups, non-normal | Kruskal-Wallis test | Coming in a later wave |
| Two proportions | Two-proportion z-test | Two-proportion z-test calculator |
| One proportion vs a value | One-proportion z-test | One-proportion z-test calculator |
| 2×2 table, expected counts ≥ 5 | Chi-square test | Chi-square test calculator |
| 2×2 table, any expected count < 5 | Fisher’s exact test | Fisher’s exact test calculator |
| 2×2 table — how strong is the association? | Odds ratio | Odds ratio calculator |
| Two continuous variables — association | Correlation | Correlation calculator |
| Two continuous variables — prediction | Linear regression | Linear regression calculator |
| Effect size for two means | Cohen’s d | Cohen’s d calculator |
| Confidence interval for a mean | CI from mean and SD | CI from mean and SD calculator |
| How large a sample do I need? | Sample size | Sample size calculator |
Worked routing: two clinics, one question
Two clinics compare recovery days after the same surgery. Clinic A (n = 8): 5, 6, 6, 7, 7, 8, 9, 12. Clinic B (n = 8): 7, 8, 9, 9, 10, 11, 12, 14. Which test do you run? Walk the tree:
1. What kind of data? Recovery days are a continuous measurement — so we go to question 2.
2. How many groups? Two — Clinic A and Clinic B — so we go to question 3.
3. Paired or independent? Different patients in each clinic, so the groups are independent — we go to question 4.
4. Are the data roughly normal? The Shapiro-Wilk test does not reject normality here (Clinic A: W = 0.9009, p = 0.2945; Clinic B: W = 0.9694, p = 0.8929). But with n = 8 the test has almost no power, and recovery days are counts with a long right tail (12 and 14), so the safe choice is the non-parametric branch: the Mann-Whitney U test.
The result: U = 12, exact two-tailed p = 0.0379 — reject H0 at α = 0.05. The rank-biserial is 0.625 (a large effect). P(A < B) = 0.1875 — a random Clinic A patient recovers faster than a random Clinic B patient only 18.75% of the time.
For contrast, the Welch t-test on the same data gives t = −2.2361, p = 0.0422 — the same conclusion here, but its p-value relies on a normality assumption these data cannot support. The tree sends small, skewed samples to the rank test even when the t-test would agree.
Wrong test, wrong answer
A paired test on independent groups. A paired test subtracts one group from the other pair by pair — but with different patients there are no pairs to subtract, so the differences it computes are meaningless and the p-value is nonsense.
A t-test on skewed data. A t-test compares means, and a mean is pulled by the tail. With a long right tail (a few very long recoveries), the mean overstates the typical patient and the t-test can miss a real difference — or find one that is only in the tail. Rank tests compare the whole distribution, so the tail cannot hijack them.
Frequently Asked Questions
What if my data are neither clearly normal nor clearly non-normal?
Run the Shapiro-Wilk test and read the Q-Q plot alongside the p-value. If the test does not reject and the points sit close to the line, use the parametric test. If you are still unsure, use the rank test — it costs little power when the data are normal, and it protects you when they are not.
What if my groups have very different sizes?
Unequal group sizes are fine for the tests in this guide — the two-sample t-test, the Mann-Whitney U test and the chi-square test all handle them. What matters is that each group is large enough to judge its shape, and that the smaller group is not so small that a single value dominates it. If one group has fewer than about 10 values, lean on the rank test.
Can I use a t-test if n is large (n ≥ 30)?
Often, yes. The central limit theorem says the sampling distribution of the mean becomes approximately normal as n grows, so a t-test on a large sample is usually safe even if the data themselves are skewed. But “large” depends on how skewed the data are — with a very long tail, even n = 50 can mislead. If the data are heavily skewed, the rank test is still the safer choice.
What if I have three groups but only two matter?
Compare just the two groups that matter, and ignore the third — the test for two groups is the right one. But if you picked those two groups because they looked the most different, you are cherry-picking, and the p-value will be too small. Decide which groups to compare before you look at the results.
Which test do I use for a before-and-after study?
A before-and-after study is paired data — the same subjects measured twice. If the differences are roughly normal, use the paired t-test. If they are clearly not normal, use the Wilcoxon signed-rank test. The tree’s question 3 (“paired or independent?”) sends you to the right branch.
What if my outcome is yes/no, not a measurement?
Yes/no outcomes are counts in categories, so you use the table branch of the tree. For a 2×2 table with every expected count at least 5, use the chi-square test. If any expected count is below 5, use Fisher’s exact test. To measure how strong the association is, use the odds ratio calculator.
Check Tools
Try the numbers from this guide with the site’s free calculators:
Paired t-Test Calculator
The test for before-and-after data on the same subjects.
Try the paired t-test calculator →
Two-Sample t-Test Calculator
The test for two independent groups.
Try the two-sample t-test calculator →
Shapiro-Wilk Test Calculator
Check normality before you choose a parametric test.
Try the Shapiro-Wilk test calculator →
Explore All Statistical Tools
See every calculator on the site in one place.
Browse all our statistical tools →
Check Articles
Go deeper with these guides:
Hypothesis Testing Explained
The five steps of a hypothesis test, with worked z, t and chi-square examples.
Read the guide →
Z-Test vs t-Test
When to use a z-test and when to use a t-test.
Read the guide →
Paired vs Unpaired t-Test
Same subjects twice, or two different groups?
Read the guide →
What Is a P-Value?
What a p-value really means — and what it does not.
Read the guide →