ANOVA vs t-Test: When to Use Each One

ANOVA and a t-test both compare means, but they are not interchangeable: a t-test handles exactly two groups, while ANOVA handles three or more — and running many t-tests inflates the false-positive rate to 40% with ten groups. This guide shows the 14.3% trap, proves F equals t squared, and works a full one-way ANOVA with Tukey post-hoc tests.

The short version: two groups, use a t-test. Three or more groups, use ANOVA. Running a t-test for every pair instead of one ANOVA does not keep your error rate at 5% — it inflates it, to 14.3% with three groups and 40% with ten.

This guide explains what each test does, shows the arithmetic behind the false-positive trap, proves that ANOVA with two groups is the t-test squared, and works a full one-way ANOVA — sums of squares, the F-statistic, the p-value and Tukey post-hoc tests — on real numbers.

What ANOVA does

ANOVA — analysis of variance — tests whether the means of three or more groups are all equal. The null hypothesis is that every group mean is the same; the alternative is that at least one mean differs. It answers that question by comparing two sources of variation: the variation between the group means (how far apart the groups sit) and the variation within the groups (how spread out the values are inside each group). If the between-group variation is large compared with the within-group variation, the groups are genuinely different — the F-statistic is large and the p-value is small.

The point of ANOVA is that it asks one question about all the groups at once, and answers it with a single test at your chosen α. That is why it exists: it is the correct test when you have three or more groups, because it keeps the false-positive rate at α instead of letting it inflate across many pairwise tests. The method follows the NIST/SEMATECH e-Handbook, 1.3.5.4. One-Factor ANOVA.

What a t-test does

A t-test compares exactly two means. The one-sample t-test compares one mean with a claimed value; the two-sample t-test compares two independent groups; the paired t-test compares two measurements on the same subjects. All three are t-tests, and all three handle exactly two things. For two groups, a t-test is the right tool — and, as the next sections show, ANOVA with two groups gives exactly the same answer, squared.

If you are choosing between a z-test and a t-test, or between a paired and an unpaired t-test, our guides z-test vs t-test and paired vs unpaired t-test give the decision rules. The five steps every hypothesis test follows are in hypothesis testing explained.

The 14.3% trap

Here is the trap. Suppose you have three groups and you run a t-test for every pair — A against B, A against C, B against C — each at α = 0.05. Each test has a 5% chance of a false positive. But the chance that at least one of the three tests gives a false positive is not 5%. It is much higher, because you get three chances to be wrong.

For k independent comparisons at level α, the family-wise error rate — the chance of at least one false positive — is 1 − (1 − α)ᵏ. At α = 0.05:

Pairwise t-tests (k)GroupsFamily-wise error rate
33 groups1 − 0.95³ = 0.1426 = 14.26% (about 14.3%)
64 groups1 − 0.95⁶ = 0.2649 = 26.49% (about 26.5%)
105 groups1 − 0.95¹⁰ = 0.4013 = 40.13% (about 40.1%)

Three groups need 3 pairwise t-tests. Four groups need 6. Five groups need 10. Ten groups need 45 — ten times nine, divided by two — and the error rate keeps climbing with every test you add. Running t-tests instead of ANOVA means the “5% error rate” is a fiction: with five groups you already have a 40% chance of declaring a difference that is not there. That is the trap, and it is why ANOVA exists.

ANOVA avoids it by running one test at α = 0.05 for all the groups at once, so the family-wise error rate stays at 5%. If that one test is significant, the post-hoc tests — Tukey HSD, covered below — then control the error rate across the pairwise comparisons. The arithmetic is the same whether the p-value comes from a t-test or an F-test; if you are unsure what a p-value means, read what is a p-value?, and the five steps of the test are in hypothesis testing explained.

Family-wise error rate: 1 − (1 − α)k at α = 0.05

14.3% 26.5% 40.1% 3 comparisons 6 comparisons 10 comparisons 0% 10% 20% 30% 40% α = 5% — one ANOVA
3 comparisons: 14.3% · 6 comparisons: 26.5% · 10 comparisons: 40.1% — one ANOVA keeps it at 5%.

Each bar is the chance of at least one false positive when you run that many t-tests at α = 0.05 each.

F = t²

For exactly two groups, ANOVA and the t-test are the same test. The F-statistic with (1, df) degrees of freedom equals the square of the t-statistic with df degrees of freedom — F(1, df) = t(df)² — and the p-values are identical. ANOVA with two groups is the t-test, squared.

Take t(18) = 2.10. Square it: F(1, 18) = 2.10² = 4.41. The t-test’s two-tailed p is 0.0501; the F-test’s p is 0.0501. Same p, same conclusion, same data. There is no extra information in running ANOVA on two groups, and no extra information in running a t-test when ANOVA would do — the tests agree, because they are the same test in different clothes. The choice between them is about how many groups you have, not about which is “better”. Critical F values are tabulated in the NIST/SEMATECH e-Handbook, 1.3.6.7.3. Upper Critical Values of the F Distribution; for two groups, the two-sample t-test calculator gives the same answer either way.

Worked example: one-way ANOVA

Three groups of five. Group 1: [6, 8, 7, 9, 5], mean 7.00. Group 2: [9, 11, 10, 8, 12], mean 10.00. Group 3: [7, 6, 8, 5, 9], mean 7.00. The grand mean is 8.00.

SourceSSdfMSFp
Between groups30.00215.006.000.0156
Within groups30.00122.50  
Total60.0014   

How the numbers are built: SSB = 5 × [(7 − 8)² + (10 − 8)² + (7 − 8)²] = 5 × (1 + 4 + 1) = 30.00. SSW = 30.00 is the variation inside the groups. SST = SSB + SSW = 60.00. MSB = 30.00 / 2 = 15.00; MSW = 30.00 / 12 = 2.50. F = MSB / MSW = 15.00 / 2.50 = 6.0000, with p = 0.0156. Since 0.0156 < 0.05, reject H₀ — the group means are not all equal. And η² = SSB / SST = 30.00 / 60.00 = 0.5000: half the total variation is between groups.

The same F-test appears in regression: the ANOVA table of a fitted line tests whether the slope differs from zero. Our linear regression calculator shows that table, and linear regression explained walks through slope, intercept and R².

Three groups, one grand mean — where the variation lives

Group 1 Group 2 Group 3 Grand mean 8.00 5 6 7 8 9 10 11 12

● Observation    ◆ Group mean    Grand mean

Group means 7.00 · 10.00 · 7.00 · Grand mean 8.00 — the means differ (between-group), the dots spread around them (within-group).

Each dot is one observation. The red diamonds are the group means; the dashed line is the grand mean. ANOVA compares how far apart the diamonds are with how spread out the dots are.

After a significant F: Tukey HSD

A significant F says “somewhere there is a difference” — it does not say where. To find which pairs differ, use a post-hoc test. Tukey’s HSD (honestly significant difference) compares every pair while controlling the family-wise error rate, so the trap from the last section does not come back through the back door.

For the worked example, the Tukey critical value is q = 3.7729 (k = 3 groups, df = 12, α = 0.05), and HSD = q × √(MSW / n) = 3.7729 × √(2.50 / 5) = 3.7729 × √0.5 = 2.6678. Any pair whose means differ by more than 2.6678 is significantly different.

PairDifferenceVerdict
Group 2 − Group 110.00 − 7.00 = 3.003.00 > 2.6678 — significant
Group 2 − Group 310.00 − 7.00 = 3.003.00 > 2.6678 — significant
Group 1 − Group 37.00 − 7.00 = 00 < 2.6678 — not significant

So the F says “somewhere”; Tukey says where: group 2 differs from groups 1 and 3, and groups 1 and 3 do not differ from each other. Only run post-hoc tests after a significant F, and only if you need to know which pairs differ. If the F is not significant, stop — there is no difference to locate.

When a t-test is still right

ANOVA is the right test for three or more groups, but a t-test is still right when:

Exactly two groups. ANOVA adds nothing — F = t², and the p-values are identical. Use the two-sample t-test calculator (Welch or pooled) or the paired t-test calculator for before-and-after data.

Planned contrasts specified before seeing the data. If you decided in advance to compare two specific groups — one comparison, stated before you looked — a single t-test at α = 0.05 is valid, because you are running one test, not many.

The question is a single pre-chosen pair, not “which pairs differ”. If you only ever cared about A versus B, test A versus B. ANOVA and Tukey are for when you want to compare everything and find out where the differences are.

Decision table

Choose by the number of groups and the question you are asking:

ScenarioUse
Exactly two independent groupsTwo-sample t-test — or ANOVA, the same answer (F = t²)
Two measurements on the same subjectsPaired t-test
Three or more independent groupsOne-way ANOVA
Three or more groups, then which pairs differOne-way ANOVA + Tukey HSD
Two groups, variances unequalWelch’s t-test (the two-sample t-test calculator offers Welch)
Three or more groups, variances unequalWelch’s ANOVA — the one-way ANOVA tool will cover it when it lands
Comparing a regression slope to zeroLinear regression — its ANOVA table is the same F-test

In short: two groups, a t-test; three or more, ANOVA; and if ANOVA is significant and you need to know where, Tukey. The decision rules for the two-group cases are in z-test vs t-test and paired vs unpaired t-test.

Frequently Asked Questions

Can I run ANOVA on two groups?

Yes — it is the t-test, squared. For exactly two groups, F(1, df) = t(df)² and the p-values are identical. There is no reason to prefer ANOVA for two groups, and no reason to avoid it either; the answer is the same. Use whichever is convenient — the two-sample t-test calculator gives it directly.

Why not just run three t-tests?

The 14.3% trap. At α = 0.05 each, three independent t-tests give a family-wise error rate of 1 − 0.95³ = 14.26% — nearly three times the 5% you think you have. With ten comparisons (five groups) it is 40.1%. One ANOVA keeps the error rate at 5%.

What does a significant F actually tell me?

That at least one group mean differs from the others — “somewhere, not where”. It does not tell you which pairs differ. To find that out, run a post-hoc test such as Tukey HSD, as in the worked example above.

What if my variances are unequal?

Use Welch’s ANOVA, which does not assume equal variances. The one-way ANOVA tool will cover it when it lands. For two groups, the two-sample t-test calculator already offers Welch’s t-test.

What is the difference between ANOVA and regression?

The regression ANOVA table is the same F-test on the fitted line. A one-way ANOVA is a regression with the group as a categorical predictor; the F in the regression ANOVA table tests whether the group means differ. See linear regression explained for slope, intercept and R².

Do I need a post-hoc test every time?

Only after a significant F, and only if you need to know which pairs differ. If the F is not significant, stop — there is no difference to locate.

What is eta-squared?

η² = SSB / SST. It is the share of the total variation that is between groups — the effect size for ANOVA. In the worked example η² = 30.00 / 60.00 = 0.50, so half the total variation is between groups.

Check Tools

Try the numbers from this guide with the site’s free calculators:
One-Sample T-Test Calculator
Test one mean against a claimed value with the t-distribution.
Try the One-Sample T-Test Calculator →
Two-Sample t-Test Calculator
Compare two independent groups with Welch or pooled t intervals.
Try the Two-Sample t-Test Calculator →
Linear Regression Calculator
Fit a line, read the slope and R² — with a full ANOVA table.
Try the Linear Regression Calculator →
Explore All Statistical Tools
See every calculator on the site in one place.
Browse all our statistical tools →

Check Articles

Go deeper with these guides:
Hypothesis Testing Explained
The five steps of a hypothesis test, with worked z, t and chi-square examples.
Read the guide →
Linear Regression Explained
Slope, intercept and R² — the regression behind the ANOVA table.
Read the guide →
Z-Test vs t-Test
When to use a z-test and when to use a t-test — sigma known or not.
Read the guide →
Paired vs Unpaired t-Test
The same dataset through both tests — paired p < 0.0001, unpaired p = 0.1558.
Read the guide →