Which test should I use? Answer & find out.
Walk through the standard decision path — design first, then assumption checks, then the test. Every recommendation comes with assumptions, post-hoc, effect size, and a copyable APA template.
The universal 7-step spine
- 1Identify the designoutcome type · groups · paired?
- 2Descriptives + plotsn, mean, SD, boxplot, QQ
- 3Check assumptionsShapiro · Levene · Mauchly
- 4Choose the test pathparametric / Welch / nonparametric
- 5Omnibus → post-hocwith multiplicity correction
- 6Effect size + CId, g, η², ω², r, V
- 7Report & concludeAPA + plain language
The Universal Workflow · how to approach any problem
Seven steps, always in this order. Click each step to expand.
The golden rule: the test is chosen by the design and the data's behavior, not by which result you'd like. Fix α and the tail (one- vs. two-sided) before looking at p-values. One-tailed only when H₁ is directional a priori; then p₁ = p₂ / 2 if the effect is in the hypothesized direction.
Never skip steps 2–3. Descriptives and plots catch outliers, skew, and entry errors that p-values hide. An assumption check is itself a hypothesis test: p > .05 means "no evidence of violation," which is what licenses the parametric path.
Assumption checks — the gatekeepers
| Check | Test / tool | Rule of thumb |
|---|---|---|
| Normality (per group; on differences if paired) | Shapiro–Wilk + QQ-plot, |skew| < 1 | p > .05 → normal OK p ≤ .05 → nonparametric / transform |
| Equal variances (independent groups) | Levene's test | p > .05 → pooled test p ≤ .05 → Welch version |
| Sphericity (3+ repeated measures) | Mauchly's test | p > .05 → standard RM-ANOVA p ≤ .05 → Greenhouse–Geisser ε |
| Independence | The design, not a test | Same subject measured twice → paired branch, always |
| Expected counts (χ² tables) | All expected ≥ 5 | Any < 5 → Fisher's exact |
Robustness & sample-size notes
- CLT cushion: with n ≳ 30 per group, t/ANOVA tolerate mild non-normality; with small n Shapiro has low power — lean on QQ-plots.
- Welch by default is safe: when unsure about equal variances (esp. unequal n), Welch's t / Welch ANOVA protect Type I error at little power cost.
- Sensitivity analysis: a strong result should survive the robust alternative (Welch next to classic ANOVA; Mann–Whitney next to the t-test). Agreement = confidence.
- Outliers: investigate, don't auto-delete. If influential, report results with and without.
Decision Trees · click any test node for full details
The full maps: continuous outcomes first, then categorical data, relationships, and modeling.
Design question
Assumption check
Parametric test
Nonparametric alternative
Post-hoc / follow-up
A · Comparing a continuous outcome
1 group vs. known value μ₀
Then: d = (x̄ − μ₀)/s and the 95% CI for the mean
2 groups
Then: Cohen's d / Hedges' g + CI of the difference · tail fixed a priori
3+ groups
Omnibus significant → corrected post-hoc → effect size per pair
B · Categorical data, relationships & modeling
Categorical outcome (counts)
Effect size: φ (2×2) or Cramér's V · report % and counts
Relationship (2 continuous)
r² = variance explained · correlation ≠ causation
Prediction / adjustment
Group comparison + covariate → ANCOVA
Why never multiple t-tests instead of ANOVA? k groups → k(k−1)/2 comparisons; with 5 groups at α = .05 the family-wise error ≈ 1 − 0.95¹⁰ ≈ 40%. Run the omnibus first; only if it is significant, run corrected post-hocs. Two factors at once? → two-way ANOVA with interaction. Adjusting for a covariate? → ANCOVA.
Reference · post-hoc, effect sizes, APA & pitfalls
Everything needed to finish an analysis properly — copy the APA templates straight into your report.
Which post-hoc after which omnibus?
| Omnibus (significant) | Post-hoc | Why |
|---|---|---|
| One-way ANOVA (equal var.) | Tukey HSD | All pairwise, exact family-wise control |
| Welch ANOVA (unequal var.) | Games–Howell | No equal-variance assumption |
| RM-ANOVA | Paired t + Bonferroni (or Holm) | Within-subject pairs; pcorr = p × m |
| Kruskal–Wallis | Dunn + Bonferroni | Rank-based pairwise |
| Friedman | Wilcoxon + Bonferroni | Paired rank-based |
| vs. one control only | Dunnett | Fewer comparisons → more power |
| Planned contrasts (a priori) | Contrasts / Holm | Pre-specified, most powerful |
Effect sizes — magnitude guide
| Measure | Small | Medium | Large |
|---|---|---|---|
| Cohen's d / Hedges' g (g corrects small-n bias) | 0.2 | 0.5 | 0.8 |
| η² / ω² (ANOVA; ω² less biased) | .01 | .06 | .14 |
| r (also U/W-based r = Z/√n) | .10 | .30 | .50 |
| φ / Cramér's V (χ², df = 1) | .10 | .30 | .50 |
Parametric ↔ nonparametric map
| Design | Parametric | Nonparametric twin |
|---|---|---|
| 1 sample vs. μ₀ | One-sample t | Wilcoxon signed-rank |
| 2 independent groups | Student / Welch t | Mann–Whitney U |
| 2 paired measures | Paired t | Wilcoxon signed-rank |
| 3+ independent groups | One-way ANOVA | Kruskal–Wallis |
| 3+ repeated measures | RM-ANOVA | Friedman |
| Correlation | Pearson r | Spearman ρ / Kendall τ |
CI logic: a 95% CI for a difference that excludes 0 ⇔ p < .05 (two-tailed). Report the CI of the difference, not just each group's CI — overlapping group CIs do not prove "no difference."
APA-style templates — click to copy
Pre-submission pitfall checklist
Key statistics at a glance
| Test | Statistic | df |
|---|---|---|
| One-sample t | t = (x̄ − μ₀)/(s/√n) | n − 1 |
| Independent t | t = (x̄₁ − x̄₂)/SE_pooled | n₁ + n₂ − 2 |
| Paired t | t = d̄/(s_d/√n) | n − 1 |
| One-way ANOVA | F = MS_between/MS_within | k − 1, N − k |
| RM-ANOVA | F = MS_cond/MS_error | (k−1), (k−1)(n−1) |
| χ² | χ² = Σ (O − E)²/E | (r−1)(c−1) |
| Cohen's d | d = (x̄₁ − x̄₂)/SD_pooled | — |
| CI of difference | Δ ± t_crit × SE_diff | — |
Step-by-Step Theory · from the mean to regression
A guided path through the statistics that power every test — in learning order. Each step: what it is → the math → a worked example → a footnote. Most examples use one running dataset: x = {2, 4, 5, 5, 6, 8} (n = 6).
Theories & Notation · symbols, pronunciation, definitions, equations
Everything behind the tests: how to read and say every symbol, what each term precisely means, and the math that makes it work.
ΣSymbols — read them, say them
Greek letters and statistical notation, with plain-English pronunciation.
| Symbol | Name | Say it | What it means |
|---|
🗣Test & statistician names — pronunciation
Say these confidently in talks, meetings, and papers.
| Name | Say it | What it is |
|---|
📔Definitions — the precise meaning of every term
The precise interpretations that matter — especially for the p-value and the confidence interval.
🧮Equations — with every term explained
The formulas behind each step of the workflow, matching what the code computes.
💡The big ideas — why the machinery works
Five pieces of theory that connect everything above.
Python & R References · copy-paste for every step
Every snippet follows the standard workflow — same libraries (pandas · scipy.stats · pingouin) on the Python side, base R + car / effsize / rstatix / afex on the R side. Use the search box to jump to a test.