Week 4, Session 1 — Two-sample and paired t-tests

Course 1 — #courses

R. Heller

Note

Inference labs use the five-step template: Hypothesis → Visualise → Assumptions → Conduct → Conclude.

Learning objectives

  • Choose between Student’s and Welch’s t-test, and between paired and independent-samples tests.
  • Compute Cohen’s d and Hedges’ g as effect sizes.
  • Report a two-sample comparison with point estimate, CI, and effect size.

Prerequisites

Lab 3.4.

Background

The two-sample t-test compares the means of two independent groups. Student’s version assumes equal variances across the groups; Welch’s does not. In most applied settings Welch’s is the safer default — it behaves nearly as well as Student’s when variances are actually equal and much better when they are not. The t.test() default in R is Welch.

When the same subjects are measured twice, the samples are paired and the two measurements are correlated. Ignoring the pairing inflates the standard error and lowers power. Pairing is handled by computing the within-subject difference and running a one-sample t-test on it.

Effect sizes remove units from the comparison. Cohen’s d is the standardised mean difference: the mean difference divided by the pooled standard deviation. Hedges’ g applies a small-sample correction. Both are essential alongside a p-value — a large d at a small n will often produce a non-significant p, but the d is still a warning that a larger study might find a real effect.

Setup

1. Hypothesis

Scenario A (independent): mean systolic blood pressure in treatment vs placebo.

  • H0: μ_trt = μ_pla, H1: μ_trt ≠ μ_pla, α = 0.05.

Scenario B (paired): SBP before and after a four-week intervention in the same 30 patients.

  • H0: mean within-subject change = 0, H1: ≠ 0.

2. Visualise

3. Assumptions

Independent t: approximate normality of the sample means; Welch’s does not require equal variances. Paired t: approximate normality of the within-subject differences.

4. Conduct

Two-sample (Welch)


    Welch Two Sample t-test

data:  sbp by arm
t = -2.9095, df = 72.3, p-value = 0.004807
alternative hypothesis: true difference in means between group treatment and group placebo is not equal to 0
95 percent confidence interval:
 -14.209770  -2.655371
sample estimates:
mean in group treatment   mean in group placebo 
               134.5256                142.9581 

Two-sample Student (for comparison)


    Two Sample t-test

data:  sbp by arm
t = -2.9095, df = 78, p-value = 0.004715
alternative hypothesis: true difference in means between group treatment and group placebo is not equal to 0
95 percent confidence interval:
 -14.202597  -2.662545
sample estimates:
mean in group treatment   mean in group placebo 
               134.5256                142.9581 

Paired


    Paired t-test

data:  df_pair$post and df_pair$pre
t = -2.4747, df = 29, p-value = 0.01943
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
 -5.7941048 -0.5505107
sample estimates:
mean difference 
      -3.172308 

Effect size

$d_ind
Cohen's d |         95% CI
--------------------------
-0.65     | [-1.10, -0.20]

- Estimated using pooled SD.
$g_ind
Hedges' g |         95% CI
--------------------------
-0.64     | [-1.09, -0.20]

- Estimated using pooled SD.
$d_paired
Cohen's d |         95% CI
--------------------------
-0.45     | [-0.82, -0.07]

5. Concluding statement

Independent comparison. Mean SBP was 134.5 mmHg (SD 14.7) in the treatment arm and 143 (SD 11) in placebo. A Welch’s t-test gave t(72.3) = -2.91, p = 0.0048, mean difference (treatment − placebo) -8.4 mmHg (95% CI -14.2 to -2.7). Cohen’s d = -0.65 (Hedges’ g = -0.64).

Paired comparison. The within-subject change (post − pre) had mean -3.2 mmHg (SD 7); paired t-test t(29) = -2.47, p = 0.019, 95% CI -5.8 to -0.6.

The effect size is the number that generalises. A p-value tells you whether to reject; d and its CI tell you how much.

Common pitfalls

  • Running an independent-samples t on paired data and losing power.
  • Reporting Student’s t in the era of Welch; there is rarely a good reason.
  • Reporting p without d (or an equivalent effect size).
  • Confusing effect size with statistical significance.

Further reading

  • Delacre M et al. (2017). Why psychologists should by default use Welch’s t-test.
  • Cohen J. Statistical Power Analysis for the Behavioral Sciences.

Session info

R version 4.6.1 (2026-06-24)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 26.04.1 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so;  LAPACK version 3.12.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

time zone: UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
 [1] effectsize_1.0.3 lubridate_1.9.5  forcats_1.0.1    stringr_1.6.0   
 [5] dplyr_1.2.1      purrr_1.2.2      readr_2.2.0      tidyr_1.3.2     
 [9] tibble_3.3.1     ggplot2_4.0.3    tidyverse_2.0.0 

loaded via a namespace (and not attached):
 [1] generics_0.1.4     renv_1.3.1         stringi_1.8.9      lattice_0.22-9    
 [5] hms_1.1.4          digest_0.6.39      magrittr_2.0.5     evaluate_1.0.5    
 [9] grid_4.6.1         timechange_0.4.0   estimability_2.0.0 RColorBrewer_1.1-3
[13] mvtnorm_1.4-2      fastmap_1.2.0      jsonlite_2.0.0     scales_1.4.0      
[17] cli_3.6.6          rlang_1.3.0        withr_3.0.3        yaml_2.3.12       
[21] otel_0.2.0         tools_4.6.1        datawizard_1.4.1   tzdb_0.5.0        
[25] coda_0.19-4.1      bayestestR_0.20.0  vctrs_0.7.3        R6_2.6.1          
[29] lifecycle_1.0.5    emmeans_2.0.4      insight_1.6.1      pkgconfig_2.0.3   
[33] pillar_1.11.1      gtable_0.3.6       glue_1.8.1         xfun_0.62         
[37] tidyselect_1.2.1   parameters_0.30.0  knitr_1.52         farver_2.1.2      
[41] xtable_1.8-8       htmltools_0.5.9    labeling_0.4.3     rmarkdown_2.32    
[45] compiler_4.6.1     S7_0.2.2