Aggregate participant metadata to course composition

Collapses a sdt_participants register to one row per course, keyed by the JSON file name so it joins straight onto the corpus, and carries the composition covariates a model can use.

Usage

sdt_course_composition(
  x,
  impute = c("flag", "median", "none"),
  min_participants = 3L
)

Arguments

x

A sdt_participants object from sdt_read_participants().

impute

How to handle a covariate a course has no observation for: "flag", "median" or "none". See the details.

min_participants

Below this many recorded participants a course is marked as too small to average.

Value

A sdt_course_composition object: a tibble with one row per course, including courses with no participants at all, which come through with has_participants = FALSE, n_participants = 0 and every covariate NA or imputed-and-flagged. A course whose sheet was not read carries sheet_read = FALSE and n_participants = NA, so a sheets = subset never masquerades as a set of cancelled courses.

Why course level, and what that costs

The register has no group, team or "Gruppe" column, so a participant cannot be placed in group A rather than group B. Composition is therefore a block-level covariate: in y ~ card + (1 | block) + (1 | group) it can compete with the session random effect and can never explain the team random effect. Every group inside one course gets the same row. Fitting on this table is a between-course regression with as many points as there are courses, and a covariate that barely varies between courses cannot explain anything at all — check the spread before using one.

The two halves of the missingness policy

Every covariate comes with three companions: n_<covariate>, the number of participants the course mean was taken over; n_imputed_<covariate>, how many of those numbers came from a participant-level fill rather than an answer somebody wrote down (only the academic ladder has one by default); and <covariate>_missing, whether the course value itself was filled. The two count columns always tell the truth, whatever impute is set to.

• impute = "flag" (the default) fills a missing covariate with the corpus-wide value — the pooled mean over all participants, not the mean of course means — and sets the indicator. A model can then carry "this was missing" as information instead of pretending it was observed.

• impute = "median" fills with the between-course median and does not set the indicator. It exists because a downstream pipeline can refuse a column that is perfectly collinear with has_participants, which is exactly what the indicator is when the only empty courses are the ones with no participants; dropping the indicator then buys rank instead of information. It is not the default because it makes an imputed value indistinguishable from an observed one, which is the thing this module exists to prevent. Only n_<covariate> still gives it away.

• impute = "none" leaves NA and still sets the indicator.

A course with fewer than min_participants recorded people is marked in below_min_participants. Its observed value is kept, not overwritten: the flag is what stops it being read as if it were solid, and n_participants is there to weight or filter on. A share computed from two people is not missing, it is noisy, and those are different problems.

See also

sdt_read_participants(), sdt_participant_summary().

Examples

if (FALSE) { # \dontrun{
p <- sdt_read_participants("TDSC_course_participant_relations.xlsx")
comp <- sdt_course_composition(p)
comp[, c("short_course_id", "n_participants", "share_consultant")]

# How much does each covariate actually vary between courses?
vapply(comp[, c("share_female", "share_consultant")],
       stats::sd, numeric(1), na.rm = TRUE)
} # }