W4 · What imaging, gait and tissue models measure

Workshop W4, teaching units 13–16: a spectral index under changing light, image split against animal split, elective stations on gait and tissue, and a first model passport with a decision on permissible exports.
Author

R. Heller

Part II · Testing and comparing models · Level: application · Teaching units: 13–16 (4 × 45 min) · Required reading: core readings of B10 and B19 (about 20 minutes) · Lab: Lab W4

Guiding question

Does our test measure the recognition of individuals the model has already seen, or generalisation to new ones, and which measurement error could change the conclusion?

Learning objectives

By the end of this workshop you can

  1. build a simple spectral index from synthetic hyperspectral data, show which change of illumination it is robust to and propose a reference measurement;
  2. split repeated measurements once by image and once by animal, verify in R that the animal split shares no identifiers, and state which target claim each split supports;
  3. report a performance figure with the independent denominator (animals or people, not spectra or steps);
  4. for one elective station, show how a change of camera perspective or of a segmentation threshold changes a measured quantity, with unit and independent sample size;
  5. judge whether a gait video or a tissue export could identify a person, and decide which export is permissible for the stated purpose.

Preparation

Teaching units

Unit Task Minutes Observable evidence
13 From synthetic hyperspectral data, build a simple spectral index, introduce a change of illumination and propose a reference measurement (Lab W4, Part A). 45 A spectral figure, a baseline and one clearly named confounder.
14 Split several recordings per animal once by image and once by animal. Check the overlap of identifiers in R and compare the target claims (Part B). 45 A test protocol with genuinely new individuals, a comparison model and the limit of generalisation.
15 Elective station gait: angle and method differences under a change of perspective. Elective station tissue: shift a segmentation threshold, measure cell distances and summarise per person (Part C). 45 One figure per station with unit, independent sample size and a realistic consequence of the measurement error.
16 Teams review each other’s findings. For gait video, discuss re-identification by movement or metadata; for tissue images, inference from rare combinations of features. 45 A first model passport, a counter-check and a reasoned decision on permissible exports.

The animal split in four lines (unit 14)

Four synthetic animals provide three spectra each; two animals belong to class A, two to class B. As a deliberately trivial comparison model, the class already seen for a known animal is returned; for an unknown animal the pre-specified default class A applies.

Table 1: Two splits of the same twelve synthetic spectra.
Test target Split Observation
Another spectrum of a known animal Two spectra per animal in training, one in the test Looking up the known class solves 4/4 cases. This measures recognition.
Spectra of two new animals Two animals in training; in the test one new A and one new B animal with three spectra each The default class solves 3/6 spectra, or 1/2 animals. The independent denominator is two animals.

In the lab you check with intersect(unique(train$id), unique(test$id)) whether identifiers overlap. For the second split you expect an empty intersection.

Review questions for unit 16

  • Gait video. Could a person be recognised from their walking pattern, their clothing, the room or the file’s metadata (date, device, location)? Which derived quantity (for example joint angles per step) would answer the clinical question without the raw video? Who decides how long the raw video is kept?
  • Tissue images. Could a rare combination of features (a rare diagnosis, an unusual age, a small centre) point to a single person? Which aggregate (per person, per group) is enough for the research claim?
  • Both. Which export would you permit for the stated purpose, to whom, and with which deletion point?

Materials

Four perspectives in this workshop

  • Ethics. All animals are fictitious. For a real experiment, the split decides how many animals a claim rests on; an inflated denominator can justify further experiments that are not needed (3Rs: Replacement, Reduction, Refinement; see NC3Rs).
  • Regulation. A tool that is tested only on known animals has no evidence for a new species or for humans; a claim about patients needs a study in the target population.
  • Data protection. Gait videos and tissue images can identify people even without names. Units 15 and 16 separate what the research question needs from what is merely available.
  • Security. An export that leaves the protected environment is a copy; fewer and smaller exports mean fewer copies to protect.

Transition to W5

You now have a comparison table, an error pattern and a first model passport. In W5 the question shifts from “does it work?” to “why does the pattern arise, and which assumption matters most?”

Further reading

Sources