Lab W2 · Next-word frequencies

Teaching unit 05: a tiny next-word model built from twelve synthetic sentences shows how a fluent, plausible and false sentence can become the most probable output.
Author

R. Heller

Workshop: W2 · Language models and scientific evidence · Teaching unit: 05 · Data: twelve synthetic teaching sentences · Script: r/w2-next-word.R (runs on its own in a fresh R session) · Tables: w2-corpus.csv, w2-bigrams.csv · Run time: about 1 second · No external AI account needed

Large language models are trained on enormous text collections with neural networks, not with a table of word pairs. The principle of generation is nevertheless the same: given the text so far, the model assigns a probability to every possible next piece of text (a token) and picks one. Nothing in this step checks whether the resulting sentence is true. This lab makes that principle visible with a model small enough to count by hand.

library(ggplot2)

Part A · Count which word follows which

Decision. You will later judge a generated answer to a patient’s question. First you need your own explanation of how such an answer comes about.

Look at the data. The corpus has twelve short sentences. Each is correct in its own context: a CT scan can exclude a fracture, an alert can miss one, and a negative alert does not exclude one.

# Twelve synthetic teaching sentences. Each one is true in its own context.
corpus <- c(
  "a negative alert does not exclude a fracture",
  "a careful review can exclude a fracture",
  "a ct scan can exclude a fracture in most patients",
  "an mri can exclude a fracture",
  "the alert can help the team",
  "the alert can miss a fracture",
  "the alert can be wrong",
  "the alert suggests a second look",
  "the alert suggests a fracture",
  "a second look can change the decision",
  "the team reviews the image again",
  "a fracture needs a splint"
)
# Split into words (tokens) and mark sentence start <s> and end </s>
tokens <- lapply(strsplit(corpus, " "), function(w) c("<s>", w, "</s>"))
length(unlist(tokens))
#> [1] 101

The model counts every pair of neighbouring words (a bigram). Before running the next chunk, find all words that follow “alert” and all words that follow “can” in the twelve sentences and count them.

# Count every pair of neighbouring words (bigram)
pairs <- do.call(rbind, lapply(tokens, function(w) {
  data.frame(word = head(w, -1), next_word = tail(w, -1))
}))
bigrams <- as.data.frame(table(word = pairs$word, next_word = pairs$next_word), stringsAsFactors = FALSE)
bigrams <- bigrams[bigrams$Freq > 0, ]
names(bigrams)[3] <- "count"
# What follows "alert", and what follows "can"?
bigrams[bigrams$word == "alert", ]
#>       word next_word count
#> 208  alert       can     3
#> 378  alert      does     1
#> 1024 alert  suggests     2
bigrams[bigrams$word == "can", ]
#>     word next_word count
#> 177  can        be     1
#> 279  can    change     1
#> 415  can   exclude     3
#> 483  can      help     1
#> 619  can      miss     1
stopifnot(
  sum(bigrams$count[bigrams$word == "alert"]) == 6,
  bigrams$count[bigrams$word == "alert" & bigrams$next_word == "can"] == 3,
  bigrams$count[bigrams$word == "alert" & bigrams$next_word == "does"] == 1,
  bigrams$count[bigrams$word == "can" & bigrams$next_word == "exclude"] == 3
)

Assumption. The model knows nothing but these counts. The probability of the next word is its share among all words that followed the current word.

# Probability of the next word given the current word, with a temperature T:
# counts are raised to the power 1/T and renormalised (T < 1 sharpens, T > 1 flattens)
next_prob <- function(word, temperature = 1) {
  b <- bigrams[bigrams$word == word, ]
  w <- b$count^(1 / temperature)
  setNames(w / sum(w), b$next_word)
}
round(next_prob("alert"), 3)
#>      can     does suggests 
#>    0.500    0.167    0.333
stopifnot(isTRUE(all.equal(next_prob("alert")[["can"]], 3 / 6)),
          isTRUE(all.equal(next_prob("alert")[["does"]], 1 / 6)))

Part B · Temperature

The temperature reshapes these probabilities before sampling. A low temperature makes the most frequent continuation even more likely; a high temperature gives rare continuations more chance. Temperature does not add knowledge; it only changes how often the model leaves the most frequent path.

temp_df <- do.call(rbind, lapply(c(0.5, 1, 2), function(t) {
  p <- next_prob("alert", t)
  data.frame(temperature = paste("T =", t), next_word = names(p), probability = unname(p))
}))
temp_df$next_word <- factor(temp_df$next_word, levels = c("can", "suggests", "does"))
p_temp <- ggplot(temp_df, aes(x = next_word, y = probability, fill = temperature)) +
  geom_col(position = position_dodge(width = 0.75), width = 0.7) +
  geom_text(aes(label = sprintf("%.2f", probability)), position = position_dodge(width = 0.75),
            vjust = -0.3, size = 3.2) +
  scale_fill_manual(values = unname(cdsr_colours[c("blue", "orange", "grey")])) +
  scale_y_continuous(limits = c(0, 0.85)) +
  labs(x = "Next word after \"alert\"", y = "Probability", fill = "Temperature",
       caption = "Bigram model from 12 synthetic teaching sentences; 6 occurrences of \"alert\" followed by another word.") +
  theme_cdsr()
show_figure(p_temp, "w2-temperature.png", height = 3.8)
Grouped bar chart. At temperature 0.5 the probabilities are 0.64 for 'can', 0.29 for 'suggests' and 0.07 for 'does'. At temperature 1 they are 0.50, 0.33 and 0.17. At temperature 2 they are 0.42, 0.34 and 0.24.
Figure 1: Probability of the word that follows “alert” in the synthetic bigram model at three temperatures. Counts: can 3, suggests 2, does 1 (6 occurrences in 12 sentences).

Part C · A plausible sentence that is false

Now compare two continuations of the start “a negative alert”. The first is the sentence in the corpus; the second was never written by anyone.

# Probability of a whole continuation given its first words: the product of next-word probabilities
sentence_prob <- function(sentence, given = 3) {
  w <- c("<s>", strsplit(sentence, " ")[[1]], "</s>")
  steps <- seq(given + 1, length(w) - 1)   # condition on the first `given` words of the sentence
  prod(sapply(steps, function(i) {
    p <- next_prob(w[i])
    if (w[i + 1] %in% names(p)) p[[w[i + 1]]] else 0
  }))
}
true_sentence  <- "a negative alert does not exclude a fracture"
false_sentence <- "a negative alert can exclude a fracture"
probs <- c(true = sentence_prob(true_sentence), false = sentence_prob(false_sentence))
round(probs, 4)
#>   true  false 
#> 0.0641 0.0824
# Target value: the false continuation is more probable than the true one
stopifnot(probs[["false"]] > probs[["true"]],
          !false_sentence %in% corpus)   # ... although it never occurs in the corpus

Report. Under this model the false continuation “a negative alert can exclude a fracture” has a probability of 0.082, the true one only 0.064. Every single step of the false sentence is attested in the corpus (“alert can”, “can exclude”, “exclude a”, “a fracture”); the sentence as a whole contradicts the corpus. Fluency is a property of word sequences, truth a property of claims.

TipCounter-check: predict first

If you always choose the most probable next word (greedy decoding, roughly temperature close to zero), which of the two sentences will the model write? And how many of 20 random continuations at temperature 1 will reproduce the true corpus sentence word for word: none, about five, or about ten?

# Continue "a negative alert" word by word; greedy = always the most probable next word
generate <- function(start = c("<s>", "a", "negative", "alert"), temperature = 1, greedy = FALSE,
                     max_words = 12) {
  w <- start
  while (tail(w, 1) != "</s>" && length(w) < max_words + 1) {
    p <- next_prob(tail(w, 1), temperature)
    w <- c(w, if (greedy) names(p)[which.max(p)] else sample(names(p), 1, prob = p))
  }
  paste(w[!w %in% c("<s>", "</s>")], collapse = " ")
}
greedy_text <- generate(greedy = TRUE)
greedy_text
#> [1] "a negative alert can exclude a fracture"
set.seed(2026)
samples <- vapply(1:20, function(i) generate(temperature = 1), character(1))
sample_table <- sort(table(samples), decreasing = TRUE)
sample_table
#> samples
#>                         a negative alert can exclude a fracture 
#>                                                               5 
#>                            a negative alert suggests a fracture 
#>                                                               4 
#>           a negative alert suggests a fracture in most patients 
#>                                                               2 
#>                                   a negative alert can be wrong 
#>                                                               1 
#>      a negative alert can help the alert suggests a ct scan can 
#>                                                               1 
#>                            a negative alert can miss a fracture 
#>                                                               1 
#> a negative alert does not exclude a careful review can be wrong 
#>                                                               1 
#>   a negative alert does not exclude a fracture in most patients 
#>                                                               1 
#>  a negative alert does not exclude a second look can change the 
#>                                                               1 
#>    a negative alert suggests a ct scan can change the alert can 
#>                                                               1 
#>                         a negative alert suggests a second look 
#>                                                               1 
#>     a negative alert suggests a second look can miss a fracture 
#>                                                               1
stopifnot(greedy_text == false_sentence,
          sum(samples == false_sentence) == 5,     # the false sentence, word for word, 5 times in 20
          sum(samples == true_sentence) == 0)      # the true sentence never appears complete

The greedy continuation is the false sentence. Among 20 random continuations, the false sentence appears 5 times; the true sentence never appears in full, and several outputs drift into fluent nonsense.

WarningCommon mistake

Concluding that a more confident (lower-temperature) answer is more likely to be correct. Lowering the temperature makes the model repeat its most probable path more reliably, including a wrong one.

Part D · Back to the two stored answers

Return to the two stored answers to the patient’s question on the workshop page. For the sentence you underlined as most contentious:

  1. Write one sentence in your own words explaining how a language model could produce it without any source that says so.
  2. Write a verification task for it: which source, which passage, and what would count as confirmation or refutation? This becomes your evidence card in unit 06.

Your record for the workshop

  • Your explanation of the generation principle (two to three sentences, no jargon).
  • The verification task for the contentious sentence.

Further reading