A small FT-Transformer-shaped model for the workbench: one self-attention
block over per-feature embeddings, mean-pooled to a softmax. Registered as
the "transformer" engine in sdt_engines() and graded "closed", because
it decides in a representation it learned rather than in the columns it was
given, so a per-column attribution describes something it no longer reasons
in.
Usage
sdt_transformer(d_model = 16L, num_heads = 2L, epochs = 30L)
Arguments
- d_model
Embedding width per feature.
- num_heads
Attention heads.
- epochs
Training epochs.
Value
A parsnip model specification.
Details
On a corpus whose effective sample size is the number of distinct cards, this
is expected to match rather than beat a logistic regression, and to say so.
It becomes the right tool when the training set is expanded with simulated
cohorts.
Examples
if (FALSE) { # \dontrun{
sdt_transformer(d_model = 16, num_heads = 2)
} # }
