Models and objectives

Maturity: verified

RFFM keeps the model boundary separate from the training objective and optimizer step. A model transforms prepared inputs into outputs, an objective turns those outputs into a loss, and an optimizer step updates parameters while the caller retains ownership of the run lifecycle.

Model implementation

The shipped model is deliberately small. It exists to prove data, causal alignment, objective, optimizer, checkpoint, and execution contracts. It is not the Base or Large architecture from the research plan.

Model input and sequence

The current bounded Tiny/NPP path uses CausalPatchModel as its model contract. Its prepared sequence contains three masked physical-metadata positions followed by fixed I/Q patches:

[sample rate] [center frequency] [bandwidth] [patch 0] ... [patch N-1]

Given metadata (B, M) and patches (B, N, 2P), CausalPatchModel returns hidden states (B, M + N, D), the metadata prefix length, and a boolean mask aligned to the sequence. State i may depend only on valid positions through i.

The tiny_causal_iq registration constructs:

  • one scalar-to-token projection for metadata;

  • one 2P-to-token projection for I/Q patches;

  • learned absolute position embeddings;

  • a PyTorch transformer encoder driven by a causal mask;

  • GELU, a feed-forward width of 2D, zero dropout, and the configured number of layers and attention heads.

The committed preset uses P=64, D=64, two layers, four heads, and a maximum combined sequence length of 64. These are bounded-smoke choices, not a production model recommendation.

Next-patch prediction

NextPatchPredictionTask uses patch hidden state i to predict input patch i+1. Its compact GELU head maps D -> max(1, D/2) -> 2P.

Each target patch is normalized over valid I/Q elements only. The mean-squared error is averaged only where the source patch, target patch, and target element are valid. Partial-patch padding therefore contributes neither to target normalization nor loss.

Optimizer step

The shipped adamw registration constructs PyTorch AdamW with YAML-owned learning rate and weight decay. The reusable step checks finite input tensors, pre-step parameters, forward outputs and loss, gradients, and post-update parameters. A completed step returns scalar loss and valid target-patch count.

The caller owns iteration, global step, metrics, checkpoints, and failure artifacts.

See Also