Project layout

Maturity: implemented

This map names the current repository, not a proposed package tree.

rf-foundation-models/
  common/                       shared preprocessing and validation primitives
  configs/
    config.yaml                 Hydra composition root
    experiment/                 complete bounded experiment presets
    dataset/                    experiment-facing dataset selection
    datasets/registry.yaml      canonical dataset locations and readiness
    dataloader/ normalization/ patching/
    model/ objective/ optimizer/ training/ checkpoint/
    launch/                     provider launch configuration
  preprocessing/                dataset-specific canonicalization adapters
  src/rffm/
    applications/pretrain/      bounded application and attempt lifecycle
    cli/main.py                 pretraining command dispatcher
    data/                       canonical reader, registry, collation
    model/                      causal model contract and tiny implementation
    signal/                     normalization, patching, model inputs
    training/                   NPP, guarded step, diagnostics, checkpoints
    storage.py                  filesystem and mounted-GCS path resolution
    types.py                    semantic RF sample and batch records
  training/tests/               executable contracts for the bounded path
  infra/runtime/                training image and Gemini Enterprise Agent Platform submission adapter
  docs/preview/                 reader-facing Sphinx source

Import boundary

The repository has no root pyproject.toml and does not publish an rffm distribution. Training CI runs with PYTHONPATH=src:. and installs dependencies from analysis/eda/pyproject.toml. The training container sets the same source and repository paths explicitly.

Two command surfaces

The root ./rffm script dispatches the existing preprocessing workflow command from common.cli. Bounded pretraining uses the distinct Python application entry point python -m rffm.cli.main pretrain. Neither command is installed as a console script by this repository.

Stable and specific homes

  • Common canonical bundle, validation, hashing, and persistence concepts live in common/ because preprocessing and training both consume them.

  • Dataset-specific source parsing stays under preprocessing/.

  • Reusable learning primitives stay in the data, signal, model, and training packages.

  • Bounded application composition stays under applications/pretrain.

  • Gemini Enterprise Agent Platform resource and submission behavior stays under infra/runtime.

See Also