Expand description
MOMENT-1-large inference engine.
Architecture: T5 encoder-only transformer with patch embeddings.
- RevIN normalization → patch embedding (value + position) → 24 T5 encoder blocks (RMSNorm + relative-bias MHA + gated-GELU FFN) → final RMSNorm → per-patch reconstruction head
- Zero-shot forecasting: iteratively predict 8 steps at a time using the reconstruction head applied to the last patch’s encoder output.