Expand description
Lag-Llama inference on the Burn backend (Flex CPU).
Two-phase inference:
- Prefill: full forward pass over context tokens on Burn tensors, collects K/V cache per layer.
- Decode: per-step single-token forward pass with pure-Rust host
kernels (
rms_norm_raw,raw_gemv,rope_single_inplace,mha_decode_raw,silu_mlp_raw).
Weights load straight from GGUF via zsfm_burn::GgufWeights.