Expand description
Lag-Llama inference engine with KV caching.
Two-phase inference:
- Prefill: full forward pass over context tokens, collects K/V cache per layer.
- Decode: per-step single-token forward pass in pure Rust (zero Candle overhead).
Lag-Llama inference engine with KV caching.
Two-phase inference: