Skip to main content

Module infer

Module infer 

Source
Expand description

Toto-2 inference engine.

Unit-scaling (μP) inference notes:

  • uu.LinearF.linear(x, w, b) / sqrt(fan_in)
  • uu.LinearReadoutF.linear(x, w, b) / fan_in
  • U.siluF.silu(x) * 1.766782948 (unit-scaled)
  • τ-rule: residual_split is identity; residual_add(h,skip,τ) = h*(τ/d) + skip*(1/d) where d=√(1+τ²)
  • PerDimScale(q, w) = q * softplus(w) / log(2) — the 0.52103 unit-scaling factors cancel

Structs§

InferConfig
TotoModel
TotoModelBuilder
Fluent constructor for TotoModel: point it at a GGUF file, optionally layer on a parsed config.json and/or explicit InferConfig overrides, then call build.