Crate zsfm_nn 

Shared candle tensor primitives duplicated (byte-for-byte, or modulo a hardcoded-vs-parameterized constant) across the per-model infer/mod.rs files in crates/models/*. Extracted so the arithmetic lives in one place; every call site keeps producing bit-identical results to its former private copy.

Functions

layer_norm

Standard LayerNorm over the last dim: (x - mean) / sqrt(var + eps) * weight + bias.

linear

y = x @ w^T + b, flattening any leading dims of x into a batch dim so this works for rank-2 or higher inputs against a rank-2 weight [d_out, d_in].

linear_bias

linear_nobias

load_tensor

Dequantize a named GGUF tensor and cast it to dtype.

load_vec

Dequantize a named GGUF tensor to F32 and flatten it to a plain Vec<f32>.

load_weight

Load an F32 weight PyTorch stores as (d_out, d_in). Candle reverses the GGUF shape back to (d_out, d_in); the dim(0) check guards against a stray transposed store (the Q8_0 transpose trick some converters apply) by flipping the tensor back.

make_causal_mask

Additive causal mask of shape [1, 1, seq, seq]: 0.0 where num_masked <= k <= q, -inf elsewhere. num_masked lets a prefix of keys be masked out regardless of query position (used for left-padded contexts); pass 0 for a plain causal mask.

rms_norm

RMSNorm over the last dim: x / sqrt(mean(x^2) + eps) * weight (weight optional — some models fold the scale into a separate op and call this with None). Computes in x’s own dtype — cast beforehand if a caller needs a fixed compute precision regardless of input dtype.

swiglu_ffn

SwiGLU feed-forward: w2(silu(w1(x)) * w3(x)), all projections bias-free.

try_load_tensor

Same as load_tensor, but returns Ok(None) instead of erroring when the tensor is absent.