Skip to main content

rms_norm

Function rms_norm 

Source
pub fn rms_norm(x: &Tensor, weight: Option<&Tensor>, eps: f64) -> Result<Tensor>
Expand description

RMSNorm over the last dim: x / sqrt(mean(x^2) + eps) * weight (weight optional — some models fold the scale into a separate op and call this with None). Computes in x’s own dtype — cast beforehand if a caller needs a fixed compute precision regardless of input dtype.