Skip to main content

q8_dst_or_fallback

Function q8_dst_or_fallback 

Source
pub fn q8_dst_or_fallback(
    shape: &[u64],
    out_dtype: GGMLType,
) -> (GGMLType, bool)
Expand description

Q8_0 block-quantization fallback: tensors whose innermost dim or total element count is not a multiple of 32 stay F32. Centralizes the innermost % 32 check duplicated in generic.rs, the Python write_generic_gguf, and every per-model convert.rs.