Expand description
TabDPT inference engine — zero-shot forward pass only, single-pass (no ensembling; upstream
defaults to averaging 8 class-permuted passes, but n_ensembles=1 is a genuine, documented
mode in the reference implementation, not a shortcut — see classifier.py’s
predict()/ensemble_predict_proba() split).
Architecture: 32-layer transformer over [thinking rows][support rows][query rows]. Each
layer’s attention lets every position attend to the context (thinking + support) only, with
a per-layer y-embedding (a small MLP, re-run per layer) folded into V, RMSNorm’d Q/K, a
length-adaptive temperature scale, and a sigmoid output gate per head. One checkpoint serves
both tasks — the head produces max_num_classes + regression_bin_count outputs; classifier
reads the first n_classes, regressor reads the rest as a binned distribution over
[regression_bin_min, regression_bin_max].