Compute-Optimal Quantization-Aware Training
Quantization-aware training (QAT) is a leading technique for improving the accuracy of quantized neural networks. Previ- ous work has shown that decomposing training into a full-precision (FP) phase followed by a QAT phase yields superior accuracy compared to QAT alone. However, the optimal allocation of compute between the FP and QAT phases remains unclear. We …