Train Smarter, Not Harder: Switching Signal-Guided Training in Active Learning
Abstract
HybridAL adaptively switches from retraining to fine-tuning during active learning based on online stabilization signals, reducing training time while preserving accuracy and improving calibration.
Training strategy, namely whether to retrain from scratch or fine-tune from the previous checkpoint, is an overlooked decision variable in active learning. We show that this choice has exploitable structure: retraining is most useful in early rounds, when each batch can substantially reshape the labeled distribution, while fine-tuning becomes safer once the model trajectory stabilizes. We propose HybridAL, an adaptive training schedule that monitors an online stabilization signal and switches from retraining to fine-tuning after sustained stabilization. Two complementary signals, spectral exponent change Δα (weight-based) and accuracy change ΔAcc (validation-based), span different points on the time-calibration trade-off. Across three encoder backbones and six text-classification tasks (five seeds each), HybridAL keeps endpoint macro-F1 non-inferior to retraining and fine-tuning at a 0.010 margin, saves up to 49% of retraining time, and recovers a substantial fraction of retraining's calibration advantage as measured by negative log-likelihood (NLL). Compared with schedules that switch at a pre-committed round, HybridAL obtains lower NLL at moderate additional cost, showing that trajectory-dependent switching provides a stronger time-calibration trade-off than fixed early switching.
Community
HybridAL treats the training strategy in active learning (retrain from scratch each round, or fine-tune from the previous checkpoint) as a decision variable rather than a fixed implementation detail.
The structure we found: retraining is most useful in early rounds, when each new batch can substantially reshape the labeled distribution, while fine-tuning becomes safe once the model trajectory stabilizes. HybridAL monitors an online stabilization signal and switches once, after sustained stabilization. We study two signals, spectral exponent change (Δα, weight-based) and accuracy change (ΔAcc, validation-based), which sit at different points on the time–calibration trade-off.
Across three encoder backbones and six text classification tasks, five seeds each: endpoint macro-F1 non-inferior to both retraining and fine-tuning at a 0.010 margin, up to 49% of retraining time saved, and a substantial fraction of retraining's calibration advantage recovered in NLL. Against schedules that switch at a pre-committed round, HybridAL reaches lower NLL at moderate additional cost.
Accepted to EMNLP 2026 Main Conference.
Get this paper in your agent:
hf papers read 2609.06806 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper