CosineAnnealingLrScheduler displays non-standard behavior
#5,128 opened on Jul 6, 2026
Repository metrics
- Stars
- (15,550 stars)
- PR merge metrics
- (PR metrics pending)
Description
I'm using the following setup:
let cosine_config = CosineAnnealingLrSchedulerConfig::new(max_lr, 250)
.with_min_lr(min_lr);
where max_lr = 1e-3 and min_lr = 1e-6
And get the following recurring half-cosine LR waveform:
For pytorch, documentation suggests that once the cosine wave reaches its minima, the LR remains there for the duration (i.e. without restarting the cycle - although I'm unsure what happens in the case where the minima approaches 0.0 which would be an unfortunate place to continue training, as it would be for Burn without configuring a .with_min_lr(), as the default minima would be 0.0 also).
Burn's current implementation which cycles a half cosine wave between max and min does not seem like an acceptable or expected outcome, and clearly could contribute to training instability.