tracel-ai/burn

CosineAnnealingLrScheduler displays non-standard behavior

Open

#5,128 opened on Jul 6, 2026

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Rust (964 forks)auto 404
buggood first issue

Repository metrics

Stars
 (15,550 stars)
PR merge metrics
 (PR metrics pending)

Description

I'm using the following setup:

    let cosine_config = CosineAnnealingLrSchedulerConfig::new(max_lr, 250)
        .with_min_lr(min_lr);

where max_lr = 1e-3 and min_lr = 1e-6

And get the following recurring half-cosine LR waveform:

For pytorch, documentation suggests that once the cosine wave reaches its minima, the LR remains there for the duration (i.e. without restarting the cycle - although I'm unsure what happens in the case where the minima approaches 0.0 which would be an unfortunate place to continue training, as it would be for Burn without configuring a .with_min_lr(), as the default minima would be 0.0 also).

Burn's current implementation which cycles a half cosine wave between max and min does not seem like an acceptable or expected outcome, and clearly could contribute to training instability.

Contributor guide