grafana/mimir

Validate -blocks-storage.bucket-store.ignore-blocks-within when running in monolithic mode

Open

#2,675 opened on Aug 9, 2022

View on GitHub
 (3 comments) (2 reactions) (0 assignees)Go (808 forks)auto 404
ease-of-usehacktoberfest

Repository metrics

Stars
 (5,181 stars)
PR merge metrics
 (PR metrics pending)

Description

I'm seeing help requests from several people lowering -querier.query-store-after without changing -blocks-storage.bucket-store.ignore-blocks-within, causing some queries to fail with "consistency check failed" errors.

We should validate -blocks-storage.bucket-store.ignore-blocks-within setting to make sure it has been set to a value lower than -querier.query-store-after minus a delta (in jsonnet we used a 2h delta).

Unfortunately, -querier.query-store-after is set on the querier, while -blocks-storage.bucket-store.ignore-blocks-within is set on the store-gateway, so there's no reliable way to validate it when running in microservices mode. However, we could at least validate it when running monolithic mode.

Contributor guide