Aiven-Open/cloud-storage-connectors-for-apache-kafka

S3 source silently fails with `No space in queue`

Open

#513 opened on Aug 21, 2025

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Java (37 forks)auto 404
documentationgood first issue

Repository metrics

Stars
 (56 stars)
PR merge metrics
 (PR metrics pending)

Description

Original title was: S3 source silently fails with No space in queue Documentation should be extracted from the information provided by @Claudenw below.

After setting up the S3 sink & S3 source plugins - I noticed that while the S3 source is able to see the files and create the missing topics, it was failing to actually commit anything due to this No space in queue error and it would still go through with committing the file.

[Worker-0b4cf60df7c1ad1cd] [2025-08-21 05:28:29,254] INFO [127.0.0.1](http://127.0.0.1/) - - [21/Aug/2025:05:28:29 +0000] "GET / HTTP/1.1" 200 82 "-" "curl/8.3.0" 1 (org.apache.kafka.connect.runtime.rest.RestServer:62)
[Worker-0b4cf60df7c1ad1cd] [2025-08-21 05:28:31,094] INFO [s3-to-kafka|task-0] No space in queue (io.aiven.kafka.connect.s3.source.S3SourceTask:198)
[Worker-0b4cf60df7c1ad1cd] [2025-08-21 05:28:34,733] INFO [s3-to-kafka|task-0] No space in queue (io.aiven.kafka.connect.s3.source.S3SourceTask:198)
[Worker-0b4cf60df7c1ad1cd] [2025-08-21 05:28:39,125] INFO [s3-to-kafka|task-0] No space in queue (io.aiven.kafka.connect.s3.source.S3SourceTask:198)
[Worker-0b4cf60df7c1ad1cd] [2025-08-21 05:28:43,558] INFO [s3-to-kafka|task-0] No space in queue (io.aiven.kafka.connect.s3.source.S3SourceTask:198)
[Worker-0b4cf60df7c1ad1cd] [2025-08-21 05:28:44,150] INFO [s3-to-kafka|task-0|offsets] Committed all records through last poll() (io.aiven.kafka.connect.s3.source.S3SourceTask:128)

Contributor guide

Research direction
Investigate the 'No space in queue' error in S3SourceTask and trace why it still commits despite the queue being full. Check the logic around queue capacity and commit behavior.
Tech stack
java
Domain
backendapi
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
JavaKafka
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50