FilOzone/dealbot

Avoid duplicating the full piece buffer when parsing sampled-retrieval CARs

Open

#616 opened on Jun 19, 2026

View on GitHub
 (1 comment) (0 reactions) (0 assignees)TypeScript (9 forks)github user discovery
enhancementgood first issuehelp wantedready-for-work

Repository metrics

Stars
 (2 stars)
PR merge metrics
 (PR metrics pending)

Description

readBlocks in the sampled-retrieval piece validation wraps the incoming piece buffer in new Uint8Array(pieceBytes) before handing it to CarReader.fromBytes:

https://github.com/FilOzone/dealbot/blob/2452e7deeb177ff6e94383386458f2a914ebf679/apps/backend/src/retrieval-anon/piece-validation.service.ts#L192-L200

pieceBytes is a Buffer, which already is a Uint8Array, so that wrapper allocates a second full copy of the entire piece. Sampled pieces run up to ~500 MiB, so peak memory for one check is roughly 2x the piece size for no reason.

Fix: pass pieceBytes straight to CarReader.fromBytes(pieceBytes). Same behavior, drops the redundant allocation.

This is pre-existing from #487 and untouched by the rename in #612, so tracking it on its own. The file moves to apps/backend/src/sampled-retrieval/piece-validation.service.ts once #612 lands.

Contributor guide

Research direction
Remove the redundant `new Uint8Array(pieceBytes)` wrapper and pass `pieceBytes` directly to `CarReader.fromBytes`. Verify the fix reduces memory usage and passes existing tests.
Tech stack
typescript
Domain
backend
Issue type
Performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
1
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Node.jsBuffers
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75