filecoin-project/filecoin-pin
View on GitHubAdd CLI error-scenario smoke test matrix for devs
Open
#470 opened on May 8, 2026
enhancementgood first issuehelp wantedready-for-workteam/filecoin-pinteam/fs-wg
Repository metrics
- Stars
- (22 stars)
- PR merge metrics
- (PR metrics pending)
Description
Description
Manually verifying CLI error scenarios across all commands is slow and ad-hoc. During PR #468 (CliFatal sentinel work) we hand-rolled a 15-scenario bash sweep to confirm error output and exit codes. We should turn this into a reusable script or test suite so future CLI changes can re-run the matrix in one command.
Goals
- Cover every command + every documented error scenario (invalid args, missing flags, bad RPC, no auth, insufficient funds, user cancel, etc.)
- Capture stdout, stderr, exit code per scenario
- Run without live RPC where possible (use mocks or no-network failure paths)
- Clear pass/fail per scenario; easy to add new scenarios
Acceptance criteria
-
pnpm run smoke(or similar) executes the full matrix - Output format makes it easy to spot regressions (snapshot diff or labeled per-scenario block)
- At minimum covers:
payments setup/fund/deposit/withdraw/status,add,import,data-set show/list/terminate,provider list/show/ping,rm - Argument-validation scenarios (no network, no auth) run in CI
- Network-dependent scenarios (insufficient funds, bad RPC) are clearly tagged and skipped in CI by default
- README or
CONTRIBUTING.mddocuments how to run + add scenarios
Notes
- See PR #468 conversation for the manual sweep template
- Reuse existing
src/test/mocks/synapse-sdk.jswhere mocking is preferable to live RPC - Consider snapshot testing on captured stderr (vitest
toMatchInlineSnapshot) - Out of scope: replacing existing unit/integration tests