filecoin-project/filecoin-pin

Add CLI error-scenario smoke test matrix for devs

Open

#470 opened on May 8, 2026

View on GitHub
 (7 comments) (0 reactions) (1 assignee)TypeScript (18 forks)github user discovery
enhancementgood first issuehelp wantedready-for-workteam/filecoin-pinteam/fs-wg

Repository metrics

Stars
 (22 stars)
PR merge metrics
 (PR metrics pending)

Description

Description

Manually verifying CLI error scenarios across all commands is slow and ad-hoc. During PR #468 (CliFatal sentinel work) we hand-rolled a 15-scenario bash sweep to confirm error output and exit codes. We should turn this into a reusable script or test suite so future CLI changes can re-run the matrix in one command.

Goals

  • Cover every command + every documented error scenario (invalid args, missing flags, bad RPC, no auth, insufficient funds, user cancel, etc.)
  • Capture stdout, stderr, exit code per scenario
  • Run without live RPC where possible (use mocks or no-network failure paths)
  • Clear pass/fail per scenario; easy to add new scenarios

Acceptance criteria

  • pnpm run smoke (or similar) executes the full matrix
  • Output format makes it easy to spot regressions (snapshot diff or labeled per-scenario block)
  • At minimum covers: payments setup/fund/deposit/withdraw/status, add, import, data-set show/list/terminate, provider list/show/ping, rm
  • Argument-validation scenarios (no network, no auth) run in CI
  • Network-dependent scenarios (insufficient funds, bad RPC) are clearly tagged and skipped in CI by default
  • README or CONTRIBUTING.md documents how to run + add scenarios

Notes

  • See PR #468 conversation for the manual sweep template
  • Reuse existing src/test/mocks/synapse-sdk.js where mocking is preferable to live RPC
  • Consider snapshot testing on captured stderr (vitest toMatchInlineSnapshot)
  • Out of scope: replacing existing unit/integration tests

Contributor guide