facebook/TestSlide

Add `--fail-if-warning`

Open

#195 opened on Jun 26, 2020

View on GitHub
 (4 comments) (2 reactions) (0 assignees)Python (58 forks)auto 404
Hacktoberfestenhancementhelp wanted

Repository metrics

Stars
 (147 stars)
PR merge metrics
 (PR metrics pending)

Description

Let's add an option --fail-if-warning, which'll look over issued warnings during test execution and fail them if warning issued. This can probably be done easily by patching Python's warning method and checking if it was called.

I see 3 cases to check for warnings:

1 - When importing test modules. 2 - When running individual tests. 3 - On shutdown.

For 1 & 2, this should be easy, not sure about 3.

An important feature of this, is to allow include / exclude paths to accept / ignore warnings. Eg: if a warning came from a third party library, there's sometimes little one can do, and we should ignore it. If warning came from the code being tested, it should surface.

Contributor guide

Research direction
Explore Python's warnings module to intercept warnings. Add a command line flag that enables warning capture. Override the warning handler to record or fail on warnings. Consider three phases: module import, individual test execution, and test teardown. Implement path filtering to ignore third party warnings.
Tech stack
python
Domain
testing
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75