awslabs/benchmark-ai

Is there a definitive FAIL / SUCCEED event status?

Open

#1,035 opened on Jul 27, 2020

View on GitHub
 (1 comment) (0 reactions) (1 assignee)Python (6 forks)auto 404
bai-clientenhancementgood first issue

Repository metrics

Stars
 (19 stars)
PR merge metrics
 (PR metrics pending)

Description

It would be good to have a definitive FAIL/SUCCEED status for Actions. This way they can be more directly tested against to inform end user.. One case for this is the use of --terminate (with --watch) We are looking at the status event message string. There can be a tighter contract than that.

Contributor guide

Research direction
Investigate the current event status mechanism in the codebase, identify where status strings are generated and consumed, and propose a concrete FAIL/SUCCEED status enumeration with clear semantics.
Tech stack
python
Domain
backendtesting
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
Python
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65