tektoncd/pipeline

Design: Partial Pipeline execution

Open

#50 opened on Sep 16, 2018

View on GitHub
 (15 comments) (9 reactions) (0 assignees)Go (1,943 forks)auto 404
area/apiarea/roadmapdesignhelp wantedkind/featurelifecycle/frozenpriority/important-longterm

Repository metrics

Stars
 (9,013 stars)
PR merge metrics
 (PR metrics pending)

Description

The work for this task is to design this feature and present one or more proposals (before implementing).

Expected Behavior

If a pipeline has many tasks and takes a long time to run (e.g. tens of minutes, or even hours), and one Task fails, it might be desirable to be able to pick up execution where the Task failed, with different PipelineParams (e.g. from a different git commit), so you can resume the Pipeline without having to rerun the whole thing.

Some ideas for how to implement this:

  1. Fields in a PipelineRun which override which Tasks to run from / refer to a previous PipelineRun from which results should be taken
  2. A tool which makes it easy to create a new Pipeline from an existing one which only runs a subset of the Tasks

It is also worth considering what this could be like via a UI: if one is viewing a Pipeline in a UI, and wants to re-run only a portion of the Pipeline, they probably want the user experience to be as if they were still running the same Pipeline, even if underneath a new Pipeline is created.

Actual Behavior

At the moment, if any Task in a Pipeline fails, your options to rerun the rest of the Pipeline would be:

  1. Run the entire Pipeline again
  2. Create a new Pipeline from the previous one which contains only the Tasks you wish to run

Additional Info

This originally came up in discussion about #39, in the context of whether or not we'd want to always use the same git commit from a source for all Tasks in a Pipeline, or if we wanted sometimes for a Task to always use HEAD. This would allow a user to change a repo, by updating HEAD, between Task executions.

The feature of partial pipeline execution could be an alternative to this.

Contributor guide