OpenFreeEnergy/alchemiscale

Create API-enforced, configurable upper bounds on number of 'waiting' `Task`s on a `Transformation`, number of `Task`s ACTIONed on a `TaskHub`

Open

#134 opened on Apr 27, 2023

View on GitHub
 (0 comments) (0 reactions) (1 assignee)Python (10 forks)auto 404
good first issue

Repository metrics

Stars
 (35 stars)
PR merge metrics
 (PR metrics pending)

Description

Currently, there exist no enforced limit on the number of Tasks a user can create for a given Transformation, meaning it is very easy to (accidentally or otherwise) create very large (thousands or more) Tasks on a single Transformation. There is also no enforced limit on the number of such Tasks that can have ACTIONSrelationships to a givenTaskHub`, which can choke the compute services due to the bottleneck of the claiming code (reduced by #121, but not able to scale infinitely).

To address this, we should create server-side configurable limits to:

  • the number of waiting Tasks allowed on a Transformation at a time
  • the number of ACTIONS relationships on a TaskHub at a time

User-client side, this will mean that when a user calls AlchemiscaleClient.create_tasks(...), they will get back ScopedKeys for Tasks created until no more can be created for that Transformation; if a Task can't be created due to the limit, this will be reflected in the list of ScopedKeys returned, with a None given in its place.

Likewise, when a user calls AlchemiscaleClient.action_tasks(...), the method will action as many of the Tasks as it can up till the limit, then a None will be given for those Tasks that were not actioned.

Contributor guide