REST API - Tasks

iceprod.rest.handlers.tasks.setup(handler_cfg)

Setup method for Tasks REST API.

Parameters:

handler_cfg (dict) – args to pass to the route

Returns:

routes, database, indexes

Return type:

dict

class iceprod.rest.handlers.tasks.MultiTasksHandler(*args, **kwargs)

Handle multi tasks requests.

async get()

Get task entries.

Params (optional):

status: | separated list of task status to filter by site: site to filter on keys: | separated list of keys to return for each task sort: | separated list of sort key=values, with values of 1 or -1 limit: number of tasks to return

Returns:

{‘tasks’: [<task>]}

Return type:

dict

async post()

Create a task entry.

Body should contain the task data.

Returns:

{‘result’: <task_id>}

Return type:

dict

class iceprod.rest.handlers.tasks.TasksHandler(*args, **kwargs)

Handle single task requests.

async get(task_id)

Get a task entry.

Parameters:

task_id (str) – the task id

Returns:

task entry

Return type:

dict

async patch(task_id)

Update a task entry.

Body should contain the task data to update. Note that this will perform a merge (not replace).

Parameters:

task_id (str) – the task id

Returns:

updated task entry

Return type:

dict

class iceprod.rest.handlers.tasks.TasksStatusHandler(*args, **kwargs)

Handle single task requests.

async put(task_id)

Set a task status.

Body args:

status (str): new status instance_id (str): matching instance id (optional)

Parameters:

task_id (str) – the task id

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.TaskCountsStatusHandler(*args, **kwargs)

Handle task summary grouping by status.

async get()

Get the task counts for all tasks, group by status.

Params (optional):

status: | separated list of task status to filter by

Returns:

{<status>: num}

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetMultiTasksHandler(*args, **kwargs)

Handle multi tasks requests.

async get(dataset_id)

Get task entries.

Params (optional):

status: | separated list of task status to filter by job_id: job_id to filter by job_index: job_index to filter by keys: | separated list of keys to return for each task

Parameters:

dataset_id (str) – dataset id

Returns:

{‘task_id’: {task data}}

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTasksHandler(*args, **kwargs)

Handle single task requests.

async get(dataset_id, task_id)

Get a task entry.

Parameters:

• dataset_id (str) – dataset id

• task_id (str) – the task id

Params (optional):

keys: | separated list of keys to return for each task

Returns:

task entry

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTasksStatusHandler(*args, **kwargs)

Handle single task requests.

async put(dataset_id, task_id)

Set a task status, following possible state transitions.

Body should have {‘status’: <new_status>}

Parameters:

• dataset_id (str) – dataset id

• task_id (str) – the task id

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTasksForceStatusHandler(*args, **kwargs)

Handle single task requests.

async put(dataset_id, task_id)

Set a task status.

Body should have {‘status’: <new_status>}

Parameters:

• dataset_id (str) – dataset id

• task_id (str) – the task id

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskSummaryStatusHandler(*args, **kwargs)

Handle task summary grouping by status.

async get(dataset_id)

Get the task summary for all tasks in a dataset, group by status.

Parameters:

dataset_id (str) – dataset id

Returns:

{<status>: [<task_id>,]}

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskCountsStatusHandler(*args, **kwargs)

Handle task summary grouping by status.

async get(dataset_id)

Get the task counts for all tasks in a dataset, group by status.

Parameters:

dataset_id (str) – dataset id

Returns:

{<status>: num}

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskCountsNameStatusHandler(*args, **kwargs)

Handle task summary grouping by name and status.

async get(dataset_id)

Get the task counts for all tasks in a dataset, group by name,status.

Parameters:

dataset_id (str) – dataset id

Returns:

{<name>: {<status>: num}}

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskStatsHandler(*args, **kwargs)

Handle task stats

async get(dataset_id)

Get the task statistics for all tasks in a dataset, group by name.

Parameters:

dataset_id (str) – dataset id

Returns:

{<name>: {<stat>: <value>}}

Return type:

dict

class iceprod.rest.handlers.tasks.TasksActionsWaitingHandler(*args, **kwargs)

Handle task action for idle -> waiting.

async post()

Take a list of task_ids and move them to waiting.

Body args (json):

task_ids: list

Returns:

{waiting: num tasks waiting}

Return type:

dict

class iceprod.rest.handlers.tasks.TasksActionsQueueHandler(*args, **kwargs)

Handle task action for waiting -> queued.

async post()

Take one waiting task, set its status to queued, and return it.

Body args (json):

requirements: dict query_params: (optional) dict of mongodb params

Returns:

<task dict>

Return type:

dict

class iceprod.rest.handlers.tasks.TasksActionsProcessingHandler(*args, **kwargs)

Handle task action for queued -> processing.

async post(task_id)

Take one queued task, set its status to processing.

Parameters:

task_id (str) – task id

Body args (json):

instance_id (str): task instance id site (str): (optional) site the task is running at

Returns:

<task dict>

Return type:

dict

class iceprod.rest.handlers.tasks.TasksActionsErrorHandler(*args, **kwargs)

Handle task action on error (* -> waiting).

final_status = 'waiting'

async post(task_id)

Take one task, set its status to waiting.

Parameters:

task_id (str) – task id

Body args (json):

instance_id (str): task instance id time_used (int): (optional) time used to run task, in seconds resources (dict): (optional) resources used by task site (str): (optional) site the task was running at reason (str): (optional) reason for error

Returns:

<task dict>

Return type:

dict

class iceprod.rest.handlers.tasks.TasksActionsFailedHandler(*args, **kwargs)

final_status = 'failed'

class iceprod.rest.handlers.tasks.TasksActionsCompleteHandler(*args, **kwargs)

Handle task action on processing -> complete.

async post(task_id)

Take one task, set its status to complete.

Parameters:

task_id (str) – task id

Body args (json):

instance_id (str): task instance id time_used (int): (optional) time used to run task, in seconds site (str): (optional) site the task was running at

Returns:

<task dict>

Return type:

dict

class iceprod.rest.handlers.tasks.TaskBulkStatusHandler(*args, **kwargs)

Update the status of multiple tasks at once.

async post(status)

Set multiple tasks’ status.

Body should have {‘tasks’: [<task_id>, <task_id>, …]}

Parameters:

status (str) – the status

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskBulkStatusHandler(*args, **kwargs)

Update the status of multiple tasks at once.

async post(dataset_id, status)

Set multiple tasks’ status.

Body should have {‘tasks’: [<task_id>, <task_id>, …]}

Parameters:

• dataset_id (str) – dataset id

• status (str) – the status

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskBulkSuspendHandler(*args, **kwargs)

Perform a suspend of all tasks in a dataset.

async post(dataset_id)

Set multiple tasks’ status to suspended.

Body may have {‘jobs’: [<job_id>, <job_id>, …]} or {‘tasks’: [<task_id>, <task_id>, …]}. If it does not, all tasks in a dataset are suspended.

Parameters:

dataset_id (str) – dataset id

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskBulkResetHandler(*args, **kwargs)

Perform a reset of all tasks in a dataset.

async post(dataset_id)

Set multiple tasks’ status back to the starting status.

Body may have {‘jobs’: [<job_id>, <job_id>, …]} or {‘tasks’: [<task_id>, <task_id>, …]}. If it does not, all tasks in a dataset are reset.

Parameters:

dataset_id (str) – dataset id

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskBulkHardResetHandler(*args, **kwargs)

Perform a hard reset of all tasks in a dataset.

async post(dataset_id)

Set multiple tasks’ status back to the starting status.

Body may have {‘jobs’: [<job_id>, <job_id>, …]} or {‘tasks’: [<task_id>, <task_id>, …]}. If it does not, all tasks in a dataset are hard reset.

Parameters:

dataset_id (str) – dataset id

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskBulkRequirementsHandler(*args, **kwargs)

Update the requirements of multiple tasks at once.

async patch(dataset_id, name)

Set multiple tasks’ requirements. Sets for all tasks in a dataset with the specified name.

Body should have {<resource>: <requirement>}.

Parameters:

• dataset_id (str) – dataset id

• name (str) – the task name

Returns:

empty dict

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetMultiFilesHandler(*args, **kwargs)

Handle multi files requests, by dataset.

async get(dataset_id)

Get dataset_files entries.

Parameters:

dataset_id (str) – dataset id

Returns:

{‘files’: [<file>]}

Return type:

dict

async post(dataset_id)

Create a dataset_files entry.

Body should contain the file data.

Parameters:

• filename (str) – the full url filename

• movement (str) – [input | output | both]

• job_index (int) – the job index to add to

• task_name (str) – the name of the task

• local (str) – (optional) the local filename the task sees

• transfer (str) – whether to transfer the file (can be bool or str)

• compression (str) – whether to automatically compress/decompress the file

Returns:

{‘result’: <task_id>}

Return type:

dict

class iceprod.rest.handlers.tasks.DatasetTaskFilesHandler(*args, **kwargs)

Handle multi files requests, by task.

async get(dataset_id, task_id)

Get dataset_files entries.

Parameters:

• dataset_id (str) – dataset id

• task_id (str) – task_id

Returns:

{‘files’: [<file>]}

Return type:

dict

async post(dataset_id, task_id)

Create a dataset_files entry.

Body should contain the file data.

Parameters:

• filename (str) – the full url filename

• movement (str) – [input | output | both]

• local (str) – (optional) the local filename the task sees

• transfer (str) – whether to transfer the file (can be bool or str)

• compression (str) – whether to automatically compress/decompress the file

Returns:

{}

Return type:

dict

async delete(dataset_id, task_id)

Delete dataset_files entries.

Parameters:

• dataset_id (str) – dataset id

• task_id (str) – task_id

Returns:

{}

Return type:

dict