aiida.engine.processes.workchains package

Module for the WorkChain process and related utilities.

aiida.engine.processes.workchains.ToContext

alias of builtins.dict

aiida.engine.processes.workchains.assign_(target)[source]

Convenience function that will construct an Awaitable for a given class instance with the context action set to ASSIGN. When the awaitable target is completed it will be assigned to the context for a key that is to be defined later

Parameters

target – an instance of a Process or Awaitable

Returns

the awaitable

Return type

Awaitable

aiida.engine.processes.workchains.append_(target)[source]

Convenience function that will construct an Awaitable for a given class instance with the context action set to APPEND. When the awaitable target is completed it will be appended to a list in the context for a key that is to be defined later

Parameters

target – an instance of a Process or Awaitable

Returns

the awaitable

Return type

Awaitable

class aiida.engine.processes.workchains.BaseRestartWorkChain(*args, **kwargs)[source]

Bases: aiida.engine.processes.workchains.workchain.WorkChain

Base restart work chain.

This work chain serves as the starting point for more complex work chains that will be designed to run a sub process that might need multiple restarts to come to a successful end. These restarts may be necessary because a single process run is not sufficient to achieve a fully converged result, or certain errors maybe encountered which are recoverable.

This work chain implements the most basic functionality to achieve this goal. It will launch the sub process, restarting until it is completed successfully or the maximum number of iterations is reached. After completion of the sub process it will be inspected, and a list of process handlers are called successively. These process handlers are defined as class methods that are decorated with process_handler().

The idea is to sub class this work chain and leverage the generic error handling that is implemented in the few outline methods. The minimally required outline would look something like the following:

cls.setup
while_(cls.should_run_process)(
    cls.run_process,
    cls.inspect_process,
)

Each of these methods can of course be overriden but they should be general enough to fit most process cycles. The run_process method will take the inputs for the process from the context under the key inputs. The user should, therefore, make sure that before the run_process method is called, that the to be used inputs are stored under self.ctx.inputs. One can update the inputs based on the results from a prior process by calling an outline method just before the run_process step, for example:

cls.setup
while_(cls.should_run_process)(
    cls.prepare_inputs,
    cls.run_process,
    cls.inspect_process,
)

Where in the prepare_calculation method, the inputs dictionary at self.ctx.inputs is updated before the next process will be run with those inputs.

The _process_class attribute should be set to the Process class that should be run in the loop. Finally, to define handlers that will be called during the inspect_process simply define a class method with the signature (self, node) and decorate it with the process_handler decorator, for example:

@process_handler
def handle_problem(self, node):
    if some_problem:
        self.ctx.inputs = improved_inputs
        return ProcessHandlerReport()

The process_handler and ProcessHandlerReport support various arguments to control the flow of the logic of the inspect_process. Refer to their respective documentation for details.

_Process__called = True
__abstractmethods__ = frozenset({})
__init__(*args, **kwargs)[source]

Construct the instance.

__module__ = 'aiida.engine.processes.workchains.restart'
_abc_impl = <_abc_data object>
_considered_handlers_extra = 'considered_handlers'
_handlers()[source]

Return the list of all methods decorated with the process_handler decorator.

Returns

list of process handler methods

_process_class = None
_spec = <aiida.engine.processes.workchains.workchain.WorkChainSpec object>
_wrap_bare_dict_inputs(port_namespace, inputs)[source]

Wrap bare dictionaries in inputs in a Dict node if dictated by the corresponding inputs portnamespace.

Parameters
  • port_namespace – a PortNamespace

  • inputs – a dictionary of inputs intended for submission of the process

Returns

an attribute dictionary with all bare dictionaries wrapped in Dict if dictated by the port namespace

classmethod define(spec)[source]

Define the process specification.

inspect_process()[source]

Analyse the results of the previous process and call the handlers when necessary.

If the process is excepted or killed, the work chain will abort. Otherwise any attached handlers will be called in order of their specified priority. If the process was failed and no handler returns a report indicating that the error was handled, it is considered an unhandled process failure and the process is relaunched. If this happens twice in a row, the work chain is aborted. In the case that at least one handler returned a report the following matrix determines the logic that is followed:

Process Handler Handler Action result report? exit code —————————————– Success yes == 0 Restart Success yes != 0 Abort Failed yes == 0 Restart Failed yes != 0 Abort

If no handler returned a report and the process finished successfully, the work chain’s work is considered done and it will move on to the next step that directly follows the while conditional, if there is one defined in the outline.

on_terminated()[source]

Clean the working directories of all child calculation jobs if clean_workdir=True in the inputs.

results()[source]

Attach the outputs specified in the output specification from the last completed process.

run_process()[source]

Run the next process, taking the input dictionary from the context at self.ctx.inputs.

setup()[source]

Initialize context variables that are used during the logical flow of the BaseRestartWorkChain.

should_run_process()[source]

Return whether a new process should be run.

This is the case as long as the last process has not finished successfully and the maximum number of restarts has not yet been exceeded.

class aiida.engine.processes.workchains.ProcessHandlerReport(do_break, exit_code)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

__module__ = 'aiida.engine.processes.workchains.utils'
static __new__(_cls, do_break=False, exit_code=ExitCode(status=0, message=None, invalidates_cache=False))

Create new instance of ProcessHandlerReport(do_break, exit_code)

__repr__()

Return a nicely formatted representation string

__slots__ = ()
_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('do_break', 'exit_code')
_fields_defaults = {}
classmethod _make(iterable)

Make a new ProcessHandlerReport object from a sequence or iterable

_replace(**kwds)

Return a new ProcessHandlerReport object replacing specified fields with new values

property do_break

Alias for field number 0

property exit_code

Alias for field number 1

aiida.engine.processes.workchains.process_handler(wrapped=None, *, priority=None, exit_codes=None)[source]

Decorator to register a BaseRestartWorkChain instance method as a process handler.

The decorator will validate the priority and exit_codes optional keyword arguments and then add itself as an attribute to the wrapped instance method. This is used in the inspect_process to return all instance methods of the class that have been decorated by this function and therefore are considered to be process handlers.

Requirements on the function signature of process handling functions. The function to which the decorator is applied needs to take two arguments:

  • self: This is the instance of the work chain itself

  • node: This is the process node that finished and is to be investigated

The function body typically consists of a conditional that will check for a particular problem that might have occurred for the sub process. If a particular problem is handled, the process handler should return an instance of the aiida.engine.ProcessHandlerReport tuple. If no other process handlers should be considered, the set do_break attribute should be set to True. If the work chain is to be aborted entirely, the exit_code of the report can be set to an ExitCode instance with a non-zero status.

Parameters
  • cls – the work chain class to register the process handler with

  • priority – optional integer that defines the order in which registered handlers will be called during the handling of a finished process. Higher priorities will be handled first.

  • exit_codes – single or list of ExitCode instances. If defined, the handler will return None if the exit code set on the node does not appear in the exit_codes. This is useful to have a handler called only when the process failed with a specific exit code.

class aiida.engine.processes.workchains.WorkChain(inputs=None, logger=None, runner=None, enable_persistence=True)[source]

Bases: aiida.engine.processes.process.Process

The WorkChain class is the principle component to implement workflows in AiiDA.

_CONTEXT = 'CONTEXT'
_Process__called = True
_STEPPER_STATE = 'stepper_state'
__abstractmethods__ = frozenset({})
__init__(inputs=None, logger=None, runner=None, enable_persistence=True)[source]

Construct a WorkChain instance.

Construct the instance only if it is a sub class of WorkChain, otherwise raise InvalidOperation.

Parameters
  • inputs (dict) – work chain inputs

  • logger (logging.Logger) – aiida logger

  • runner – work chain runner

  • enable_persistence (bool) – whether to persist this work chain

Type

aiida.engine.runners.Runner

__module__ = 'aiida.engine.processes.workchains.workchain'
_abc_impl = <_abc_data object>
_auto_persist = {'_CREATION_TIME', '_awaitables', '_enable_persistence', '_future', '_parent_pid', '_paused', '_pid', '_pre_paused_status', '_status'}
_do_step()[source]

Execute the next step in the outline and return the result.

If the stepper returns a non-finished status and the return value is of type ToContext, the contents of the ToContext container will be turned into awaitables if necessary. If any awaitables were created, the process will enter in the Wait state, otherwise it will go to Continue. When the stepper returns that it is done, the stepper result will be converted to None and returned, unless it is an integer or instance of ExitCode.

_node_class

alias of aiida.orm.nodes.process.workflow.workchain.WorkChainNode

_spec = <aiida.engine.processes.workchains.workchain.WorkChainSpec object>
_spec_class

alias of WorkChainSpec

_store_nodes(data)[source]

Recurse through a data structure and store any unstored nodes that are found along the way

Parameters

data – a data structure potentially containing unstored nodes

_update_process_status()[source]

Set the process status with a message accounting the current sub processes that we are waiting for.

action_awaitables()[source]

Handle the awaitables that are currently registered with the work chain

Depending on the class type of the awaitable’s target a different callback function will be bound with the awaitable and the runner will be asked to call it when the target is completed

property ctx

Get context.

Return type

aiida.common.extendeddicts.AttributeDict

insert_awaitable(awaitable)[source]

Insert an awaitable that should be terminated before before continuing to the next step.

Parameters

awaitable (aiida.engine.processes.workchains.awaitable.Awaitable) – the thing to await

load_instance_state(saved_state, load_context)[source]

Load instance state.

Parameters
  • saved_state – saved instance state

  • load_context (plumpy.persistence.LoadSaveContext) –

on_exiting()[source]

Ensure that any unstored nodes in the context are stored, before the state is exited

After the state is exited the next state will be entered and if persistence is enabled, a checkpoint will be saved. If the context contains unstored nodes, the serialization necessary for checkpointing will fail.

on_process_finished(awaitable, pk)[source]

Callback function called by the runner when the process instance identified by pk is completed.

The awaitable will be effectuated on the context of the work chain and removed from the internal list. If all awaitables have been dealt with, the work chain process is resumed.

Parameters
  • awaitable – an Awaitable instance

  • pk (int) – the pk of the awaitable’s target

on_run()[source]
on_wait(awaitables)[source]
remove_awaitable(awaitable)[source]

Remove an awaitable.

Precondition: must be an awaitable that was previously inserted.

Parameters

awaitable – the awaitable to remove

run()[source]
save_instance_state(out_state, save_context)[source]

Save instance stace.

Parameters
  • out_state – state to save in

  • save_context (plumpy.persistence.LoadSaveContext) –

to_context(**kwargs)[source]

Add a dictionary of awaitables to the context.

This is a convenience method that provides syntactic sugar, for a user to add multiple intersteps that will assign a certain value to the corresponding key in the context of the work chain.

aiida.engine.processes.workchains.if_(condition)[source]

A conditional that can be used in a workchain outline.

Use as:

if_(cls.conditional)(
  cls.step1,
  cls.step2
)

Each step can, of course, also be any valid workchain step e.g. conditional.

Parameters

condition – The workchain method that will return True or False

aiida.engine.processes.workchains.while_(condition)[source]

A while loop that can be used in a workchain outline.

Use as:

while_(cls.conditional)(
  cls.step1,
  cls.step2
)

Each step can, of course, also be any valid workchain step e.g. conditional.

Parameters

condition – The workchain method that will return True or False

Submodules

Enums and function for the awaitables of Processes.

class aiida.engine.processes.workchains.awaitable.Awaitable(**kwargs)[source]

Bases: plumpy.utils.AttributesDict

An attribute dictionary that represents an action that a Process could be waiting for to finish.

__module__ = 'aiida.engine.processes.workchains.awaitable'
class aiida.engine.processes.workchains.awaitable.AwaitableTarget[source]

Bases: enum.Enum

Enum that describes the class of the target a given awaitable.

PROCESS = 'process'
__module__ = 'aiida.engine.processes.workchains.awaitable'
class aiida.engine.processes.workchains.awaitable.AwaitableAction[source]

Bases: enum.Enum

Enum that describes the action to be taken for a given awaitable.

APPEND = 'append'
ASSIGN = 'assign'
__module__ = 'aiida.engine.processes.workchains.awaitable'
aiida.engine.processes.workchains.awaitable.construct_awaitable(target)[source]

Construct an instance of the Awaitable class that will contain the information related to the action to be taken with respect to the context once the awaitable object is completed.

The awaitable is a simple dictionary with the following keys

  • pk: the pk of the node that is being waited on

  • action: the context action to be performed upon completion

  • outputs: a boolean that toggles whether the node itself

Currently the only awaitable classes are ProcessNode and Workflow The only awaitable actions are the Assign and Append operators

Convenience functions to add awaitables to the Context of a WorkChain.

aiida.engine.processes.workchains.context.ToContext

alias of builtins.dict

aiida.engine.processes.workchains.context.assign_(target)[source]

Convenience function that will construct an Awaitable for a given class instance with the context action set to ASSIGN. When the awaitable target is completed it will be assigned to the context for a key that is to be defined later

Parameters

target – an instance of a Process or Awaitable

Returns

the awaitable

Return type

Awaitable

aiida.engine.processes.workchains.context.append_(target)[source]

Convenience function that will construct an Awaitable for a given class instance with the context action set to APPEND. When the awaitable target is completed it will be appended to a list in the context for a key that is to be defined later

Parameters

target – an instance of a Process or Awaitable

Returns

the awaitable

Return type

Awaitable

Base implementation of WorkChain class that implements a simple automated restart mechanism for sub processes.

class aiida.engine.processes.workchains.restart.BaseRestartWorkChain(*args, **kwargs)[source]

Bases: aiida.engine.processes.workchains.workchain.WorkChain

Base restart work chain.

This work chain serves as the starting point for more complex work chains that will be designed to run a sub process that might need multiple restarts to come to a successful end. These restarts may be necessary because a single process run is not sufficient to achieve a fully converged result, or certain errors maybe encountered which are recoverable.

This work chain implements the most basic functionality to achieve this goal. It will launch the sub process, restarting until it is completed successfully or the maximum number of iterations is reached. After completion of the sub process it will be inspected, and a list of process handlers are called successively. These process handlers are defined as class methods that are decorated with process_handler().

The idea is to sub class this work chain and leverage the generic error handling that is implemented in the few outline methods. The minimally required outline would look something like the following:

cls.setup
while_(cls.should_run_process)(
    cls.run_process,
    cls.inspect_process,
)

Each of these methods can of course be overriden but they should be general enough to fit most process cycles. The run_process method will take the inputs for the process from the context under the key inputs. The user should, therefore, make sure that before the run_process method is called, that the to be used inputs are stored under self.ctx.inputs. One can update the inputs based on the results from a prior process by calling an outline method just before the run_process step, for example:

cls.setup
while_(cls.should_run_process)(
    cls.prepare_inputs,
    cls.run_process,
    cls.inspect_process,
)

Where in the prepare_calculation method, the inputs dictionary at self.ctx.inputs is updated before the next process will be run with those inputs.

The _process_class attribute should be set to the Process class that should be run in the loop. Finally, to define handlers that will be called during the inspect_process simply define a class method with the signature (self, node) and decorate it with the process_handler decorator, for example:

@process_handler
def handle_problem(self, node):
    if some_problem:
        self.ctx.inputs = improved_inputs
        return ProcessHandlerReport()

The process_handler and ProcessHandlerReport support various arguments to control the flow of the logic of the inspect_process. Refer to their respective documentation for details.

_Process__called = True
__abstractmethods__ = frozenset({})
__init__(*args, **kwargs)[source]

Construct the instance.

__module__ = 'aiida.engine.processes.workchains.restart'
_abc_impl = <_abc_data object>
_considered_handlers_extra = 'considered_handlers'
_handlers()[source]

Return the list of all methods decorated with the process_handler decorator.

Returns

list of process handler methods

_process_class = None
_spec = <aiida.engine.processes.workchains.workchain.WorkChainSpec object>
_wrap_bare_dict_inputs(port_namespace, inputs)[source]

Wrap bare dictionaries in inputs in a Dict node if dictated by the corresponding inputs portnamespace.

Parameters
  • port_namespace – a PortNamespace

  • inputs – a dictionary of inputs intended for submission of the process

Returns

an attribute dictionary with all bare dictionaries wrapped in Dict if dictated by the port namespace

classmethod define(spec)[source]

Define the process specification.

inspect_process()[source]

Analyse the results of the previous process and call the handlers when necessary.

If the process is excepted or killed, the work chain will abort. Otherwise any attached handlers will be called in order of their specified priority. If the process was failed and no handler returns a report indicating that the error was handled, it is considered an unhandled process failure and the process is relaunched. If this happens twice in a row, the work chain is aborted. In the case that at least one handler returned a report the following matrix determines the logic that is followed:

Process Handler Handler Action result report? exit code —————————————– Success yes == 0 Restart Success yes != 0 Abort Failed yes == 0 Restart Failed yes != 0 Abort

If no handler returned a report and the process finished successfully, the work chain’s work is considered done and it will move on to the next step that directly follows the while conditional, if there is one defined in the outline.

on_terminated()[source]

Clean the working directories of all child calculation jobs if clean_workdir=True in the inputs.

results()[source]

Attach the outputs specified in the output specification from the last completed process.

run_process()[source]

Run the next process, taking the input dictionary from the context at self.ctx.inputs.

setup()[source]

Initialize context variables that are used during the logical flow of the BaseRestartWorkChain.

should_run_process()[source]

Return whether a new process should be run.

This is the case as long as the last process has not finished successfully and the maximum number of restarts has not yet been exceeded.

Utilities for WorkChain implementations.

class aiida.engine.processes.workchains.utils.ProcessHandlerReport(do_break, exit_code)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

__module__ = 'aiida.engine.processes.workchains.utils'
static __new__(_cls, do_break=False, exit_code=ExitCode(status=0, message=None, invalidates_cache=False))

Create new instance of ProcessHandlerReport(do_break, exit_code)

__repr__()

Return a nicely formatted representation string

__slots__ = ()
_asdict()

Return a new OrderedDict which maps field names to their values.

_fields = ('do_break', 'exit_code')
_fields_defaults = {}
classmethod _make(iterable)

Make a new ProcessHandlerReport object from a sequence or iterable

_replace(**kwds)

Return a new ProcessHandlerReport object replacing specified fields with new values

property do_break

Alias for field number 0

property exit_code

Alias for field number 1

aiida.engine.processes.workchains.utils.process_handler(wrapped=None, *, priority=None, exit_codes=None)[source]

Decorator to register a BaseRestartWorkChain instance method as a process handler.

The decorator will validate the priority and exit_codes optional keyword arguments and then add itself as an attribute to the wrapped instance method. This is used in the inspect_process to return all instance methods of the class that have been decorated by this function and therefore are considered to be process handlers.

Requirements on the function signature of process handling functions. The function to which the decorator is applied needs to take two arguments:

  • self: This is the instance of the work chain itself

  • node: This is the process node that finished and is to be investigated

The function body typically consists of a conditional that will check for a particular problem that might have occurred for the sub process. If a particular problem is handled, the process handler should return an instance of the aiida.engine.ProcessHandlerReport tuple. If no other process handlers should be considered, the set do_break attribute should be set to True. If the work chain is to be aborted entirely, the exit_code of the report can be set to an ExitCode instance with a non-zero status.

Parameters
  • cls – the work chain class to register the process handler with

  • priority – optional integer that defines the order in which registered handlers will be called during the handling of a finished process. Higher priorities will be handled first.

  • exit_codes – single or list of ExitCode instances. If defined, the handler will return None if the exit code set on the node does not appear in the exit_codes. This is useful to have a handler called only when the process failed with a specific exit code.

Components for the WorkChain concept of the workflow engine.

class aiida.engine.processes.workchains.workchain.WorkChain(inputs=None, logger=None, runner=None, enable_persistence=True)[source]

Bases: aiida.engine.processes.process.Process

The WorkChain class is the principle component to implement workflows in AiiDA.

_CONTEXT = 'CONTEXT'
_Process__called = True
_STEPPER_STATE = 'stepper_state'
__abstractmethods__ = frozenset({})
__init__(inputs=None, logger=None, runner=None, enable_persistence=True)[source]

Construct a WorkChain instance.

Construct the instance only if it is a sub class of WorkChain, otherwise raise InvalidOperation.

Parameters
  • inputs (dict) – work chain inputs

  • logger (logging.Logger) – aiida logger

  • runner – work chain runner

  • enable_persistence (bool) – whether to persist this work chain

Type

aiida.engine.runners.Runner

__module__ = 'aiida.engine.processes.workchains.workchain'
_abc_impl = <_abc_data object>
_auto_persist = {'_CREATION_TIME', '_awaitables', '_enable_persistence', '_future', '_parent_pid', '_paused', '_pid', '_pre_paused_status', '_status'}
_do_step()[source]

Execute the next step in the outline and return the result.

If the stepper returns a non-finished status and the return value is of type ToContext, the contents of the ToContext container will be turned into awaitables if necessary. If any awaitables were created, the process will enter in the Wait state, otherwise it will go to Continue. When the stepper returns that it is done, the stepper result will be converted to None and returned, unless it is an integer or instance of ExitCode.

_node_class

alias of aiida.orm.nodes.process.workflow.workchain.WorkChainNode

_spec = <aiida.engine.processes.workchains.workchain.WorkChainSpec object>
_spec_class

alias of WorkChainSpec

_store_nodes(data)[source]

Recurse through a data structure and store any unstored nodes that are found along the way

Parameters

data – a data structure potentially containing unstored nodes

_update_process_status()[source]

Set the process status with a message accounting the current sub processes that we are waiting for.

action_awaitables()[source]

Handle the awaitables that are currently registered with the work chain

Depending on the class type of the awaitable’s target a different callback function will be bound with the awaitable and the runner will be asked to call it when the target is completed

property ctx

Get context.

Return type

aiida.common.extendeddicts.AttributeDict

insert_awaitable(awaitable)[source]

Insert an awaitable that should be terminated before before continuing to the next step.

Parameters

awaitable (aiida.engine.processes.workchains.awaitable.Awaitable) – the thing to await

load_instance_state(saved_state, load_context)[source]

Load instance state.

Parameters
  • saved_state – saved instance state

  • load_context (plumpy.persistence.LoadSaveContext) –

on_exiting()[source]

Ensure that any unstored nodes in the context are stored, before the state is exited

After the state is exited the next state will be entered and if persistence is enabled, a checkpoint will be saved. If the context contains unstored nodes, the serialization necessary for checkpointing will fail.

on_process_finished(awaitable, pk)[source]

Callback function called by the runner when the process instance identified by pk is completed.

The awaitable will be effectuated on the context of the work chain and removed from the internal list. If all awaitables have been dealt with, the work chain process is resumed.

Parameters
  • awaitable – an Awaitable instance

  • pk (int) – the pk of the awaitable’s target

on_run()[source]
on_wait(awaitables)[source]
remove_awaitable(awaitable)[source]

Remove an awaitable.

Precondition: must be an awaitable that was previously inserted.

Parameters

awaitable – the awaitable to remove

run()[source]
save_instance_state(out_state, save_context)[source]

Save instance stace.

Parameters
  • out_state – state to save in

  • save_context (plumpy.persistence.LoadSaveContext) –

to_context(**kwargs)[source]

Add a dictionary of awaitables to the context.

This is a convenience method that provides syntactic sugar, for a user to add multiple intersteps that will assign a certain value to the corresponding key in the context of the work chain.

aiida.engine.processes.workchains.workchain.if_(condition)[source]

A conditional that can be used in a workchain outline.

Use as:

if_(cls.conditional)(
  cls.step1,
  cls.step2
)

Each step can, of course, also be any valid workchain step e.g. conditional.

Parameters

condition – The workchain method that will return True or False

aiida.engine.processes.workchains.workchain.while_(condition)[source]

A while loop that can be used in a workchain outline.

Use as:

while_(cls.conditional)(
  cls.step1,
  cls.step2
)

Each step can, of course, also be any valid workchain step e.g. conditional.

Parameters

condition – The workchain method that will return True or False