aiida.daemon package

Submodules

class aiida.daemon.client.ControllerProtocol[source]

Bases: enum.Enum

The protocol to use to for the controller of the Circus daemon

IPC = 0
TCP = 1
__module__ = 'aiida.daemon.client'
class aiida.daemon.client.DaemonClient(profile_name=None)[source]

Bases: aiida.common.profile.ProfileConfig

Extension of the ProfileConfig which also provides handles to retrieve profile specific properties related to the daemon client

DAEMON_ERROR_NOT_RUNNING = 'daemon-error-not-running'
DAEMON_ERROR_TIMEOUT = 'daemon-error-timeout'
_DAEMON_NAME = 'aiida-{name}'
_DEFAULT_LOGLEVEL = 'INFO'
_ENDPOINT_PROTOCOL = 0
__init__(profile_name=None)[source]

Construct the ProfileConfig for the given profile_name or retrieve it from the backend settings

__module__ = 'aiida.daemon.client'
call_client(command)[source]

Call the client with a specific command. Will check whether the daemon is running first by checking for the pid file. When the pid is found yet the call still fails with a timeout, this means the daemon was actually not running and it was terminated unexpectedly causing the pid file to not be cleaned up properly

Parameters:command – command to call the circus client with
Returns:the result of the circus client call
circus_log_file
circus_pid_file
circus_port_file
circus_socket_endpoints
circus_socket_file
client

Return an instance of the CircusClient with the endpoint defined by the controller endpoint, which used the port that was written to the port file upon starting of the daemon

Returns:CircucClient instance
cmd_string

Return the command string to start the AiiDA daemon

daemon_log_file
daemon_name

Get the daemon name which is tied to the profile name

daemon_pid_file
decrease_workers(number)[source]

Decrease the number of workers

Parameters:number – the number of workers to remove
Returns:the client call response
delete_circus_socket_directory()[source]

Attempt to delete the directory used to store the circus endpoint sockets. Will not raise if the directory does not exist

get_available_port()[source]

Get an available port from the operating system

Returns:a currently available port
get_circus_port()[source]

Retrieve the port for the circus controller, which should be written to the circus port file. If the daemon is running, the port file should exist and contain the port to which the controller is connected. If it cannot be read, a RuntimeError will be thrown. If the daemon is not running, an available port will be requested from the operating system, written to the port file and returned

Returns:the port for the circus controller
get_circus_socket_directory()[source]

Retrieve the absolute path of the directory where the circus sockets are stored if the IPC protocol is used and the daemon is running. If the daemon is running, the sockets file should exist and contain the absolute path of the directory that contains the sockets of the circus endpoints. If it cannot be read, a RuntimeError will be thrown. If the daemon is not running, a temporary directory will be created and its path will be written to the sockets file and returned.

Note

A temporary folder needs to be used for the sockets because UNIX limits the filepath length to 107 bytes. Placing the socket files in the AiiDA config folder might seem like the more logical choice but that folder can be placed in an arbitrarily nested directory, the socket filename will exceed the limit. The solution is therefore to always store them in the temporary directory of the operation system whose base path is typically short enough as to not exceed the limit

Returns:the absolute path of directory to write the sockets to
get_controller_endpoint()[source]

Get the endpoint string for the circus controller. For the IPC protocol a profile specific socket will be used, whereas for the TCP protocol an available port will be found and saved in the profile specific port file

Returns:the endpoint string
get_daemon_info()[source]

Get statistics about this daemon itself

Returns:the client call response
get_daemon_pid()[source]

Get the daemon pid which should be written in the daemon pid file specific to the profile

Returns:the pid of the circus daemon process or None if not found
get_ipc_endpoint(endpoint)[source]

Get the ipc endpoint string for a circus daemon endpoint for a given socket

Parameters:endpoint – the circus endpoint for which to return a socket
Returns:the ipc endpoint string
get_pubsub_endpoint()[source]

Get the endpoint string for the circus pubsub endpoint. For the IPC protocol a profile specific socket will be used, whereas for the TCP protocol any available port will be used

Returns:the endpoint string
get_stats_endpoint()[source]

Get the endpoint string for the circus stats endpoint. For the IPC protocol a profile specific socket will be used, whereas for the TCP protocol any available port will be used

Returns:the endpoint string
get_status()[source]

Get the daemon running status

Returns:the client call response
get_tcp_endpoint(port=None)[source]

Get the tcp endpoint string for a circus daemon endpoint. If the port is unspecified, the operating system will be asked for a currently available port.

Parameters:port – a port to use for the endpoint
Returns:the tcp endpoint string
get_worker_info()[source]

Get workers statistics for this daemon

Returns:the client call response
increase_workers(number)[source]

Increase the number of workers

Parameters:number – the number of workers to add
Returns:the client call response
is_daemon_running

Return whether the daemon is running, which is determined by seeing if the daemon pid file is present

Returns:True if daemon is running, False otherwise
loglevel
restart_daemon(wait)[source]

Restart the daemon

Parameters:wait – boolean to indicate whether to wait for the result of the command
Returns:the client call response
stop_daemon(wait)[source]

Stop the daemon

Parameters:wait – boolean to indicate whether to wait for the result of the command
Returns:the client call response
virtualenv

This file contains the main routines to submit, check and retrieve calculation results. These are general and contain only the main logic; where appropriate, the routines make reference to the suitable plugins for all plugin-specific operations.

aiida.daemon.execmanager._retrieve_singlefiles(job, transport, folder, retrieve_file_list, logger_extra=None)[source]
aiida.daemon.execmanager.parse_results(job, retrieved_temporary_folder=None)[source]

Parse the results for a given JobCalculation (job)

Returns:integer exit code, where 0 indicates success and non-zero failure
aiida.daemon.execmanager.retrieve_all(job, transport, retrieved_temporary_folder)[source]

Retrieve all the files of a completed job calculation using the given transport. If the job defined anything in the retrieve_temporary_list, those entries will be stored in the retrieved_temporary_folder. The caller is responsible for creating and destroying this folder.

Parameters:
  • job – the finished JobCalculation whose files to retrieve
  • transport – the Transport instance to use for the file retrieval
  • retrieved_temporary_folder – the absolute path to a directory in which to store the files listed, if any, in the retrieved_temporary_folder of the jobs CalcInfo
aiida.daemon.execmanager.retrieve_files_from_list(calculation, transport, folder, retrieve_list)[source]

Retrieve all the files in the retrieve_list from the remote into the local folder instance through the transport. The entries in the retrieve_list can be of two types:

  • a string
  • a list

If it is a string, it represents the remote absolute filepath of the file. If the item is a list, the elements will correspond to the following:

  • remotepath
  • localpath
  • depth

If the remotepath contains file patterns with wildcards, the localpath will be treated as the work directory of the folder and the depth integer determines upto what level of the original remotepath nesting the files will be copied.

Parameters:
  • transport – the Transport instance
  • folder – an absolute path to a folder to copy files in
  • retrieve_list – the list of files to retrieve
aiida.daemon.execmanager.submit_calc(calc, authinfo, transport=None)[source]

Submit a calculation

Note:

if no transport is passed, a new transport is opened and then closed within this function. If you want to use an already opened transport, pass it as further parameter. In this case, the transport has to be already open, and must coincide with the transport of the the computer defined by the authinfo.

Parameters:
  • calc – the calculation to submit (an instance of the aiida.orm.JobCalculation class)
  • authinfo – the AuthInfo object for this calculation.
  • transport – if passed, must be an already opened transport. No checks are done on the consistency of the given transport with the transport of the computer defined in the AuthInfo.
aiida.daemon.execmanager.update_job_calc_from_detailed_job_info(calc, detailed_job_info)[source]

Updates the detailed job info for a JobCalculation as obtained from the scheduler

Parameters:
  • calc – The job calculation
  • detailed_job_info – the detailed information as returned by the scheduler for this job
aiida.daemon.execmanager.update_job_calc_from_job_info(calc, job_info)[source]

Updates the job info for a JobCalculation using job information as obtained from the scheduler.

Parameters:
  • calc – The job calculation
  • job_info – the information returned by the scheduler for this job
Returns:

True if the job state is DONE, False otherwise

Return type:

bool

aiida.daemon.runner.legacy_workflow_stepper()[source]

Function to tick the legacy workflows

aiida.daemon.runner.start_daemon()[source]

Start a daemon runner for the currently configured profile

aiida.daemon.runner.tick_legacy_workflows(runner, interval=30)[source]

Function that will call the legacy workflow stepper and ask the runner to call the same function back after a certain interval, essentially polling the worklow stepper

Parameters:
  • runner – the DaemonRunner instance to perform the callback
  • interval – the number of seconds to wait between callbacks
aiida.daemon.timestamps.get_timestamp_workflow_stepper(when='stop')[source]

Return the timestamp stored in the DbSettings table for the workflow_stepper task

Parameters:when – can either be ‘start’ or ‘stop’, to get when the task started or stopped
Returns:a datetime.datetime object. Return None if no information is found in the DB.
aiida.daemon.timestamps.set_timestamp_workflow_stepper(when)[source]

Store the current timestamp in the DbSettings for the workflow_stepper task

Parameters:when – can either be ‘start’ or ‘stop’, to set when the task started or stopped
aiida.daemon.workflowmanager.advance_workflow(w, step)[source]

The method tries to advance a step running its next method and handling possible errors.

If the method to advance is the Workflow exit method and there are no more steps RUNNING or in ERROR state then the workflow is set to FINISHED, otherwise an error is added to the report and the Workflow is flagged as ERROR.

If the method is the wf_default_call this means the step had no next, and possibly is part of a branching. In this case the Workflow is not advanced but the method returns True to let the other steps kick in.

Finally the methos tries to load the Workflow and execute the selected step, reporting the errors and the stack trace in the report in case of problems. Is no errors are reported the method returns True, in all the other cases the Workflow is set to ERROR state and the method returns False.

Parameters:
  • w – Workflow object to advance
  • step – DbWorkflowStep to execute
Returns:

True if the step has been executed, False otherwise

aiida.daemon.workflowmanager.execute_steps()[source]

This method loops on the RUNNING workflows and handled the execution of the steps until each workflow reaches an end (or gets stopped for errors).

In the loop for each RUNNING workflow the method loops also in each of its RUNNING steps, testing if all the calculation and subworkflows attached to the step are FINISHED. In this case the step is set as FINISHED and the workflow is advanced to the step’s next method present in the db with advance_workflow, otherwise if any step’s JobCalculation is found in NEW state the method will submit. If none of the previous conditions apply the step is flagged as ERROR and cannot proceed anymore, blocking the future execution of the step and, connected, the workflow.

Finally, for each workflow the method tests if there are INITIALIZED steps to be launched, and in case reloads the workflow and execute the specific those steps. In case or error the step is flagged in ERROR state and the stack is reported in the workflow report.