aiida.common documentation

Calculation datastructures

This module defines the main data structures used by the Calculation.

class aiida.common.datastructures.CalcInfo(init={})[source]

This object will store the data returned by the calculation plugin and to be passed to the ExecManager

class aiida.common.datastructures.CodeInfo(init={})[source]

This attribute-dictionary contains the information needed to execute a code. Possible attributes are:

  • cmdline_params: a list of strings, containing parameters to be written on the command line right after the call to the code, as for example:

    code.x cmdline_params[0] cmdline_params[1] ... < stdin > stdout
    
  • stdin_name: (optional) the name of the standard input file. Note, it is only possible to use the stdin with the syntax:

    code.x < stdin_name
    

    If no stdin_name is specified, the string “< stdin_name” will not be passed to the code. Note: it is not possible to substitute/remove the ‘<’ if stdin_name is specified; if that is needed, avoid stdin_name and use instead the cmdline_params to specify a suitable syntax.

  • stdout_name: (optional) the name of the standard output file. Note, it is only possible to pass output to stdout_name with the syntax:

    code.x ... > stdout_name
    

    If no stdout_name is specified, the string “> stdout_name” will not be passed to the code. Note: it is not possible to substitute/remove the ‘>’ if stdout_name is specified; if that is needed, avoid stdout_name and use instead the cmdline_params to specify a suitable syntax.

  • stderr_name: (optional) a string, the name of the error file of the code.

  • join_files: (optional) if True, redirects the error to the output file. If join_files=True, the code will be called as:

    code.x ... > stdout_name 2>&1
    

    otherwise, if join_files=False and stderr is passed:

    code.x ... > stdout_name 2> stderr_name
    
  • withmpi: if True, executes the code with mpirun (or another MPI installed on the remote computer)

  • code_uuid: the uuid of the code associated to the CodeInfo

aiida.common.datastructures.sort_states(list_states)[source]

Given a list of state names, return a sorted list of states (the first is the most recent) sorted according to their logical appearance in the DB (i.e., NEW before of SUBMITTING before of FINISHED).

Note

The order of the internal variable _sorted_datastates is used.

Parameters:list_states – a list (or tuple) of state strings.
Returns:a sorted list of the given data states.
Raises ValueError:
 if any of the given states is not a valid state.

Exceptions

exception aiida.common.exceptions.AiidaException[source]

Base class for all AiiDA exceptions.

Each module will have its own subclass, inherited from this (e.g. ExecManagerException, TransportException, ...)

exception aiida.common.exceptions.AuthenticationError[source]

Raised when a user tries to access a resource for which it is not authenticated, e.g. an aiidauser tries to access a computer for which there is no entry in the AuthInfo table.

exception aiida.common.exceptions.ConfigurationError[source]

Error raised when there is a configuration error in AiiDA.

exception aiida.common.exceptions.ContentNotExistent[source]

Raised when trying to access an attribute, a key or a file in the result nodes that is not present

exception aiida.common.exceptions.DbContentError[source]

Raised when the content of the DB is not valid. This should never happen if the user does not play directly with the DB.

exception aiida.common.exceptions.FailedError[source]

Raised when accessing a calculation that is in the FAILED status

exception aiida.common.exceptions.FeatureDisabled[source]

Raised when a feature is requested, but the user has chosen to disable it (e.g., for submissions on disabled computers).

exception aiida.common.exceptions.FeatureNotAvailable[source]

Raised when a feature is requested from a plugin, that is not available.

exception aiida.common.exceptions.InputValidationError[source]

The input data for a calculation did not validate (e.g., missing required input data, wrong data, ...)

exception aiida.common.exceptions.InternalError[source]

Error raised when there is an internal error of AiiDA.

exception aiida.common.exceptions.InvalidOperation[source]

The allowed operation is not valid (e.g., when trying to add a non-internal attribute before saving the entry), or deleting an entry that is protected (e.g., because it is referenced by foreign keys)

exception aiida.common.exceptions.LicensingException[source]

Raised when requirements for data licensing are not met.

exception aiida.common.exceptions.LockPresent[source]

Raised when a lock is requested, but cannot be acquired.

exception aiida.common.exceptions.MissingPluginError[source]

Raised when the user tries to use a plugin that is not available or does not exist.

exception aiida.common.exceptions.ModificationNotAllowed[source]

Raised when the user tries to modify a field, object, property, ... that should not be modified.

exception aiida.common.exceptions.MultipleObjectsError[source]

Raised when more than one entity is found in the DB, but only one was expected.

exception aiida.common.exceptions.NotExistent[source]

Raised when the required entity does not exist.

exception aiida.common.exceptions.ParsingError[source]

Generic error raised when there is a parsing error

exception aiida.common.exceptions.PluginInternalError[source]

Error raised when there is an internal error which is due to a plugin and not to the AiiDA infrastructure.

exception aiida.common.exceptions.ProfileConfigurationError[source]

Configuration error raised when a wrong/inexistent profile is requested.

exception aiida.common.exceptions.RemoteOperationError[source]

Raised when an error in a remote operation occurs, as in a failed kill() of a scheduler job.

exception aiida.common.exceptions.UniquenessError[source]

Raised when the user tries to violate a uniqueness constraint (on the DB, for instance).

exception aiida.common.exceptions.ValidationError[source]

Error raised when there is an error during the validation phase of a property.

exception aiida.common.exceptions.WorkflowInputValidationError[source]

The input data for a workflow did not validate (e.g., missing required input data, wrong data, ...)

Extended dictionaries

class aiida.common.extendeddicts.AttributeDict(init={})[source]

This class internally stores values in a dictionary, but exposes the keys also as attributes, i.e. asking for attrdict.key will return the value of attrdict[‘key’] and so on.

Raises an AttributeError if the key does not exist, when called as an attribute, while the usual KeyError if the key does not exist and the dictionary syntax is used.

copy()[source]

Shallow copy.

class aiida.common.extendeddicts.DefaultFieldsAttributeDict(init={})[source]

A dictionary with access to the keys as attributes, and with an internal value storing the ‘default’ keys to be distinguished from extra fields.

Extra methods defaultkeys() and extrakeys() divide the set returned by keys() in default keys (i.e. those defined at definition time) and other keys. There is also a method get_default_fields() to return the internal list.

Moreover, for undefined default keys, it returns None instead of raising a KeyError/AttributeError exception.

Remember to define the _default_fields in a subclass! E.g.:

class TestExample(DefaultFieldsAttributeDict):
    _default_fields = ('a','b','c')

When the validate() method is called, it calls in turn all validate_KEY methods, where KEY is one of the default keys. If the method is not present, the field is considered to be always valid. Each validate_KEY method should accept a single argument ‘value’ that will contain the value to be checked.

It raises a ValidationError if any of the validate_KEY function raises an exception, otherwise it simply returns. NOTE: the validate_ functions are called also for unset fields, so if the field can be empty on validation, you have to start your validation function with something similar to:

if value is None:
    return

Todo

Decide behavior if I set to None a field. Current behavior, if a is an instance and ‘def_field’ one of the default fields, that is undefined, we get:

  • a.get('def_field'): None
  • a.get('def_field','whatever'): ‘whatever’
  • Note that a.defaultkeys() does NOT contain ‘def_field’

if we do a.def_field = None, then the behavior becomes

  • a.get('def_field'): None
  • a.get('def_field','whatever'): None
  • Note that a.defaultkeys() DOES contain ‘def_field’

See if we want that setting a default field to None means deleting it.

defaultkeys()[source]

Return the default keys defined in the instance.

extrakeys()[source]

Return the extra keys defined in the instance.

classmethod get_default_fields()[source]

Return the list of default fields, either defined in the instance or not.

validate()[source]

Validate the keys, if any validate_* method is available.

class aiida.common.extendeddicts.FixedFieldsAttributeDict(init={})[source]

A dictionary with access to the keys as attributes, and with filtering of valid attributes. This is only the base class, without valid attributes; use a derived class to do the actual work. E.g.:

class TestExample(FixedFieldsAttributeDict):
    _valid_fields = ('a','b','c')
classmethod get_valid_fields()[source]

Return the list of valid fields.

Folders

class aiida.common.folders.Folder(abspath, folder_limit=None)[source]

A class to manage generic folders, avoiding to get out of specific given folder borders.

Todo

fix this, os.path.commonprefix of /a/b/c and /a/b2/c will give a/b, check if this is wanted or if we want to put trailing slashes. (or if we want to use os.path.relpath and check for a string starting with os.pardir?)

Todo

rethink whether the folder_limit option is still useful. If not, remove it alltogether (it was a nice feature, but unfortunately all the calls to os.path.abspath or normpath are quite slow).

abspath

The absolute path of the folder.

create()[source]

Creates the folder, if it does not exist on the disk yet.

It will also create top directories, if absent.

It is always safe to call it, it will do nothing if the folder already exists.

create_file_from_filelike(src_filelike, dest_name)[source]

Create a file from a file-like object.

Note:

if the current file position in src_filelike is not 0, only the contents from the current file position to the end of the file will be copied in the new file.

Parameters:
  • src_filelike – the file-like object (e.g., if you have a string called s, you can pass StringIO.StringIO(s))
  • dest_name – the destination filename will have this file name.

Create a symlink inside the folder to the location ‘src’.

Parameters:
  • src – the location to which the symlink must point. Can be either a relative or an absolute path. Should, however, be relative to work properly also when the repository is moved!
  • name – the filename of the symlink to be created.
erase(create_empty_folder=False)[source]

Erases the folder. Should be called only in very specific cases, in general folder should not be erased!

Doesn’t complain if the folder does not exist.

Parameters:create_empty_folder – if True, after erasing, creates an empty dir.
exists()[source]

Return True if the folder exists, False otherwise.

folder_limit

The folder limit that cannot be crossed when creating files and folders.

get_abs_path(relpath, check_existence=False)[source]

Return an absolute path for a file or folder in this folder.

The advantage of using this method is that it checks that filename is a valid filename within this folder, and not something e.g. containing slashes.

Parameters:
  • filename – The file or directory.
  • check_existence – if False, just return the file path. Otherwise, also check if the file or directory actually exists. Raise OSError if it does not.
get_content_list(pattern='*', only_paths=True)[source]

Return a list of files (and subfolders) in the folder, matching a given pattern.

Example: If you want to exclude files starting with a dot, you can call this method with pattern='[!.]*'

Parameters:
  • pattern – a pattern for the file/folder names, using Unix filename pattern matching (see Python standard module fnmatch). By default, pattern is ‘*’, matching all files and folders.
  • only_paths – if False (default), return pairs (name, is_file). if True, return only a flat list.
Returns:

a list of tuples of two elements, the first is the file name and the second is True if the element is a file, False if it is a directory.

get_subfolder(subfolder, create=False, reset_limit=False)[source]

Return a Folder object pointing to a subfolder.

Parameters:
  • subfolder – a string with the relative path of the subfolder, relative to the absolute path of this object. Note that this may also contain ‘..’ parts, as far as this does not go beyond the folder_limit.
  • create – if True, the new subfolder is created, if it does not exist.
  • reset_limit – when doing b = a.get_subfolder('xxx', reset_limit=False), the limit of b will be the same limit of a. if True, the limit will be set to the boundaries of folder b.
Returns:

a Folder object pointing to the subfolder.

insert_path(src, dest_name=None, overwrite=True)[source]

Copy a file to the folder.

Parameters:
  • src – the source filename to copy
  • dest_name – if None, the same basename of src is used. Otherwise, the destination filename will have this file name.
  • overwrite – if False, raises an error on existing destination; otherwise, delete it first.
isdir(relpath)[source]

Return True if ‘relpath’ exists inside the folder and is a directory, False otherwise.

isfile(relpath)[source]

Return True if ‘relpath’ exists inside the folder and is a file, False otherwise.

mode_dir

Return the mode with which the folders should be created

mode_file

Return the mode with which the files should be created

open(name, mode='r')[source]

Open a file in the current folder and return the corresponding file object.

remove_path(filename)[source]

Remove a file or folder from the folder.

Parameters:filename – the relative path name to remove
replace_with_folder(srcdir, move=False, overwrite=False)[source]

This routine copies or moves the source folder ‘srcdir’ to the local folder pointed by this Folder object.

Parameters:
  • srcdir – the source folder on the disk; this must be a string with an absolute path
  • move – if True, the srcdir is moved to the repository. Otherwise, it is only copied.
  • overwrite – if True, the folder will be erased first. if False, a IOError is raised if the folder already exists. Whatever the value of this flag, parent directories will be created, if needed.
Raises:

OSError or IOError: in case of problems accessing or writing the files.

Raises:

ValueError: if the section is not recognized.

class aiida.common.folders.RepositoryFolder(section, uuid, subfolder='.')[source]

A class to manage the local AiiDA repository folders.

get_topdir()[source]

Returns the top directory, i.e., the section/uuid folder object.

section

The section to which this folder belongs.

subfolder

The subfolder within the section/uuid folder.

uuid

The uuid to which this folder belongs.

class aiida.common.folders.SandboxFolder[source]

A class to manage the creation and management of a sandbox folder.

Note: this class must be used within a context manager, i.e.:

with SandboxFolder as f:
## do something with f

In this way, the sandbox folder is removed from disk (if it wasn’t removed already) when exiting the ‘with’ block.

Todo

Implement check of whether the folder has been removed.

Plugin loaders

aiida.common.pluginloader.BaseFactory(module, base_class, base_modname, suffix=None)[source]

Return a given subclass of Calculation, loading the correct plugin.

Example:

If module=’quantumespresso.pw’, base_class=JobCalculation, base_modname = ‘aiida.orm.calculation.job’, and suffix=’Calculation’, the code will first look for a pw subclass of JobCalculation inside the quantumespresso module. Lacking such a class, it will try to look for a ‘PwCalculation’ inside the quantumespresso.pw module. In the latter case, the plugin class must have a specific name and be located in a specific file: if for instance plugin_name == ‘ssh’ and base_class.__name__ == ‘Transport’, then there must be a class named ‘SshTransport’ which is a subclass of base_class in a file ‘ssh.py’ in the plugins_module folder. To create the class name to look for, the code will attach the string passed in the base_modname (after the last dot) and the suffix parameter, if passed, with the proper CamelCase capitalization. If suffix is not passed, the default suffix that is used is the base_class class name.

Parameters:
  • module – a string with the module of the plugin to load, e.g. ‘quantumespresso.pw’.
  • base_class – a base class from which the returned class should inherit. e.g.: JobCalculation
  • base_modname – a basic module name, under which the module should be found. E.g., ‘aiida.orm.calculation.job’.
  • suffix – If specified, the suffix that the class name will have. By default, use the name of the base_class.
aiida.common.pluginloader.existing_plugins(base_class, plugins_module_name, max_depth=5, suffix=None)[source]

Return a list of strings of valid plugins.

Parameters:
  • base_class – Identify all subclasses of the base_class
  • plugins_module_name – a string with the full module name separated with dots that points to the folder with plugins. It must be importable by python.
  • max_depth – Maximum depth (of nested modules) to be used when looking for plugins
  • suffix – The suffix that is appended to the basename when looking for the (sub)class name. If not provided (or None), use the base class name.
Returns:

a list of valid strings that can be used using a Factory or with load_plugin.

aiida.common.pluginloader.get_class_typestring(type_string)[source]

Given the type string, return three strings: the first one is one of the first-level classes that the Node can be: “node”, “calculation”, “code”, “data”. The second string is the one that can be passed to the DataFactory or CalculationFactory (or an empty string for nodes and codes); the third one is the name of the python class that would be loaded.

aiida.common.pluginloader.load_plugin(base_class, plugins_module, plugin_type)[source]

Load a specific plugin for the given base class.

This is general and works for any plugin used in AiiDA.

NOTE: actually, now plugins_module and plugin_type are joined with a dot,
and the plugin is retrieved splitting using the last dot of the resulting string.
TODO: understand if it is probably better to join the two parameters above
to a single one.
Args:
base_class
the abstract base class of the plugin.
plugins_module
a string with the full module name separated with dots that points to the folder with plugins. It must be importable by python.
plugin_type
the name of the plugin.
Return:
the class of the required plugin.
Raise:
MissingPluginError if the plugin cannot be loaded
Example:
plugin_class = load_plugin(
aiida.transport.Transport,’aiida.transport.plugins’,’ssh.SshTransport’)

and plugin_class will be the class ‘aiida.transport.plugins.ssh.SshTransport’

Utilities

class aiida.common.utils.classproperty(getter)[source]

A class that, when used as a decorator, works as if the two decorators @property and @classmethod where applied together (i.e., the object works as a property, both for the Class and for any of its instance; and is called with the class cls rather than with the instance as its first argument).

aiida.common.utils.conv_to_fortran(val)[source]
Parameters:val – the value to be read and converted to a Fortran-friendly string.
aiida.common.utils.create_display_name(field)[source]

Given a string, creates the suitable “default” display name: replace underscores with spaces, and capitalize each word.

Returns:the converted string
aiida.common.utils.escape_for_bash(str_to_escape)[source]

This function takes any string and escapes it in a way that bash will interpret it as a single string.

Explanation:

At the end, in the return statement, the string is put within single quotes. Therefore, the only thing that I have to escape in bash is the single quote character. To do this, I substitute every single quote ‘ with ‘”’”’ which means:

First single quote: exit from the enclosing single quotes

Second, third and fourth character: “’” is a single quote character, escaped by double quotes

Last single quote: reopen the single quote to continue the string

Finally, note that for python I have to enclose the string ‘”’”’ within triple quotes to make it work, getting finally: the complicated string found below.

aiida.common.utils.export_shard_uuid(uuid)[source]

Sharding of the UUID for the import/export

aiida.common.utils.get_class_string(obj)[source]

Return the string identifying the class of the object (module + object name, joined by dots).

It works both for classes and for class instances.

aiida.common.utils.get_extremas_from_positions(positions)[source]

returns the minimum and maximum value for each dimension in the positions given

aiida.common.utils.get_new_uuid()[source]

Return a new UUID (typically to be used for new nodes). It uses the UUID version specified in aiida.djsite.settings.settings_profile.AIIDANODES_UUID_VERSION

aiida.common.utils.get_object_from_string(string)[source]

Given a string identifying an object (as returned by the get_class_string method) load and return the actual object.

aiida.common.utils.get_repository_folder(subfolder=None)[source]

Return the top folder of the local repository.

aiida.common.utils.get_suggestion(provided_string, allowed_strings)[source]

Given a string and a list of allowed_strings, it returns a string to print on screen, with sensible text depending on whether no suggestion is found, or one or more than one suggestions are found.

Args:
provided_string: the string to compare allowed_strings: a list of valid strings
Returns:
A string to print on output, to suggest to the user a possible valid value.
aiida.common.utils.get_unique_filename(filename, list_of_filenames)[source]

Return a unique filename that can be added to the list_of_filenames.

If filename is not in list_of_filenames, it simply returns the filename string itself. Otherwise, it appends a integer number to the filename (before the extension) until it finds a unique filename.

Parameters:
  • filename – the filename to add
  • list_of_filenames – the list of filenames to which filename should be added, without name duplicates
Returns:

Either filename or its modification, with a number appended between the name and the extension.

aiida.common.utils.grouper(n, iterable)[source]

Given an iterable, returns an iterable that returns tuples of groups of elements from iterable of length n, except the last one that has the required length to exaust iterable (i.e., there is no filling applied).

Parameters:
  • n – length of each tuple (except the last one,that will have length <= n
  • iterable – the iterable to divide in groups
aiida.common.utils.gunzip_string(string)[source]

Gunzip string contents.

Parameters:string – a gzipped string
Returns:a string
aiida.common.utils.gzip_string(string)[source]

Gzip string contents.

Parameters:string – a string
Returns:a gzipped string
aiida.common.utils.md5_file(filename, block_size_factor=128)[source]

Open a file and return its md5sum (hexdigested).

Parameters:
  • filename – the filename of the file for which we want the md5sum
  • block_size_factor – the file is read at chunks of size block_size_factor * md5.block_size, where md5.block_size is the block_size used internally by the hashlib module.
Returns:

a string with the hexdigest md5.

Raises:

No checks are done on the file, so if it doesn’t exists it may raise IOError.

aiida.common.utils.sha1_file(filename, block_size_factor=128)[source]

Open a file and return its sha1sum (hexdigested).

Parameters:
  • filename – the filename of the file for which we want the sha1sum
  • block_size_factor – the file is read at chunks of size block_size_factor * sha1.block_size, where sha1.block_size is the block_size used internally by the hashlib module.
Returns:

a string with the hexdigest sha1.

Raises:

No checks are done on the file, so if it doesn’t exists it may raise IOError.

aiida.common.utils.str_timedelta(dt, max_num_fields=3, short=False, negative_to_zero=False)[source]

Given a dt in seconds, return it in a HH:MM:SS format.

Parameters:
  • dt – a TimeDelta object
  • max_num_fields – maximum number of non-zero fields to show (for instance if the number of days is non-zero, shows only days, hours and minutes, but not seconds)
  • short – if False, print always max_num_fields fields, even if they are zero. If True, do not print the first fields, if they are zero.
  • negative_to_zero – if True, set dt = 0 if dt < 0.
aiida.common.utils.validate_list_of_string_tuples(val, tuple_length)[source]

Check that:

  1. val is a list or tuple
  2. each element of the list:
  1. is a list or tuple
  2. is of length equal to the parameter tuple_length
  3. each of the two elements is a string

Return if valid, raise ValidationError if invalid

aiida.common.utils.xyz_parser_iterator(string)[source]

Yields a tuple (natoms, comment, atomiter)`for each frame in a XYZ file where `atomiter is an iterator yielding a nested tuple (symbol, (x, y, z)) for each entry.

Parameters:string – a string containing XYZ-structured text