aiida.tools.importexport.common package

Common utility functions, classes, and exceptions

class aiida.tools.importexport.common.Archive(filepath)[source]

Bases: object

Utility class to operate on exported archive files or directories.

The main usage should be to construct the class with the filepath of the export archive as an argument. The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance within a context and which will be automatically cleaned upon leaving that context. Example:

with Archive('/some/path/archive.aiida') as archive:
    archive.version
FILENAME_DATA = 'data.json'
FILENAME_METADATA = 'metadata.json'
__dict__ = mappingproxy({'__module__': 'aiida.tools.importexport.common.archive', '__doc__': "Utility class to operate on exported archive files or directories.\n\n The main usage should be to construct the class with the filepath of the export archive as an argument.\n The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance\n within a context and which will be automatically cleaned upon leaving that context. Example::\n\n with Archive('/some/path/archive.aiida') as archive:\n archive.version\n\n ", 'FILENAME_DATA': 'data.json', 'FILENAME_METADATA': 'metadata.json', '__init__': <function Archive.__init__>, '__enter__': <function Archive.__enter__>, '__exit__': <function Archive.__exit__>, 'ensure_within_context': <FunctionWrapper at 0x7f04e2f2e9a0 for function>, 'ensure_unpacked': <FunctionWrapper at 0x7f04e2f2ea08 for function>, 'unpack': <FunctionWrapper at 0x7f04e2f2ea70 for function>, 'filepath': <property object>, 'folder': <property object>, 'data': <property object>, 'meta_data': <property object>, 'unpacked': <property object>, 'get_info': <FunctionWrapper at 0x7f04e2f2ec10 for function>, 'get_data_statistics': <FunctionWrapper at 0x7f04e2f2ec78 for function>, 'version_aiida': <property object>, 'version_format': <property object>, 'conversion_info': <property object>, '_read_json_file': <FunctionWrapper at 0x7f04e2f2ee80 for FunctionWrapper>, '__dict__': <attribute '__dict__' of 'Archive' objects>, '__weakref__': <attribute '__weakref__' of 'Archive' objects>})
__enter__()[source]

Instantiate a SandboxFolder into which the archive can be lazily unpacked.

__exit__(exc_type, exc_value, traceback)[source]

Clean the sandbox folder if it was instatiated.

__init__(filepath)[source]

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.tools.importexport.common.archive'
__weakref__

list of weak references to the object (if defined)

_read_json_file(filename)[source]

Read the contents of a JSON file from the unpacked archive contents.

Parameters:filename – the filename relative to the sandbox folder
Returns:a dictionary with the loaded JSON content
conversion_info

Return information about migration events that were applied to this archive.

Returns:list of conversion notifications
data

Return the loaded content of the data file

Returns:dictionary with contents of data file
ensure_unpacked(instance, args, kwargs)[source]

Decorator to ensure that the archive is unpacked before entering the decorated function.

ensure_within_context(instance, args, kwargs)[source]

Decorator to ensure that the instance is called within a context manager.

filepath

Return the filepath of the archive

Returns:the archive filepath
folder

Return the sandbox folder

Returns:sandbox folder aiida.common.folders.SandboxFolder
get_data_statistics()[source]

Return dictionary with statistics about data content, i.e. how many entries of each entity type it contains.

Returns:a dictionary with basic details
get_info()[source]

Return a dictionary with basic information about the archive.

Returns:a dictionary with basic details
meta_data

Return the loaded content of the meta data file

Returns:dictionary with contents of meta data file
unpack()[source]

Unpack the archive and store the contents in a sandbox.

unpacked

Return whether the archive has been unpacked into the sandbox folder.

version_aiida

Return the version of AiiDA the archive was created with.

Returns:version number
version_format

Return the version of the archive format.

Returns:version number
aiida.tools.importexport.common.extract_zip(infile, folder, nodes_export_subfolder=None, silent=False)[source]

Extract the nodes to be imported from a zip file.

Parameters:
  • infile (str) – file path
  • folder (SandboxFolder) – a temporary folder used to extract the file tree
  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
  • silent (bool) – suppress debug print
Raises:
  • TypeError – if parameter types are not respected
  • CorruptArchive – if the archive misses files or files have incorrect formats
aiida.tools.importexport.common.extract_tar(infile, folder, nodes_export_subfolder=None, silent=False)[source]

Extract the nodes to be imported from a (possibly zipped) tar file.

Parameters:
  • infile (str) – file path
  • folder (SandboxFolder) – a temporary fodler used to extract the file tree
  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
  • silent (bool) – suppress debug print
Raises:
  • TypeError – if parameter types are not respected
  • CorruptArchive – if the archive misses files or files have incorrect formats
aiida.tools.importexport.common.extract_tree(infile, folder)[source]

Prepare to import nodes from plain file system tree.

Parameters:
  • infile (str) – path
  • folder (SandboxFolder) – a temporary folder used to extract the file tree
exception aiida.tools.importexport.common.ExportImportException[source]

Bases: aiida.common.exceptions.AiidaException

Base class for all AiiDA export/import module exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ArchiveExportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ArchiveImportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA import exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.CorruptArchive[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when an operation is applied to a corrupt export archive, e.g. missing files or invalid formats.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.IncompatibleArchiveVersionError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when trying to import an export archive with an incompatible schema version.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ExportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveExportError

Raised when validation fails during export, e.g. for non-sealed ProcessNode s.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ImportUniquenessError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when the user tries to violate a uniqueness constraint.

Similar to UniquenessError.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ImportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when validation fails during import, e.g. for parameter types and values.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.ArchiveMigrationError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export archive migration exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.MigrationValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveMigrationError

Raised when validation fails during migration of export archives.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.DanglingLinkError[source]

Bases: aiida.tools.importexport.common.exceptions.MigrationValidationError

Raised when an export archive is detected to contain dangling links when importing.

__module__ = 'aiida.tools.importexport.common.exceptions'

Submodules

Utility functions and classes to interact with AiiDA export archives.

class aiida.tools.importexport.common.archive.Archive(filepath)[source]

Bases: object

Utility class to operate on exported archive files or directories.

The main usage should be to construct the class with the filepath of the export archive as an argument. The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance within a context and which will be automatically cleaned upon leaving that context. Example:

with Archive('/some/path/archive.aiida') as archive:
    archive.version
FILENAME_DATA = 'data.json'
FILENAME_METADATA = 'metadata.json'
__dict__ = mappingproxy({'__module__': 'aiida.tools.importexport.common.archive', '__doc__': "Utility class to operate on exported archive files or directories.\n\n The main usage should be to construct the class with the filepath of the export archive as an argument.\n The contents will be lazily unpacked into a sand box folder which is constructed upon entering the instance\n within a context and which will be automatically cleaned upon leaving that context. Example::\n\n with Archive('/some/path/archive.aiida') as archive:\n archive.version\n\n ", 'FILENAME_DATA': 'data.json', 'FILENAME_METADATA': 'metadata.json', '__init__': <function Archive.__init__>, '__enter__': <function Archive.__enter__>, '__exit__': <function Archive.__exit__>, 'ensure_within_context': <FunctionWrapper at 0x7f04e2f2e9a0 for function>, 'ensure_unpacked': <FunctionWrapper at 0x7f04e2f2ea08 for function>, 'unpack': <FunctionWrapper at 0x7f04e2f2ea70 for function>, 'filepath': <property object>, 'folder': <property object>, 'data': <property object>, 'meta_data': <property object>, 'unpacked': <property object>, 'get_info': <FunctionWrapper at 0x7f04e2f2ec10 for function>, 'get_data_statistics': <FunctionWrapper at 0x7f04e2f2ec78 for function>, 'version_aiida': <property object>, 'version_format': <property object>, 'conversion_info': <property object>, '_read_json_file': <FunctionWrapper at 0x7f04e2f2ee80 for FunctionWrapper>, '__dict__': <attribute '__dict__' of 'Archive' objects>, '__weakref__': <attribute '__weakref__' of 'Archive' objects>})
__enter__()[source]

Instantiate a SandboxFolder into which the archive can be lazily unpacked.

__exit__(exc_type, exc_value, traceback)[source]

Clean the sandbox folder if it was instatiated.

__init__(filepath)[source]

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.tools.importexport.common.archive'
__weakref__

list of weak references to the object (if defined)

_read_json_file(filename)[source]

Read the contents of a JSON file from the unpacked archive contents.

Parameters:filename – the filename relative to the sandbox folder
Returns:a dictionary with the loaded JSON content
conversion_info

Return information about migration events that were applied to this archive.

Returns:list of conversion notifications
data

Return the loaded content of the data file

Returns:dictionary with contents of data file
ensure_unpacked(instance, args, kwargs)[source]

Decorator to ensure that the archive is unpacked before entering the decorated function.

ensure_within_context(instance, args, kwargs)[source]

Decorator to ensure that the instance is called within a context manager.

filepath

Return the filepath of the archive

Returns:the archive filepath
folder

Return the sandbox folder

Returns:sandbox folder aiida.common.folders.SandboxFolder
get_data_statistics()[source]

Return dictionary with statistics about data content, i.e. how many entries of each entity type it contains.

Returns:a dictionary with basic details
get_info()[source]

Return a dictionary with basic information about the archive.

Returns:a dictionary with basic details
meta_data

Return the loaded content of the meta data file

Returns:dictionary with contents of meta data file
unpack()[source]

Unpack the archive and store the contents in a sandbox.

unpacked

Return whether the archive has been unpacked into the sandbox folder.

version_aiida

Return the version of AiiDA the archive was created with.

Returns:version number
version_format

Return the version of the archive format.

Returns:version number
aiida.tools.importexport.common.archive.extract_zip(infile, folder, nodes_export_subfolder=None, silent=False)[source]

Extract the nodes to be imported from a zip file.

Parameters:
  • infile (str) – file path
  • folder (SandboxFolder) – a temporary folder used to extract the file tree
  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
  • silent (bool) – suppress debug print
Raises:
  • TypeError – if parameter types are not respected
  • CorruptArchive – if the archive misses files or files have incorrect formats
aiida.tools.importexport.common.archive.extract_tar(infile, folder, nodes_export_subfolder=None, silent=False)[source]

Extract the nodes to be imported from a (possibly zipped) tar file.

Parameters:
  • infile (str) – file path
  • folder (SandboxFolder) – a temporary fodler used to extract the file tree
  • nodes_export_subfolder (str) – name of the subfolder for AiiDA nodes
  • silent (bool) – suppress debug print
Raises:
  • TypeError – if parameter types are not respected
  • CorruptArchive – if the archive misses files or files have incorrect formats
aiida.tools.importexport.common.archive.extract_tree(infile, folder)[source]

Prepare to import nodes from plain file system tree.

Parameters:
  • infile (str) – path
  • folder (SandboxFolder) – a temporary folder used to extract the file tree

Configuration file for AiiDA Import/Export module

Module that defines the exceptions thrown by AiiDA’s export/import module.

Note: In order to not override the built-in ImportError, both ImportError and ExportError are prefixed with
Archive.
exception aiida.tools.importexport.common.exceptions.ExportImportException[source]

Bases: aiida.common.exceptions.AiidaException

Base class for all AiiDA export/import module exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ArchiveExportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ArchiveImportError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA import exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.CorruptArchive[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when an operation is applied to a corrupt export archive, e.g. missing files or invalid formats.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.IncompatibleArchiveVersionError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Raised when trying to import an export archive with an incompatible schema version.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ExportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveExportError

Raised when validation fails during export, e.g. for non-sealed ProcessNode s.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ImportUniquenessError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when the user tries to violate a uniqueness constraint.

Similar to UniquenessError.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ImportValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveImportError

Raised when validation fails during import, e.g. for parameter types and values.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.ArchiveMigrationError[source]

Bases: aiida.tools.importexport.common.exceptions.ExportImportException

Base class for all AiiDA export archive migration exceptions.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.MigrationValidationError[source]

Bases: aiida.tools.importexport.common.exceptions.ArchiveMigrationError

Raised when validation fails during migration of export archives.

__module__ = 'aiida.tools.importexport.common.exceptions'
exception aiida.tools.importexport.common.exceptions.DanglingLinkError[source]

Bases: aiida.tools.importexport.common.exceptions.MigrationValidationError

Raised when an export archive is detected to contain dangling links when importing.

__module__ = 'aiida.tools.importexport.common.exceptions'

Utility functions for import/export of AiiDA entities

class aiida.tools.importexport.common.utils.HTMLGetLinksParser(filter_extension=None)[source]

Bases: html.parser.HTMLParser

If a filter_extension is passed, only links with extension matching the given one will be returned.

__init__(filter_extension=None)[source]

Initialize and reset this instance.

If convert_charrefs is True (the default), all character references are automatically converted to the corresponding Unicode characters.

__module__ = 'aiida.tools.importexport.common.utils'

Return the links that were found during the parsing phase.

handle_starttag(tag, attrs)[source]

Store the urls encountered, if they match the request.

aiida.tools.importexport.common.utils.export_shard_uuid(uuid)[source]

Sharding of the UUID for the import/export

Parameters:uuid (str) – UUID to be sharded (v4)
Returns:Sharded UUID as a subfolder path
Return type:str

Open the given URL, parse the HTML and return a list of valid links where the link file has a .aiida extension.

aiida.tools.importexport.common.utils.schema_to_entity_names(class_string)[source]

Mapping from classes path to entity names (used by the SQLA import/export) This could have been written much simpler if it is only for SQLA but there is an attempt the SQLA import/export code to be used for Django too.