Readers and writers for archive formats, that work independently of a connection to an AiiDA profile.
aiida.tools.importexport.archive.
ArchiveMetadata
Bases: object
object
Class for storing metadata about this archive.
Required fields are necessary for importing the data back into AiiDA, whereas optional fields capture information about the export/migration process(es)
__annotations__
__dataclass_fields__
__dataclass_params__
__dict__
__eq__
Return self==value.
__hash__
__init__
Initialize self. See help(type(self)) for accurate signature.
__module__
__repr__
Return repr(self).
__weakref__
list of weak references to the object (if defined)
aiida_version
all_fields_info
conversion_info
entities_starting_set
export_version
graph_traversal_rules
include_comments
include_logs
unique_identifiers
ArchiveMigratorAbstract
Bases: abc.ABC
abc.ABC
An abstract base class to define an archive migrator.
__abstractmethods__
Initialise the migrator
filepath – the path to the archive file
version – the version of the archive file or, if None, the version will be auto-retrieved.
_abc_impl
filepath
Return the input file path.
migrate
Migrate the archive to another version
version – the version to migrate to
filename – the file path to migrate to. If None, the migrated archive will not be copied from the work_dir.
force – overwrite output file if it already exists
work_dir – The directory in which to perform the migration. If None, a temporary folder will be created and destroyed at the end of the process.
kwargs – key-word arguments specific to the concrete migrator implementation
path to the migrated archive or None if no migration performed (if filename is None, this will point to a path in the work_dir)
CorruptArchive: if the archive cannot be read
CorruptArchive
ArchiveMigrationError: if the archive cannot migrated to the requested version
ArchiveMigrationError
ArchiveMigratorJsonBase
Bases: aiida.tools.importexport.archive.migrators.ArchiveMigratorAbstract
aiida.tools.importexport.archive.migrators.ArchiveMigratorAbstract
A migrator base for the JSON compressed formats.
_compress_archive_tar
Create a new zip compressed tar from a folder.
_compress_archive_zip
Create a new zip compressed zip from a folder.
_extract_archive
Extract the archive to a filepath.
_move_file
Move a file to a another path, deleting the target path first if it exists.
_perform_migration
Perform the migration(s) in the work directory, compress (if necessary), then move to the out_path (if not None).
_retrieve_version
Retrieve the version of the input archive.
ArchiveMigratorJsonTar
Bases: aiida.tools.importexport.archive.migrators.ArchiveMigratorJsonBase
aiida.tools.importexport.archive.migrators.ArchiveMigratorJsonBase
A migrator for a JSON tar compressed format.
ArchiveMigratorJsonZip
A migrator for a JSON zip compressed format.
ArchiveReaderAbstract
An abstract interface for AiiDA archive readers.
An ArchiveReader implementation is intended to be used with a context:
ArchiveReader
with ArchiveReader(filename) as reader: reader.entity_count('Node')
__enter__
__exit__
An archive reader
filename – the filename (possibly including the absolute path) of the file to import.
assert_within_context
Assert that the method is called within a context.
~aiida.common.exceptions.InvalidOperation: if not called within a context
check_version
Check the version compatibility of the archive.
~aiida.tools.importexport.common.exceptions.IncompatibleArchiveVersionError: If the version is not compatible
compatible_export_version
Return the export version that this reader is compatible with.
entity_count
Return the count of an entity or None if not contained in the archive.
entity_names
Return list of all entity names.
Return the export version.
CorruptArchive – If the version cannot be retrieved.
file_format_verbose
The file format name.
filename
Return the name of the file that is being read from.
iter_entity_fields
Iterate over entities and yield their pk and database fields.
iter_group_uuids
Iterate over group UUIDs and the a set of node UUIDs they contain.
iter_link_data
Iterate over links: {‘input’: <UUID>, ‘output’: <UUID>, ‘label’: <LABEL>, ‘type’: <TYPE>}
iter_node_repos
Yield temporary folders containing the contents of the repository for each node.
uuids – UUIDs of the nodes over whose repository folders to iterate
callback –
a callback to report on the process, callback(action, value), with the following callback signatures:
callback(action, value)
callback('init', {'total': <int>, 'description': <str>})
to signal the start of a process, its total iterations and description
callback('update', <int>)
to signal an update to the process and the number of iterations to progress
CorruptArchive – If the repository does not exist.
iter_node_uuids
Iterate over node UUIDs.
link_count
Return the count of links.
metadata
Return the full (validated) archive metadata.
node_repository
Return a temporary folder with the contents of the repository for a single node.
uuid – The UUID of the node
ArchiveWriterAbstract
An abstract interface for AiiDA archive writers.
Open the contextmanager
Initiate the writer.
filepath – the path to the file to export to.
kwargs – keyword arguments specific to the writer implementation.
_remove_filepath
To run before moving the final export from a temporary folder.
add_export_info
Record information about the export process.
This information can be accessed by writer.export_info, it is reset on entrance of the context manager.
writer.export_info
close
Finalise the export.
This method is called on exiting a context manager.
excepted – Whether
export_info
Return information stored during the export process.
Return the file format name.
Return the filepath to write to.
open
Setup the export process.
This method is called on entering a context manager.
write_entity_data
Write the data for a single DB entity.
name – the name of the entity (e.g. ‘NODE’)
pk – the primary key of the entity (unique for the current DB only)
id_key – the key within fields that denotes the unique identifier of the entity (unique for all DBs)
fields
fields – mapping of DB fields to store -> values
write_group_nodes
Write a mapping of a group to the nodes it contains.
uuid – the UUID of the group
node_uuids – the list of node UUIDs the group contains
write_link
Write a dictionary of information for a single provenance link.
data – {'input': <UUID_STR>, 'output': <UUID_STR>, 'label': <LABEL_STR>, 'type': <TYPE_STR>}
{'input': <UUID_STR>, 'output': <UUID_STR>, 'label': <LABEL_STR>, 'type': <TYPE_STR>}
write_metadata
Write the metadata of the export process.
write_node_repo_folder
Write a node repository to the archive.
path – The path to the repository folder on disk
overwrite – Allow to overwrite existing path in archive
CacheFolder
A class to encapsulate a folder path with cached read/writes.
The class can be used as a context manager, and will flush the cache on exit:
with CacheFolder(path) as folder: # these are stored in memory (no disk write) folder.write_text('path/to/file.txt', 'content') folder.write_json('path/to/data.json', {'a': 1}) # these will be read from memory text = folder.read_text('path/to/file.txt') text = folder.load_json('path/to/data.json') # all files will now have been written to disk
Enter the contextmanager.
Exit the contextmanager.
Initialise cached folder.
path – folder path to cache
encoding – encoding of text to read/write
_limit_cache
Ensure the cache does not exceed a set limit.
Content is uncached on a First-In-First-Out basis.
_write_object
Write an object from the cache to disk.
path – relative path of file
ctype – the type of the content
content – the content to write
flush
Flush the cache.
get_path
Return the path.
flush – flush the cache before returning
load_json
Load a json file from the cache folder.
Important: if the dict is returned directly from the cache, any mutations will affect the cached dict.
path – path relative to base folder
ensure_copy – ensure the dict is a copy of that from the cache
(from cache, the content) If from cache, mutations will directly affect the cache
read_text
write text from the cache or base folder.
remove_file
Remove a file from both the cache and base folder (if present).
write_json
Write dict to the folder, to be serialized as json.
The dictionary is stored in memory, until the cache is flushed, at which point the dictionary is serialized to json and written to disk.
write_text
write text to the cache.
ReaderJsonBase
Bases: aiida.tools.importexport.archive.readers.ArchiveReaderAbstract
aiida.tools.importexport.archive.readers.ArchiveReaderAbstract
A reader base for the JSON compressed formats.
FILENAME_DATA
FILENAME_METADATA
REPO_FOLDER
A reader for JSON compressed archives.
filename – the filename (possibly including the absolute path) of the file on which to export.
sandbox_in_repo – Create the temporary uncompressed folder within the aiida repository
_extract
Extract repository data to a temporary folder.
path_prefix – Only extract paths starting with this prefix.
TypeError – if parameter types are not respected
_get_data
Retrieve the data JSON.
_get_metadata
Retrieve the metadata JSON.
ReaderJsonFolder
Bases: aiida.tools.importexport.archive.readers.ReaderJsonBase
aiida.tools.importexport.archive.readers.ReaderJsonBase
A reader for a JSON plain folder format.
ReaderJsonTar
A reader for a JSON tar compressed format.
ReaderJsonZip
A reader for a JSON zip compressed format.
WriterJsonFolder
Bases: aiida.tools.importexport.archive.writers.ArchiveWriterAbstract
aiida.tools.importexport.archive.writers.ArchiveWriterAbstract
An archive writer, which writes database data as a single JSON and repository data in a folder system.
This writer is mainly intended for backward compatibility with export_tree.
folder – a folder to write the archive to.
filepath – dummy value not used
WriterJsonTar
The entire containing folder is then compressed as a tar file.
WriterJsonZip
An archive writer, which writes database data as a single JSON and repository data in a zipped folder system.
use_compression – Whether or not to deflate the objects inside the zip file.
cache_zipinfo – Cache the zip file index on disk during the write. This reduces the RAM usage of the process, but will make the process slower.
detect_archive_type
For back-compatibility, but should be replaced with direct comparison of classes.
in_path – the path to the file
the archive type identifier (currently one of ‘zip’, ‘tar.gz’, ‘folder’)
get_migrator
Return the available archive migrator classes.
get_reader
Return the available writer classes.
get_writer
null_callback
A null callback function.
Shared resources for the archive.
aiida.tools.importexport.archive.common.
Archive migration classes, for migrating an archive to different versions.
aiida.tools.importexport.archive.migrators.
Archive reader classes.
aiida.tools.importexport.archive.readers.
Archive writer classes.
aiida.tools.importexport.archive.writers.