aiida.transport package¶
-
class
aiida.transport.
FileAttribute
(init=None)[source]¶ Bases:
aiida.common.extendeddicts.FixedFieldsAttributeDict
A class, resembling a dictionary, to describe the attributes of a file, that is returned by get_attribute(). Possible keys: st_size, st_uid, st_gid, st_mode, st_atime, st_mtime
-
__module__
= 'aiida.transport'¶
-
_valid_fields
= ('st_size', 'st_uid', 'st_gid', 'st_mode', 'st_atime', 'st_mtime')¶
-
-
class
aiida.transport.
Transport
(*args, **kwargs)[source]¶ Bases:
object
Abstract class for a generic transport (ssh, local, …) Contains the set of minimal methods
-
__dict__
= dict_proxy({'rename': <function rename>, 'copyfile': <function copyfile>, '__module__': 'aiida.transport', '__str__': <function __str__>, 'copy_from_remote_to_remote': <function copy_from_remote_to_remote>, 'chmod': <function chmod>, '__dict__': <attribute '__dict__' of 'Transport' objects>, 'copytree': <function copytree>, 'get_valid_auth_params': <classmethod object>, 'close': <function close>, 'open': <function open>, 'listdir_withattributes': <function listdir_withattributes>, '__init__': <function __init__>, 'normalize': <function normalize>, '_valid_auth_params': None, 'get_mode': <function get_mode>, 'isfile': <function isfile>, 'makedirs': <function makedirs>, '__enter__': <function __enter__>, 'mkdir': <function mkdir>, 'chown': <function chown>, '_exec_command_internal': <function _exec_command_internal>, 'getfile': <function getfile>, 'rmdir': <function rmdir>, 'putfile': <function putfile>, '__doc__': '\n Abstract class for a generic transport (ssh, local, ...)\n Contains the set of minimal methods\n ', 'isdir': <function isdir>, 'listdir': <function listdir>, '__exit__': <function __exit__>, 'get': <function get>, 'getcwd': <function getcwd>, 'glob': <function glob>, 'get_attribute': <function get_attribute>, 'has_magic': <function has_magic>, 'symlink': <function symlink>, '_set_logger_extra': <function _set_logger_extra>, 'path_exists': <function path_exists>, 'rmtree': <function rmtree>, 'put': <function put>, 'copy': <function copy>, 'logger': <property object>, 'glob1': <function glob1>, 'glob0': <function glob0>, 'gotocomputer_command': <function gotocomputer_command>, 'gettree': <function gettree>, 'exec_command_wait': <function exec_command_wait>, 'chdir': <function chdir>, 'get_valid_transports': <classmethod object>, 'remove': <function remove>, 'get_short_doc': <classmethod object>, 'iglob': <function iglob>, '__weakref__': <attribute '__weakref__' of 'Transport' objects>, 'puttree': <function puttree>, '__repr__': <function __repr__>, 'whoami': <function whoami>})¶
-
__enter__
()[source]¶ For transports that require opening a connection, opens all required channels (used in ‘with’ statements)
-
__exit__
(type, value, traceback)[source]¶ Closes connections, if needed (used in ‘with’ statements).
-
__module__
= 'aiida.transport'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
_exec_command_internal
(command, **kwargs)[source]¶ Execute the command on the shell, similarly to os.system.
Enforce the execution to be run from the cwd (as given by self.getcwd), if this is not None.
If possible, use the higher-level exec_command_wait function.
Parameters: command (str) – execute the command given as a string Returns: stdin, stdout, stderr and the session, when this exists (can be None).
-
_set_logger_extra
(logger_extra)[source]¶ Pass the data that should be passed automatically to self.logger as ‘extra’ keyword. This is typically useful if you pass data obtained using get_dblogger_extra in aiida.common.log, to automatically log also to the DbLog table.
Parameters: logger_extra – data that you want to pass as extra to the self.logger. To write to DbLog, it should be created by the aiida.common.log.get_dblogger_extra function. Pass None if you do not want to have extras passed.
-
_valid_auth_params
= None¶
-
chdir
(path)[source]¶ Change directory to ‘path’
Parameters: path (str) – path to change working directory into. Raises: IOError, if the requested path does not exist Return type: str
-
chmod
(path, mode)[source]¶ Change permissions of a path.
Parameters: - path (str) – path to file
- mode (int) – new permissions
-
chown
(path, uid, gid)[source]¶ Change the owner (uid) and group (gid) of a file. As with python’s os.chown function, you must pass both arguments, so if you only want to change one, use stat first to retrieve the current owner and group.
Parameters: - path (str) – path to the file to change the owner and group of
- uid (int) – new owner’s uid
- gid (int) – new group id
-
copy
(remotesource, remotedestination, *args, **kwargs)[source]¶ Copy a file or a directory from remote source to remote destination (On the same remote machine)
Parameters: - remotesource (str) – path of the remote source directory / file
- remotedestination (str) – path of the remote destination directory / file
Raises: IOError, if one of src or dst does not exist
-
copy_from_remote_to_remote
(transportdestination, remotesource, remotedestination, **kwargs)[source]¶ Copy files or folders from a remote computer to another remote computer.
Parameters: - transportdestination – transport to be used for the destination computer
- remotesource (str) – path to the remote source directory / file
- remotedestination (str) – path to the remote destination directory / file
- kwargs – keyword parameters passed to the call to transportdestination.put, except for ‘dereference’ that is passed to self.get
Note
the keyword ‘dereference’ SHOULD be set to False for the final put (onto the destination), while it can be set to the value given in kwargs for the get from the source. In that way, a symbolic link would never be followed in the final copy to the remote destination. That way we could avoid getting unknown (potentially malicious) files into the destination computer. HOWEVER, since dereference=False is currently NOT supported by all plugins, we still force it to True for the final put.
Note
the supported keys in kwargs are callback, dereference, overwrite and ignore_nonexisting.
-
copyfile
(remotesource, remotedestination, *args, **kwargs)[source]¶ Copy a file from remote source to remote destination (On the same remote machine)
Parameters: - remotesource (str) – path of the remote source directory / file
- remotedestination (str) – path of the remote destination directory / file
Raises: IOError – if one of src or dst does not exist
-
copytree
(remotesource, remotedestination, *args, **kwargs)[source]¶ Copy a folder from remote source to remote destination (On the same remote machine)
Parameters: - remotesource (str) – path of the remote source directory / file
- remotedestination (str) – path of the remote destination directory / file
Raises: IOError – if one of src or dst does not exist
-
exec_command_wait
(command, **kwargs)[source]¶ Execute the command on the shell, waits for it to finish, and return the retcode, the stdout and the stderr.
Enforce the execution to be run from the pwd (as given by self.getcwd), if this is not None.
Parameters: command (str) – execute the command given as a string Returns: a list: the retcode (int), stdout (str) and stderr (str).
-
get
(remotepath, localpath, *args, **kwargs)[source]¶ Retrieve a file or folder from remote source to local destination dst must be an absolute path (src not necessarily)
Parameters: - remotepath – (str) remote_folder_path
- localpath – (str) local_folder_path
-
get_attribute
(path)[source]¶ Return an object FixedFieldsAttributeDict for file in a given path, as defined in aiida.common.extendeddicts Each attribute object consists in a dictionary with the following keys:
- st_size: size of files, in bytes
- st_uid: user id of owner
- st_gid: group id of owner
- st_mode: protection bits
- st_atime: time of most recent access
- st_mtime: time of most recent modification
Parameters: path (str) – path to file Returns: object FixedFieldsAttributeDict
-
get_mode
(path)[source]¶ Return the portion of the file’s mode that can be set by chmod().
Parameters: path (str) – path to file Returns: the portion of the file’s mode that can be set by chmod()
-
classmethod
get_short_doc
()[source]¶ Return the first non-empty line of the class docstring, if available
-
getfile
(remotepath, localpath, *args, **kwargs)[source]¶ Retrieve a file from remote source to local destination dst must be an absolute path (src not necessarily)
Parameters: - remotepath (str) – remote_folder_path
- localpath (str) – local_folder_path
-
gettree
(remotepath, localpath, *args, **kwargs)[source]¶ Retrieve a folder recursively from remote source to local destination dst must be an absolute path (src not necessarily)
Parameters: - remotepath (str) – remote_folder_path
- localpath (str) – local_folder_path
-
glob
(pathname)[source]¶ Return a list of paths matching a pathname pattern.
The pattern may contain simple shell-style wildcards a la fnmatch.
-
gotocomputer_command
(remotedir)[source]¶ Return a string to be run using os.system in order to connect via the transport to the remote directory.
Expected behaviors:
- A new bash session is opened
- A reasonable error message is produced if the folder does not exist
Parameters: remotedir (str) – the full path of the remote directory
-
iglob
(pathname)[source]¶ Return an iterator which yields the paths matching a pathname pattern.
The pattern may contain simple shell-style wildcards a la fnmatch.
-
isdir
(path)[source]¶ True if path is an existing directory.
Parameters: path (str) – path to directory Returns: boolean
-
isfile
(path)[source]¶ Return True if path is an existing file.
Parameters: path (str) – path to file Returns: boolean
-
listdir
(path='.', pattern=None)[source]¶ Return a list of the names of the entries in the given path. The list is in arbitrary order. It does not include the special entries ‘.’ and ‘..’ even if they are present in the directory.
Parameters: - path (str) – path to list (default to ‘.’)
- pattern (str) – if used, listdir returns a list of files matching filters in Unix style. Unix only.
Returns: a list of strings with the file/directory names
-
listdir_withattributes
(path='.', pattern=None)[source]¶ Return a list of the names of the entries in the given path. The list is in arbitrary order. It does not include the special entries ‘.’ and ‘..’ even if they are present in the directory.
Parameters: - path (str) – path to list (default to ‘.’)
- pattern (str) – if used, listdir returns a list of files matching filters in Unix style. Unix only.
Returns: a list of dictionaries, one per entry. The schema of the dictionary is the following:
{ 'name': String, 'attributes': FileAttributeObject, 'isdir': Bool }
where ‘name’ is the file or folder directory, and any other information is metadata (if the file is a folder, a directory, …). ‘attributes’ behaves as the output of transport.get_attribute(); isdir is a boolean indicating if the object is a directory or not.
-
logger
¶ Return the internal logger. If you have set extra parameters using _set_logger_extra(), a suitable LoggerAdapter instance is created, bringing with itself also the extras.
-
makedirs
(path, ignore_existing=False)[source]¶ Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except that any intermediate path segment (not just the rightmost) will be created if it does not exist.
Parameters: - path (str) – directory to create
- ignore_existing (bool) – if set to true, it doesn’t give any error if the leaf directory does already exist
Raises: OSError, if directory at path already exists
-
mkdir
(path, ignore_existing=False)[source]¶ Create a folder (directory) named path.
Parameters: - path (str) – name of the folder to create
- ignore_existing (bool) – if True, does not give any error if the directory already exists
Raises: OSError, if directory at path already exists
-
normalize
(path='.')[source]¶ Return the normalized path (on the server) of a given path. This can be used to quickly resolve symbolic links or determine what the server is considering to be the “current folder”.
Parameters: path (str) – path to be normalized Raises: IOError – if the path can’t be resolved on the server
-
put
(localpath, remotepath, *args, **kwargs)[source]¶ Put a file or a directory from local src to remote dst. src must be an absolute path (dst not necessarily)) Redirects to putfile and puttree.
Parameters: - localpath (str) – absolute path to local source
- remotepath (str) – path to remote destination
-
putfile
(localpath, remotepath, *args, **kwargs)[source]¶ Put a file from local src to remote dst. src must be an absolute path (dst not necessarily))
Parameters: - localpath (str) – absolute path to local file
- remotepath (str) – path to remote file
-
puttree
(localpath, remotepath, *args, **kwargs)[source]¶ Put a folder recursively from local src to remote dst. src must be an absolute path (dst not necessarily))
Parameters: - localpath (str) – absolute path to local folder
- remotepath (str) – path to remote folder
-
remove
(path)[source]¶ Remove the file at the given path. This only works on files; for removing folders (directories), use rmdir.
Parameters: path (str) – path to file to remove Raises: IOError – if the path is a directory
-
rename
(oldpath, newpath)[source]¶ Rename a file or folder from oldpath to newpath.
Parameters: - oldpath (str) – existing name of the file or folder
- newpath (str) – new name for the file or folder
Raises: - IOError – if oldpath/newpath is not found
- ValueError – if oldpath/newpath is not a valid string
-
rmdir
(path)[source]¶ Remove the folder named path. This works only for empty folders. For recursive remove, use rmtree.
Parameters: path (str) – absolute path to the folder to remove
-
rmtree
(path)[source]¶ Remove recursively the content at path
Parameters: path (str) – absolute path to remove
-
-
aiida.transport.
TransportFactory
(module)[source]¶ Used to return a suitable Transport subclass.
Parameters: module (str) – name of the module containing the Transport subclass Returns: the transport subclass located in module ‘module’
-
exception
aiida.transport.
TransportInternalError
[source]¶ Bases:
aiida.common.exceptions.InternalError
Raised if there is a transport error that is raised to an internal error (e.g. a transport method called without opening the channel first).
-
__module__
= 'aiida.transport'¶
-
-
aiida.transport.
copy_from_remote_to_remote
(transportsource, transportdestination, remotesource, remotedestination, **kwargs)[source]¶ Copy files or folders from a remote computer to another remote computer.
Parameters: - transportsource – transport to be used for the source computer
- transportdestination – transport to be used for the destination computer
- remotesource (str) – path to the remote source directory / file
- remotedestination (str) – path to the remote destination directory / file
- kwargs – keyword parameters passed to the final put, except for ‘dereference’ that is passed to the initial get
Note
it uses the method transportsource.copy_from_remote_to_remote