aiida.transports.plugins package¶
Submodules¶
-
class
aiida.transports.plugins.ssh.
SshTransport
(machine, **kwargs)[source]¶ Bases:
aiida.transports.transport.Transport
Support connection, command execution and data transfer to remote computers via SSH+SFTP.
-
_DEFAULT_SAFE_OPEN_INTERVAL
= 5¶
-
__abstractmethods__
= frozenset({})¶
-
__init__
(machine, **kwargs)[source]¶ Initialize the SshTransport class.
- Parameters
machine – the machine to connect to
load_system_host_keys – (optional, default False) if False, do not load the system host keys
key_policy – (optional, default = paramiko.RejectPolicy()) the policy to use for unknown keys
Other parameters valid for the ssh connect function (see the self._valid_connect_params list) are passed to the connect function (as port, username, password, …); taken from the accepted paramiko.SSHClient.connect() params.
-
__module__
= 'aiida.transports.plugins.ssh'¶
-
_abc_impl
= <_abc_data object>¶
-
_exec_command_internal
(command, combine_stderr=False, bufsize=-1)[source]¶ Executes the specified command in bash login shell.
Before the command is executed, changes directory to the current working directory as returned by self.getcwd().
For executing commands and waiting for them to finish, use exec_command_wait.
- Parameters
command – the command to execute. The command is assumed to be already escaped using
aiida.common.escaping.escape_for_bash()
.combine_stderr – (default False) if True, combine stdout and stderr on the same buffer (i.e., stdout). Note: If combine_stderr is True, stderr will always be empty.
bufsize – same meaning of the one used by paramiko.
- Returns
a tuple with (stdin, stdout, stderr, channel), where stdin, stdout and stderr behave as file-like objects, plus the methods provided by paramiko, and channel is a paramiko.Channel object.
-
classmethod
_get_allow_agent_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_compress_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_gss_auth_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_gss_deleg_creds_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_gss_host_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_gss_kex_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_key_filename_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_key_policy_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_load_system_host_keys_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_look_for_keys_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_port_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_proxy_command_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
classmethod
_get_safe_interval_suggestion_string
(computer)[source]¶ Default time in seconds between consecutive checks.
Set to a non-zero value to be safe e.g. in the case of transports with a connection limit, to avoid overloading the server (and being banned). Should be overriden in plugins. This is anyway just a default, as the value can be changed by the user in the Computer properties, for instance. Currently both the local and the ssh transport override this value, so this is not used, but it will be the default for possible new plugins.
-
classmethod
_get_timeout_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
Provide 60s as a default timeout for connections.
-
classmethod
_get_username_suggestion_string
(computer)[source]¶ Return a suggestion for the specific field.
-
_os_path_split_asunder
(path)[source]¶ Used by makedirs. Takes path (a str) and returns a list deconcatenating the path
-
_valid_auth_options
= [('username', {'prompt': 'User name', 'help': 'user name for the computer', 'non_interactive_default': True}), ('port', {'option': <aiida.cmdline.params.options.overridable.OverridableOption object>, 'prompt': 'port Nr', 'non_interactive_default': True}), ('look_for_keys', {'switch': True, 'prompt': 'Look for keys', 'help': 'switch automatic key file discovery on / off', 'non_interactive_default': True}), ('key_filename', {'type': ABSOLUTEPATH, 'prompt': 'SSH key file', 'help': 'Manually pass a key file if default path is not set in ssh config', 'non_interactive_default': True}), ('timeout', {'type': <class 'int'>, 'prompt': 'Connection timeout in s', 'help': 'time in seconds to wait for connection before giving up', 'non_interactive_default': True}), ('allow_agent', {'switch': True, 'prompt': 'Allow ssh agent', 'help': 'switch to allow or disallow ssh agent', 'non_interactive_default': True}), ('proxy_command', {'prompt': 'SSH proxy command', 'help': 'SSH proxy command', 'non_interactive_default': True}), ('compress', {'switch': True, 'prompt': 'Compress file transfers', 'help': 'switch file transfer compression on / off', 'non_interactive_default': True}), ('gss_auth', {'type': <class 'bool'>, 'prompt': 'GSS auth', 'help': 'GSS auth for kerberos', 'non_interactive_default': True}), ('gss_kex', {'type': <class 'bool'>, 'prompt': 'GSS kex', 'help': 'GSS kex for kerberos', 'non_interactive_default': True}), ('gss_deleg_creds', {'type': <class 'bool'>, 'prompt': 'GSS deleg_creds', 'help': 'GSS deleg_creds for kerberos', 'non_interactive_default': True}), ('gss_host', {'prompt': 'GSS host', 'help': 'GSS host for kerberos', 'non_interactive_default': True}), ('load_system_host_keys', {'switch': True, 'prompt': 'Load system host keys', 'help': 'switch loading system host keys on / off', 'non_interactive_default': True}), ('key_policy', {'type': Choice(['RejectPolicy', 'WarningPolicy', 'AutoAddPolicy']), 'prompt': 'Key policy', 'help': 'SSH key policy', 'non_interactive_default': True})]¶
-
_valid_connect_options
= [('username', {'prompt': 'User name', 'help': 'user name for the computer', 'non_interactive_default': True}), ('port', {'option': <aiida.cmdline.params.options.overridable.OverridableOption object>, 'prompt': 'port Nr', 'non_interactive_default': True}), ('look_for_keys', {'switch': True, 'prompt': 'Look for keys', 'help': 'switch automatic key file discovery on / off', 'non_interactive_default': True}), ('key_filename', {'type': ABSOLUTEPATH, 'prompt': 'SSH key file', 'help': 'Manually pass a key file if default path is not set in ssh config', 'non_interactive_default': True}), ('timeout', {'type': <class 'int'>, 'prompt': 'Connection timeout in s', 'help': 'time in seconds to wait for connection before giving up', 'non_interactive_default': True}), ('allow_agent', {'switch': True, 'prompt': 'Allow ssh agent', 'help': 'switch to allow or disallow ssh agent', 'non_interactive_default': True}), ('proxy_command', {'prompt': 'SSH proxy command', 'help': 'SSH proxy command', 'non_interactive_default': True}), ('compress', {'switch': True, 'prompt': 'Compress file transfers', 'help': 'switch file transfer compression on / off', 'non_interactive_default': True}), ('gss_auth', {'type': <class 'bool'>, 'prompt': 'GSS auth', 'help': 'GSS auth for kerberos', 'non_interactive_default': True}), ('gss_kex', {'type': <class 'bool'>, 'prompt': 'GSS kex', 'help': 'GSS kex for kerberos', 'non_interactive_default': True}), ('gss_deleg_creds', {'type': <class 'bool'>, 'prompt': 'GSS deleg_creds', 'help': 'GSS deleg_creds for kerberos', 'non_interactive_default': True}), ('gss_host', {'prompt': 'GSS host', 'help': 'GSS host for kerberos', 'non_interactive_default': True})]¶
-
_valid_connect_params
= ['username', 'port', 'look_for_keys', 'key_filename', 'timeout', 'allow_agent', 'proxy_command', 'compress', 'gss_auth', 'gss_kex', 'gss_deleg_creds', 'gss_host']¶
-
chdir
(path)[source]¶ Change directory of the SFTP session. Emulated internally by paramiko.
Differently from paramiko, if you pass None to chdir, nothing happens and the cwd is unchanged.
-
chmod
(path, mode)[source]¶ Change permissions to path
- Parameters
path – path to file
mode – new permission bits (integer)
-
close
()[source]¶ Close the SFTP channel, and the SSHClient.
- Todo
correctly manage exceptions
- Raises
aiida.common.InvalidOperation – if the channel is already open
-
copy
(remotesource, remotedestination, dereference=False, recursive=True)[source]¶ Copy a file or a directory from remote source to remote destination. Flags used:
-r
: recursive copy;-f
: force, makes the command non interactive;-L
follows symbolic links- Parameters
remotesource – file to copy from
remotedestination – file to copy to
dereference – if True, copy content instead of copying the symlinks only Default = False.
recursive (bool) – if True copy directories recursively, otherwise only copy the specified file(s)
- Raises
IOError – if the cp execution failed.
Note
setting dereference equal to True could cause infinite loops.
-
copyfile
(remotesource, remotedestination, dereference=False)[source]¶ Copy a file from remote source to remote destination (On the same remote machine)
- Parameters
remotesource (str) – path of the remote source directory / file
remotedestination (str) – path of the remote destination directory / file
dereference (bool) – if True copy the contents of any symlinks found, otherwise copy the symlinks themselves
- Raises
IOError – if one of src or dst does not exist
-
copytree
(remotesource, remotedestination, dereference=False)[source]¶ Copy a folder from remote source to remote destination (On the same remote machine)
- Parameters
remotesource (str) – path of the remote source directory / file
remotedestination (str) – path of the remote destination directory / file
dereference (bool) – if True copy the contents of any symlinks found, otherwise copy the symlinks themselves
- Raises
IOError – if one of src or dst does not exist
-
exec_command_wait
(command, stdin=None, combine_stderr=False, bufsize=-1)[source]¶ Executes the specified command and waits for it to finish.
- Parameters
command – the command to execute
stdin – (optional,default=None) can be a string or a file-like object.
combine_stderr – (optional, default=False) see docstring of self._exec_command_internal()
bufsize – same meaning of paramiko.
- Returns
a tuple with (return_value, stdout, stderr) where stdout and stderr are strings.
-
get
(remotepath, localpath, callback=None, dereference=True, overwrite=True, ignore_nonexisting=False)[source]¶ Get a file or folder from remote to local. Redirects to getfile or gettree.
- Parameters
remotepath – a remote path
localpath – an (absolute) local path
dereference – follow symbolic links. Default = True (default behaviour in paramiko). False is not implemented.
overwrite – if True overwrites files and folders. Default = False
- Raises
ValueError – if local path is invalid
IOError – if the remotepath is not found
-
get_attribute
(path)[source]¶ Returns the object Fileattribute, specified in aiida.transports Receives in input the path of a given file.
-
getcwd
()[source]¶ Return the current working directory for this SFTP session, as emulated by paramiko. If no directory has been set with chdir, this method will return None. But in __enter__ this is set explicitly, so this should never happen within this class.
-
getfile
(remotepath, localpath, callback=None, dereference=True, overwrite=True)[source]¶ Get a file from remote to local.
- Parameters
remotepath – a remote path
localpath – an (absolute) local path
overwrite – if True overwrites files and folders. Default = False
- Raises
ValueError – if local path is invalid
OSError – if unintentionally overwriting
-
gettree
(remotepath, localpath, callback=None, dereference=True, overwrite=True)[source]¶ Get a folder recursively from remote to local.
- Parameters
remotepath – a remote path
localpath – an (absolute) local path
dereference – follow symbolic links. Default = True (default behaviour in paramiko). False is not implemented.
overwrite – if True overwrites files and folders. Default = False
- Raises
ValueError – if local path is invalid
IOError – if the remotepath is not found
OSError – if unintentionally overwriting
-
gotocomputer_command
(remotedir)[source]¶ Specific gotocomputer string to connect to a given remote computer via ssh and directly go to the calculation folder.
-
isdir
(path)[source]¶ Return True if the given path is a directory, False otherwise. Return False also if the path does not exist.
-
isfile
(path)[source]¶ Return True if the given path is a file, False otherwise. Return False also if the path does not exist.
-
listdir
(path='.', pattern=None)[source]¶ Get the list of files at path.
- Parameters
path – default = ‘.’
pattern – returns the list of files matching pattern. Unix only. (Use to emulate
ls *
for example)
-
makedirs
(path, ignore_existing=False)[source]¶ Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except that any intermediate path segment (not just the rightmost) will be created if it does not exist.
NOTE: since os.path.split uses the separators as the host system (that could be windows), I assume the remote computer is Linux-based and use ‘/’ as separators!
- Parameters
path – directory to create (string)
ignore_existing – if set to true, it doesn’t give any error if the leaf directory does already exist (bool)
- Raises
OSError – If the directory already exists.
-
mkdir
(path, ignore_existing=False)[source]¶ Create a folder (directory) named path.
- Parameters
path – name of the folder to create
ignore_existing – if True, does not give any error if the directory already exists
- Raises
OSError – If the directory already exists.
-
open
()[source]¶ Open a SSHClient to the machine possibly using the parameters given in the __init__.
Also opens a sftp channel, ready to be used. The current working directory is set explicitly, so it is not None.
- Raises
aiida.common.InvalidOperation – if the channel is already open
-
put
(localpath, remotepath, callback=None, dereference=True, overwrite=True, ignore_nonexisting=False)[source]¶ Put a file or a folder from local to remote. Redirects to putfile or puttree.
- Parameters
localpath – an (absolute) local path
remotepath – a remote path
dereference – follow symbolic links (boolean). Default = True (default behaviour in paramiko). False is not implemented.
overwrite – if True overwrites files and folders (boolean). Default = False.
- Raises
ValueError – if local path is invalid
OSError – if the localpath does not exist
-
putfile
(localpath, remotepath, callback=None, dereference=True, overwrite=True)[source]¶ Put a file from local to remote.
- Parameters
localpath – an (absolute) local path
remotepath – a remote path
overwrite – if True overwrites files and folders (boolean). Default = True.
- Raises
ValueError – if local path is invalid
OSError – if the localpath does not exist, or unintentionally overwriting
-
puttree
(localpath, remotepath, callback=None, dereference=True, overwrite=True)[source]¶ Put a folder recursively from local to remote.
- Parameters
localpath – an (absolute) local path
remotepath – a remote path
dereference – follow symbolic links (boolean) Default = True (default behaviour in paramiko). False is not implemented.
overwrite – if True overwrites files and folders (boolean). Default = True
- Raises
ValueError – if local path is invalid
OSError – if the localpath does not exist, or trying to overwrite
IOError – if remotepath is invalid
Note
setting dereference equal to True could cause infinite loops. see os.walk() documentation
-
rename
(src, dst)[source]¶ Rename a file or folder from src to dst.
- Parameters
oldpath (str) – existing name of the file or folder
newpath (str) – new name for the file or folder
- Raises
IOError – if src/dst is not found
ValueError – if src/dst is not a valid string
-
rmtree
(path)[source]¶ Remove a file or a directory at path, recursively Flags used: -r: recursive copy; -f: force, makes the command non interactive;
- Parameters
path – remote path to delete
- Raises
IOError – if the rm execution failed.
-
property
sftp
¶
-
property
sshclient
¶
-
Test ssh plugin on localhost