aiida.transports.plugins package

Submodules

aiida.transports.plugins.ssh.parse_sshconfig(computername)[source]
aiida.transports.plugins.ssh.convert_to_bool(string)[source]
class aiida.transports.plugins.ssh.SshTransport(machine, **kwargs)[source]

Bases: aiida.transports.transport.Transport

Support connection, command execution and data transfer to remote computers via SSH+SFTP.

_DEFAULT_SAFE_OPEN_INTERVAL = 5
__abstractmethods__ = frozenset()
__init__(machine, **kwargs)[source]

Initialize the SshTransport class.

Parameters:
  • machine – the machine to connect to
  • load_system_host_keys – (optional, default False) if False, do not load the system host keys
  • key_policy – (optional, default = paramiko.RejectPolicy()) the policy to use for unknown keys

Other parameters valid for the ssh connect function (see the self._valid_connect_params list) are passed to the connect function (as port, username, password, …); taken from the accepted paramiko.SSHClient.connect() params.

__module__ = 'aiida.transports.plugins.ssh'
__str__()[source]

Return a useful string.

_abc_impl = <_abc_data object>
_exec_command_internal(command, combine_stderr=False, bufsize=-1)[source]

Executes the specified command in bash login shell.

Before the command is executed, changes directory to the current working directory as returned by self.getcwd().

For executing commands and waiting for them to finish, use exec_command_wait.

Parameters:
  • command – the command to execute. The command is assumed to be already escaped using aiida.common.escaping.escape_for_bash().
  • combine_stderr – (default False) if True, combine stdout and stderr on the same buffer (i.e., stdout). Note: If combine_stderr is True, stderr will always be empty.
  • bufsize – same meaning of the one used by paramiko.
Returns:

a tuple with (stdin, stdout, stderr, channel), where stdin, stdout and stderr behave as file-like objects, plus the methods provided by paramiko, and channel is a paramiko.Channel object.

_exec_cp(cp_exe, cp_flags, src, dst)[source]
classmethod _get_allow_agent_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_compress_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_gss_auth_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_gss_deleg_creds_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_gss_host_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_gss_kex_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_key_filename_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_key_policy_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_load_system_host_keys_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_look_for_keys_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_port_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_proxy_command_suggestion_string(computer)[source]

Return a suggestion for the specific field.

classmethod _get_safe_interval_suggestion_string(computer)[source]

Default time in seconds between consecutive checks.

Set to a non-zero value to be safe e.g. in the case of transports with a connection limit, to avoid overloading the server (and being banned). Should be overriden in plugins. This is anyway just a default, as the value can be changed by the user in the Computer properties, for instance. Currently both the local and the ssh transport override this value, so this is not used, but it will be the default for possible new plugins.

classmethod _get_timeout_suggestion_string(computer)[source]

Return a suggestion for the specific field.

Provide 60s as a default timeout for connections.

classmethod _get_username_suggestion_string(computer)[source]

Return a suggestion for the specific field.

_local_listdir(path, pattern=None)[source]

Acts on the local folder, for the rest, same as listdir

_os_path_split_asunder(path)[source]

Used by makedirs. Takes path (a str) and returns a list deconcatenating the path

_valid_auth_options = [('username', {'prompt': 'User name', 'help': 'user name for the computer', 'non_interactive_default': True}), ('port', {'option': <aiida.cmdline.params.options.overridable.OverridableOption object>, 'prompt': 'port Nr', 'non_interactive_default': True}), ('look_for_keys', {'switch': True, 'prompt': 'Look for keys', 'help': 'switch automatic key file discovery on / off', 'non_interactive_default': True}), ('key_filename', {'type': ABSOLUTEPATH, 'prompt': 'SSH key file', 'help': 'Manually pass a key file if default path is not set in ssh config', 'non_interactive_default': True}), ('timeout', {'type': <class 'int'>, 'prompt': 'Connection timeout in s', 'help': 'time in seconds to wait for connection before giving up', 'non_interactive_default': True}), ('allow_agent', {'switch': True, 'prompt': 'Allow ssh agent', 'help': 'switch to allow or disallow ssh agent', 'non_interactive_default': True}), ('proxy_command', {'prompt': 'SSH proxy command', 'help': 'SSH proxy command', 'non_interactive_default': True}), ('compress', {'switch': True, 'prompt': 'Compress file transfers', 'help': 'switch file transfer compression on / off', 'non_interactive_default': True}), ('gss_auth', {'type': <class 'bool'>, 'prompt': 'GSS auth', 'help': 'GSS auth for kerberos', 'non_interactive_default': True}), ('gss_kex', {'type': <class 'bool'>, 'prompt': 'GSS kex', 'help': 'GSS kex for kerberos', 'non_interactive_default': True}), ('gss_deleg_creds', {'type': <class 'bool'>, 'prompt': 'GSS deleg_creds', 'help': 'GSS deleg_creds for kerberos', 'non_interactive_default': True}), ('gss_host', {'prompt': 'GSS host', 'help': 'GSS host for kerberos', 'non_interactive_default': True}), ('load_system_host_keys', {'switch': True, 'prompt': 'Load system host keys', 'help': 'switch loading system host keys on / off', 'non_interactive_default': True}), ('key_policy', {'type': Choice(['RejectPolicy', 'WarningPolicy', 'AutoAddPolicy']), 'prompt': 'Key policy', 'help': 'SSH key policy', 'non_interactive_default': True})]
_valid_connect_options = [('username', {'prompt': 'User name', 'help': 'user name for the computer', 'non_interactive_default': True}), ('port', {'option': <aiida.cmdline.params.options.overridable.OverridableOption object>, 'prompt': 'port Nr', 'non_interactive_default': True}), ('look_for_keys', {'switch': True, 'prompt': 'Look for keys', 'help': 'switch automatic key file discovery on / off', 'non_interactive_default': True}), ('key_filename', {'type': ABSOLUTEPATH, 'prompt': 'SSH key file', 'help': 'Manually pass a key file if default path is not set in ssh config', 'non_interactive_default': True}), ('timeout', {'type': <class 'int'>, 'prompt': 'Connection timeout in s', 'help': 'time in seconds to wait for connection before giving up', 'non_interactive_default': True}), ('allow_agent', {'switch': True, 'prompt': 'Allow ssh agent', 'help': 'switch to allow or disallow ssh agent', 'non_interactive_default': True}), ('proxy_command', {'prompt': 'SSH proxy command', 'help': 'SSH proxy command', 'non_interactive_default': True}), ('compress', {'switch': True, 'prompt': 'Compress file transfers', 'help': 'switch file transfer compression on / off', 'non_interactive_default': True}), ('gss_auth', {'type': <class 'bool'>, 'prompt': 'GSS auth', 'help': 'GSS auth for kerberos', 'non_interactive_default': True}), ('gss_kex', {'type': <class 'bool'>, 'prompt': 'GSS kex', 'help': 'GSS kex for kerberos', 'non_interactive_default': True}), ('gss_deleg_creds', {'type': <class 'bool'>, 'prompt': 'GSS deleg_creds', 'help': 'GSS deleg_creds for kerberos', 'non_interactive_default': True}), ('gss_host', {'prompt': 'GSS host', 'help': 'GSS host for kerberos', 'non_interactive_default': True})]
_valid_connect_params = ['username', 'port', 'look_for_keys', 'key_filename', 'timeout', 'allow_agent', 'proxy_command', 'compress', 'gss_auth', 'gss_kex', 'gss_deleg_creds', 'gss_host']
chdir(path)[source]

Change directory of the SFTP session. Emulated internally by paramiko.

Differently from paramiko, if you pass None to chdir, nothing happens and the cwd is unchanged.

chmod(path, mode)[source]

Change permissions to path

Parameters:
  • path – path to file
  • mode – new permission bits (integer)
close()[source]

Close the SFTP channel, and the SSHClient.

Todo:correctly manage exceptions
Raises:aiida.common.InvalidOperation – if the channel is already open
copy(remotesource, remotedestination, dereference=False, recursive=True)[source]

Copy a file or a directory from remote source to remote destination. Flags used: -r: recursive copy; -f: force, makes the command non interactive; -L follows symbolic links

Parameters:
  • remotesource – file to copy from
  • remotedestination – file to copy to
  • dereference – if True, copy content instead of copying the symlinks only Default = False.
  • recursive (bool) – if True copy directories recursively, otherwise only copy the specified file(s)
Raises:

IOError – if the cp execution failed.

Note

setting dereference equal to True could cause infinite loops.

copyfile(remotesource, remotedestination, dereference=False)[source]

Copy a file from remote source to remote destination (On the same remote machine)

Parameters:
  • remotesource (str) – path of the remote source directory / file
  • remotedestination (str) – path of the remote destination directory / file
  • dereference (bool) – if True copy the contents of any symlinks found, otherwise copy the symlinks themselves
Raises:

IOError – if one of src or dst does not exist

copytree(remotesource, remotedestination, dereference=False)[source]

Copy a folder from remote source to remote destination (On the same remote machine)

Parameters:
  • remotesource (str) – path of the remote source directory / file
  • remotedestination (str) – path of the remote destination directory / file
  • dereference (bool) – if True copy the contents of any symlinks found, otherwise copy the symlinks themselves
Raises:

IOError – if one of src or dst does not exist

exec_command_wait(command, stdin=None, combine_stderr=False, bufsize=-1)[source]

Executes the specified command and waits for it to finish.

Parameters:
  • command – the command to execute
  • stdin – (optional,default=None) can be a string or a file-like object.
  • combine_stderr – (optional, default=False) see docstring of self._exec_command_internal()
  • bufsize – same meaning of paramiko.
Returns:

a tuple with (return_value, stdout, stderr) where stdout and stderr are strings.

get(remotepath, localpath, callback=None, dereference=True, overwrite=True, ignore_nonexisting=False)[source]

Get a file or folder from remote to local. Redirects to getfile or gettree.

Parameters:
  • remotepath – a remote path
  • localpath – an (absolute) local path
  • dereference – follow symbolic links. Default = True (default behaviour in paramiko). False is not implemented.
  • overwrite – if True overwrites files and folders. Default = False
Raises:
  • ValueError – if local path is invalid
  • IOError – if the remotepath is not found
get_attribute(path)[source]

Returns the object Fileattribute, specified in aiida.transports Receives in input the path of a given file.

getcwd()[source]

Return the current working directory for this SFTP session, as emulated by paramiko. If no directory has been set with chdir, this method will return None. But in __enter__ this is set explicitly, so this should never happen within this class.

getfile(remotepath, localpath, callback=None, dereference=True, overwrite=True)[source]

Get a file from remote to local.

Parameters:
  • remotepath – a remote path
  • localpath – an (absolute) local path
  • overwrite – if True overwrites files and folders. Default = False
Raises:
  • ValueError – if local path is invalid
  • OSError – if unintentionally overwriting
gettree(remotepath, localpath, callback=None, dereference=True, overwrite=True)[source]

Get a folder recursively from remote to local.

Parameters:
  • remotepath – a remote path
  • localpath – an (absolute) local path
  • dereference – follow symbolic links. Default = True (default behaviour in paramiko). False is not implemented.
  • overwrite – if True overwrites files and folders. Default = False
Raises:
  • ValueError – if local path is invalid
  • IOError – if the remotepath is not found
  • OSError – if unintentionally overwriting
gotocomputer_command(remotedir)[source]

Specific gotocomputer string to connect to a given remote computer via ssh and directly go to the calculation folder.

isdir(path)[source]

Return True if the given path is a directory, False otherwise. Return False also if the path does not exist.

isfile(path)[source]

Return True if the given path is a file, False otherwise. Return False also if the path does not exist.

listdir(path='.', pattern=None)[source]

Get the list of files at path.

Parameters:
  • path – default = ‘.’
  • pattern – returns the list of files matching pattern. Unix only. (Use to emulate ls * for example)
makedirs(path, ignore_existing=False)[source]

Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except that any intermediate path segment (not just the rightmost) will be created if it does not exist.

NOTE: since os.path.split uses the separators as the host system (that could be windows), I assume the remote computer is Linux-based and use ‘/’ as separators!

Parameters:
  • path – directory to create (string)
  • ignore_existing – if set to true, it doesn’t give any error if the leaf directory does already exist (bool)
Raises:

OSError – If the directory already exists.

mkdir(path, ignore_existing=False)[source]

Create a folder (directory) named path.

Parameters:
  • path – name of the folder to create
  • ignore_existing – if True, does not give any error if the directory already exists
Raises:

OSError – If the directory already exists.

normalize(path)[source]

Returns the normalized path (removing double slashes, etc…)

open()[source]

Open a SSHClient to the machine possibly using the parameters given in the __init__.

Also opens a sftp channel, ready to be used. The current working directory is set explicitly, so it is not None.

Raises:aiida.common.InvalidOperation – if the channel is already open
path_exists(path)[source]

Check if path exists

put(localpath, remotepath, callback=None, dereference=True, overwrite=True, ignore_nonexisting=False)[source]

Put a file or a folder from local to remote. Redirects to putfile or puttree.

Parameters:
  • localpath – an (absolute) local path
  • remotepath – a remote path
  • dereference – follow symbolic links (boolean). Default = True (default behaviour in paramiko). False is not implemented.
  • overwrite – if True overwrites files and folders (boolean). Default = False.
Raises:
  • ValueError – if local path is invalid
  • OSError – if the localpath does not exist
putfile(localpath, remotepath, callback=None, dereference=True, overwrite=True)[source]

Put a file from local to remote.

Parameters:
  • localpath – an (absolute) local path
  • remotepath – a remote path
  • overwrite – if True overwrites files and folders (boolean). Default = True.
Raises:
  • ValueError – if local path is invalid
  • OSError – if the localpath does not exist, or unintentionally overwriting
puttree(localpath, remotepath, callback=None, dereference=True, overwrite=True)[source]

Put a folder recursively from local to remote.

Parameters:
  • localpath – an (absolute) local path
  • remotepath – a remote path
  • dereference – follow symbolic links (boolean) Default = True (default behaviour in paramiko). False is not implemented.
  • overwrite – if True overwrites files and folders (boolean). Default = True
Raises:
  • ValueError – if local path is invalid
  • OSError – if the localpath does not exist, or trying to overwrite
  • IOError – if remotepath is invalid

Note

setting dereference equal to True could cause infinite loops. see os.walk() documentation

remove(path)[source]

Remove a single file at ‘path’

rename(src, dst)[source]

Rename a file or folder from src to dst.

Parameters:
  • oldpath (str) – existing name of the file or folder
  • newpath (str) – new name for the file or folder
Raises:
  • IOError – if src/dst is not found
  • ValueError – if src/dst is not a valid string
rmdir(path)[source]

Remove the folder named ‘path’ if empty.

rmtree(path)[source]

Remove a file or a directory at path, recursively Flags used: -r: recursive copy; -f: force, makes the command non interactive;

Parameters:path – remote path to delete
Raises:IOError – if the rm execution failed.
sftp
sshclient

Create a symbolic link between the remote source and the remote destination.

Parameters:
  • remotesource – remote source. Can contain a pattern.
  • remotedestination – remote destination

Test ssh plugin on localhost

class aiida.transports.plugins.test_ssh.TestBasicConnection(methodName='runTest')[source]

Bases: unittest.case.TestCase

Test basic connections.

__module__ = 'aiida.transports.plugins.test_ssh'
test_auto_add_policy()[source]
test_closed_connection_sftp()[source]
test_closed_connection_ssh()[source]
test_invalid_param()[source]
test_no_host_key()[source]