aiida.manage.external package¶
User facing APIs to control AiiDA from the verdi cli, scripts or plugins
Submodules¶
Connect to an existing PostgreSQL cluster as the postgres superuser and execute SQL commands.
- Note: Once the API of this functionality has converged, this module should be moved out of aiida-core and into a
separate package that can then be tested on multiple OS / postgres setups. Therefore, please keep this module entirely AiiDA-agnostic.
-
class
aiida.manage.external.pgsu.
PGSU
(interactive=False, quiet=True, dbinfo=None, determine_setup=True)[source]¶ Bases:
object
Connect to an existing PostgreSQL cluster as the postgres superuser and execute SQL commands.
Tries to use psycopg2 with a fallback to psql subcommands (using
sudo su
to run as postgres user).Simple Example:
postgres = PGSU() postgres.execute("CREATE USER testuser PASSWORD 'testpw'")
Complex Example:
postgres = PGSU(interactive=True, dbinfo={'port': 5433}) postgres.execute("CREATE USER testuser PASSWORD 'testpw'")
- Note: In postgresql
you cannot drop databases you are currently connected to
‘template0’ is the unmodifiable template database (which you cannot connect to)
‘template1’ is the modifiable template database (which you can connect to)
-
__dict__
= mappingproxy({'__module__': 'aiida.manage.external.pgsu', '__doc__': '\n Connect to an existing PostgreSQL cluster as the `postgres` superuser and execute SQL commands.\n\n Tries to use psycopg2 with a fallback to psql subcommands (using ``sudo su`` to run as postgres user).\n\n Simple Example::\n\n postgres = PGSU()\n postgres.execute("CREATE USER testuser PASSWORD \'testpw\'")\n\n Complex Example::\n\n postgres = PGSU(interactive=True, dbinfo={\'port\': 5433})\n postgres.execute("CREATE USER testuser PASSWORD \'testpw\'")\n\n Note: In postgresql\n * you cannot drop databases you are currently connected to\n * \'template0\' is the unmodifiable template database (which you cannot connect to)\n * \'template1\' is the modifiable template database (which you can connect to)\n ', '__init__': <function PGSU.__init__>, 'execute': <function PGSU.execute>, 'set_setup_fail_callback': <function PGSU.set_setup_fail_callback>, 'determine_setup': <function PGSU.determine_setup>, '_no_setup_detected': <function PGSU._no_setup_detected>, 'is_connected': <property object>, '__dict__': <attribute '__dict__' of 'PGSU' objects>, '__weakref__': <attribute '__weakref__' of 'PGSU' objects>})¶
-
__init__
(interactive=False, quiet=True, dbinfo=None, determine_setup=True)[source]¶ Store postgres connection info.
- Parameters
interactive – use True for verdi commands
quiet – use False to show warnings/exceptions
dbinfo – psycopg dictionary containing keys like ‘host’, ‘user’, ‘port’, ‘database’
determine_setup – Whether to determine setup upon instantiation. You may set this to False and use the ‘determine_setup()’ method instead.
-
__module__
= 'aiida.manage.external.pgsu'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
_no_setup_detected
()[source]¶ Print a warning message and calls the failed setup callback
- Returns
False, if no successful try.
-
determine_setup
()[source]¶ Determine how to connect as the postgres superuser.
Depending on how postgres is set up, psycopg2 can be used to create dbs and db users, otherwise a subprocess has to be used that executes psql as an os user with appropriate permissions.
- Note: We aim to connect as a superuser (typically ‘postgres’) with privileges to manipulate (create/drop)
databases and database users.
- Returns success
True, if connection could be established.
- Rtype success
bool
-
execute
(command, **kwargs)[source]¶ Execute postgres command using determined connection mode.
- Parameters
command – A psql command line as a str
kwargs – will be forwarded to _execute_… function
-
property
is_connected
¶
-
class
aiida.manage.external.pgsu.
PostgresConnectionMode
[source]¶ Bases:
enum.IntEnum
Describe mode of connecting to postgres.
-
DISCONNECTED
= 0¶
-
PSQL
= 2¶
-
PSYCOPG
= 1¶
-
__module__
= 'aiida.manage.external.pgsu'¶
-
-
aiida.manage.external.pgsu.
_execute_psql
(command, user='postgres', quiet=True, interactive=False, **kwargs)[source]¶ Executes an SQL command via
psql
as another system user in a subprocess.Tries to “become” the user specified in
kwargs
(i.e. interpreted as UNIX system user) and run psql in a subprocess.- Parameters
command – A psql command line as a str
quiet – If True, don’t print warnings.
interactive – If False, sudo won’t ask for a password and fail if one is required.
kwargs – connection details to forward to psql, signature as in psycopg2.connect
-
aiida.manage.external.pgsu.
_execute_psyco
(command, **kwargs)[source]¶ executes a postgres commandline through psycopg2
- Parameters
command – A psql command line as a str
kwargs – will be forwarded to psycopg2.connect
-
aiida.manage.external.pgsu.
_sudo_exists
()[source]¶ Check that the sudo command can be found
- Returns
True if successful, False otherwise
-
aiida.manage.external.pgsu.
_try_connect_psycopg
(**kwargs)[source]¶ try to start a psycopg2 connection.
- Returns
True if successful, False otherwise
-
aiida.manage.external.pgsu.
_try_subcmd
(**kwargs)[source]¶ try to run psql in a subprocess.
- Returns
True if successful, False otherwise
-
aiida.manage.external.pgsu.
escape_for_bash
(str_to_escape)[source]¶ This function takes any string and escapes it in a way that bash will interpret it as a single string.
Explanation:
At the end, in the return statement, the string is put within single quotes. Therefore, the only thing that I have to escape in bash is the single quote character. To do this, I substitute every single quote ‘ with ‘”’”’ which means:
First single quote: exit from the enclosing single quotes
Second, third and fourth character: “’” is a single quote character, escaped by double quotes
Last single quote: reopen the single quote to continue the string
Finally, note that for python I have to enclose the string ‘”’”’ within triple quotes to make it work, getting finally: the complicated string found below.
-
aiida.manage.external.pgsu.
prompt_db_info
(interactive, dbinfo)[source]¶ Prompt interactively for postgres database connection details
Can be used as a setup fail callback for
PGSU
- Returns
dictionary with the following keys: host, port, database, user
Provides an API for postgres database maintenance tasks.
This API creates and drops postgres users and databases used by the
verdi quicksetup
commandline tool. It allows convenient access to this
functionality from within python without knowing details about how postgres is
installed by default on various systems. If the postgres setup is not the
default installation, additional information needs to be provided.
-
class
aiida.manage.external.postgres.
Postgres
(interactive=False, quiet=True, dbinfo=None, determine_setup=True)[source]¶ Bases:
aiida.manage.external.pgsu.PGSU
Adds convenience functions to pgsu.Postgres.
- Provides conenience functions for
creating/dropping users
creating/dropping databases
etc. See pgsu.Postgres for implementation details.
Example:
postgres = Postgres() postgres.create_dbuser('username', 'password') if not postgres.db_exists('dbname'): postgres.create_db('username', 'dbname')
-
__module__
= 'aiida.manage.external.postgres'¶
-
check_db_name
(dbname)[source]¶ Looks up if a database with the name exists, prompts for using or creating a differently named one.
-
create_db
(dbuser, dbname)[source]¶ Create a database in postgres
- Parameters
dbuser – (str), Name of the user which should own the db.
dbname – (str), Name of the database.
-
create_dbuser
(dbuser, dbpass)[source]¶ Create a database user in postgres
- Parameters
dbuser – (str), Name of the user to be created.
dbpass – (str), Password the user should be given.
-
db_exists
(dbname)[source]¶ Check wether a postgres database with dbname exists
- Parameters
dbname – Name of the database to check for
- Returns
(bool), True if database exists, False otherwise
-
dbuser_exists
(dbuser)[source]¶ Find out if postgres user with name dbuser exists
- Parameters
dbuser – (str) database user to check for
- Returns
(bool) True if user exists, False otherwise
-
drop_db
(dbname)[source]¶ Drop a database in postgres
- Parameters
dbname – (str), Name of the database.
-
drop_dbuser
(dbuser)[source]¶ Drop a database user in postgres
- Parameters
dbuser – (str), Name of the user to be dropped.
-
classmethod
from_profile
(profile, **kwargs)[source]¶ Create Postgres instance with dbinfo from AiiDA profile data.
- Note: This only uses host and port from the profile, since the others are not going to be relevant for the
database superuser.
- Parameters
profile – AiiDA profile instance
kwargs – keyword arguments forwarded to Postgres constructor
- Returns
Postgres instance pre-populated with data from AiiDA profile
-
class
aiida.manage.external.postgres.
PostgresConnectionMode
[source]¶ Bases:
enum.IntEnum
Describe mode of connecting to postgres.
-
DISCONNECTED
= 0¶
-
PSQL
= 2¶
-
PSYCOPG
= 1¶
-
__module__
= 'aiida.manage.external.pgsu'¶
-
Components to communicate tasks to RabbitMQ.
-
exception
aiida.manage.external.rmq.
RemoteException
[source]¶ Bases:
Exception
An exception occurred at the remote end of the call
-
__module__
= 'kiwipy.communications'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
aiida.manage.external.rmq.
CommunicationTimeout
¶ alias of
concurrent.futures._base.TimeoutError
-
exception
aiida.manage.external.rmq.
DeliveryFailed
[source]¶ Bases:
Exception
Failed to deliver a message
-
__module__
= 'kiwipy.communications'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
-
class
aiida.manage.external.rmq.
ProcessLauncher
(loop=None, persister=None, load_context=None, loader=None)[source]¶ Bases:
plumpy.process_comms.ProcessLauncher
A sub class of plumpy.ProcessLauncher to launch a Process.
It overrides the _continue method to make sure the node corresponding to the task can be loaded and that if it is already marked as terminated, it is not continued but the future is reconstructed and returned
-
__module__
= 'aiida.manage.external.rmq'¶
-
_continue
(communicator, pid, nowait, tag=None)[source]¶ Continue the task.
Note that the task may already have been completed, as indicated from the corresponding the node, in which case it is not continued, but the corresponding future is reconstructed and returned. This scenario may occur when the Process was already completed by another worker that however failed to send the acknowledgment.
- Parameters
communicator – the communicator that called this method
pid – the pid of the process to continue
nowait – if True don’t wait for the process to finish, just return the pid, otherwise wait and return the results
tag – the tag of the checkpoint to continue from
-
static
handle_continue_exception
(node, exception, message)[source]¶ Handle exception raised in _continue call.
If the process state of the node has not yet been put to excepted, the exception was raised before the process instance could be reconstructed, for example when the process class could not be loaded, thereby circumventing the exception handling of the state machine. Raising this exception will then acknowledge the process task with RabbitMQ leaving an uncleaned node in the CREATED state for ever. Therefore we have to perform the node cleaning manually.
- Parameters
exception – the exception object
message – string message to use for the log message
-