aiida.backends.sqlalchemy package

aiida.backends.sqlalchemy.get_scoped_session()

Return a scoped session (according to SQLAlchemy docs, this returns always the same object within a thread, and a different object in a different thread. Moreover, since we update the scopedsessionclass upon forking, also forks have different session objects.

Subpackages

• aiida.backends.sqlalchemy.migrations package
  • Subpackages
    • aiida.backends.sqlalchemy.migrations.versions package
      • Submodules
  • Submodules
• aiida.backends.sqlalchemy.models package
  • Submodules
• aiida.backends.sqlalchemy.tests package
  • Subpackages
    • aiida.backends.sqlalchemy.tests.migration_test package
      • Subpackages
        • aiida.backends.sqlalchemy.tests.migration_test.versions package
          • Submodules
  • Submodules

Submodules

aiida.backends.sqlalchemy.cmdline.get_computers_work_dir(calculations, user)

Get a list of computers and their remotes working directory.

calculations should be a list of JobCalculation object.

aiida.backends.sqlalchemy.cmdline.get_group_list(user, type_string, n_days_ago=None, name_filters={})

aiida.backends.sqlalchemy.cmdline.get_log_messages(obj)

Get the log messages for the object.

aiida.backends.sqlalchemy.cmdline.get_workflow_list(pk_list=(), user=None, all_states=False, n_days_ago=None)

Get a list of workflow

Parameters:

• user – A ORM User class if you want to filter by user
• pk_list – Limit the results to this list of PKs
• all_states – if False, limit results to “active” (e.g., running) wfs
• n_days_ago – an integer number of days. If specifies, limit results to workflows started up to this number of days ago

Functions to manage the global settings stored in the DB (in the DbSettings table.

aiida.backends.sqlalchemy.globalsettings.del_global_setting(key)

Return the value of the given setting, or raise a KeyError if the setting is not present in the DB.

Raises:KeyError – if the setting does not exist in the DB

aiida.backends.sqlalchemy.globalsettings.get_global_setting(key)

Return the value of the given setting, or raise a KeyError if the setting is not present in the DB.

Raises:KeyError – if the setting does not exist in the DB

aiida.backends.sqlalchemy.globalsettings.get_global_setting_description(key)

Return the description for the given setting variable, as stored in the DB, or raise a KeyError if the setting is not present in the DB or the table doesn’t exist.

aiida.backends.sqlalchemy.globalsettings.set_global_setting(key, value, description=None)

Set a global setting in the DbSetting table (therefore, stored at the DB level).

aiida.backends.sqlalchemy.globalsettings.table_check_test()

Checks if the db_setting table exists in the database. If it doesn’t exist it rainses a KeyError.

class aiida.backends.sqlalchemy.queries.QueryManagerSQLA(*args, **kwargs)

Bases: aiida.backends.general.abstractqueries.AbstractQueryManager

SQLAlchemy implementation of custom queries, for efficiency reasons

__abstractmethods__ = frozenset([])

__module__ = 'aiida.backends.sqlalchemy.queries'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 40

_abc_registry = <_weakrefset.WeakSet object>

get_creation_statistics(user_pk=None)

Return a dictionary with the statistics of node creation, summarized by day, optimized for the Django backend.

Note:Days when no nodes were created are not present in the returned ctime_by_day dictionary. Parameters:user_pk – If None (default), return statistics for all users. If user pk is specified, return only the statistics for the given user. Returns:a dictionary as follows:

{
   "total": TOTAL_NUM_OF_NODES,
   "types": {TYPESTRING1: count, TYPESTRING2: count, ...},
   "ctime_by_day": {'YYYY-MMM-DD': count, ...}

where in ctime_by_day the key is a string in the format ‘YYYY-MM-DD’ and the value is an integer with the number of nodes created that day.

class aiida.backends.sqlalchemy.querybuilder_sqla.QueryBuilderImplSQLA(*args, **kwargs)

Bases: aiida.backends.general.querybuilder_interface.QueryBuilderInterface

QueryBuilder to use with SQLAlchemy-backend and schema defined in backends.sqlalchemy.models

AiidaComputer

AiidaGroup

AiidaNode

AiidaUser

Computer

Group

Link

Node

User

__abstractmethods__ = frozenset([])

__init__(*args, **kwargs)

x.__init__(…) initializes x; see help(type(x)) for signature

__module__ = 'aiida.backends.sqlalchemy.querybuilder_sqla'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 92

_abc_registry = <_weakrefset.WeakSet object>

_get_filter_expr_from_column(operator, value, column)

count(query)

Returns:the number of results

first(query)

Executes query in the backend asking for one instance.

Returns:One row of aiida results

get_aiida_res(key, res)

Some instance returned by ORM (django or SA) need to be converted to Aiida instances (eg nodes). Choice (sqlalchemy_utils) will return their value

Parameters:

• key – The key
• res – the result returned by the query

Returns:

an aiida-compatible instance

get_filter_expr(operator, value, attr_key, is_attribute, alias=None, column=None, column_name=None)

Applies a filter on the alias given. Expects the alias of the ORM-class on which to filter, and filter_spec. Filter_spec contains the specification on the filter. Expects:

Parameters:

• operator – The operator to apply, see below for further details
• value – The value for the right side of the expression, the value you want to compare with.
• path – The path leading to the value
• attr_key – Boolean, whether the value is in a json-column, or in an attribute like table.

Implemented and valid operators:

• for any type: * == (compare single value, eg: ‘==’:5.0) * in (compare whether in list, eg: ‘in’:[5, 6, 34]

• for floats and integers:

  • >
  • <
  • <=
  • >=

• for strings:

  • like (case - sensitive), for example ‘like’:’node.calc.%’ will match node.calc.relax and node.calc.RELAX and node.calc. but not node.CALC.relax
  • ilike (case - unsensitive) will also match node.CaLc.relax in the above example

  Note

  The character % is a reserved special character in SQL, and acts as a wildcard. If you specifically want to capture a % in the string, use: _%

• for arrays and dictionaries (only for the SQLAlchemy implementation):

  • contains: pass a list with all the items that the array should contain, or that should be among the keys, eg: ‘contains’: [‘N’, ‘H’])
  • has_key: pass an element that the list has to contain or that has to be a key, eg: ‘has_key’:’N’)

• for arrays only (SQLAlchemy version):

  • of_length
  • longer
  • shorter

All the above filters invoke a negation of the expression if preceded by ~:

# first example:
filter_spec = {
    'name' : {
        '~in':[
            'halle',
            'lujah'
        ]
    } # Name not 'halle' or 'lujah'
}

# second example:
filter_spec =  {
    'id' : {
        '~==': 2
    }
} # id is not 2

get_filter_expr_from_attributes(operator, value, attr_key, column=None, column_name=None, alias=None)

A classmethod that returns an valid SQLAlchemy expression.

Parameters:

• operator – The operator provided by the user (‘==’, ‘>’, …)
• value – The value to compare with, e.g. (5.0, ‘foo’, [‘a’,’b’])
• attr_key (str) – The path to that attribute as a tuple of values. I.e. if that attribute I want to filter by is the 2nd element in a list stored under the key ‘mylist’, this is (‘mylist’, ‘2’).
• column – Optional, an instance of sqlalchemy.orm.attributes.InstrumentedAttribute or
• column_name (str) – The name of the column, and the backend should get the InstrumentedAttribute.
• alias – The aliased class.

Returns:

An instance of sqlalchemy.sql.elements.BinaryExpression

get_ormclass(cls, ormclasstype)

Return the valid ormclass for the connections

get_projectable_attribute(alias, column_name, attrpath, cast=None, **kwargs)

Returns:An attribute store in a JSON field of the give column

get_session()

Returns:a valid session, an instance of sqlalchemy.orm.session.Session

iterall(query, batch_size, tag_to_index_dict)

Returns:An iterator over all the results of a list of lists.

iterdict(query, batch_size, tag_to_projected_entity_dict)

Returns:An iterator over all the results of a list of dictionaries.

modify_expansions(alias, expansions)

For sqlalchemy, there are no additional expansions for now, so I am returning an empty list

table_groups_nodes

yield_per(query, batch_size)

Parameters:count – Number of rows to yield per step

Yields count rows at a time

Returns:a generator

class aiida.backends.sqlalchemy.querybuilder_sqla.array_length(*clauses, **kwargs)

Bases: sqlalchemy.sql.functions.FunctionElement

__module__ = 'aiida.backends.sqlalchemy.querybuilder_sqla'

_compiler_dispatch(**kw)

_compiler_dispatcher = <sqlalchemy.ext.compiler._dispatcher object>

name = 'array_len'

aiida.backends.sqlalchemy.querybuilder_sqla.compile(element, compiler, **kw)

Get length of array defined in a JSONB column

class aiida.backends.sqlalchemy.querybuilder_sqla.jsonb_array_length(*clauses, **kwargs)

Bases: sqlalchemy.sql.functions.FunctionElement

__module__ = 'aiida.backends.sqlalchemy.querybuilder_sqla'

_compiler_dispatch(**kw)

_compiler_dispatcher = <sqlalchemy.ext.compiler._dispatcher object>

name = 'jsonb_array_len'

class aiida.backends.sqlalchemy.querybuilder_sqla.jsonb_typeof(*clauses, **kwargs)

Bases: sqlalchemy.sql.functions.FunctionElement

__module__ = 'aiida.backends.sqlalchemy.querybuilder_sqla'

_compiler_dispatch(**kw)

_compiler_dispatcher = <sqlalchemy.ext.compiler._dispatcher object>

name = 'jsonb_typeof'

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.aiidadb_backend_value_sqla = 'sqlalchemy'

This scipt transitions an Django database 0.6.0 to an SQLAlchemy database 0.7.0. It also updates the needed config files.

It is supposed to be executed via ipython in the following way:

from aiida.backends.sqlalchemy.transition import transition transition(profile=”your_profile”,group_size=10000)

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.attributes_to_dict(attr_list)

Transform the attributes of a node into a dictionary. It assumes the key are ordered alphabetically, and that they all belong to the same node.

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.change_backend_to_sqla(profile=None)

Gets the main AiiDA configuration and searches if there is a backend defined. If there isn’t any then Django is added.

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.create_gin_index()

Create the GIN index for the attributes column of db_dbnode.

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.modify_link_table()

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.select_from_key(key, d)

Return element of the dict to do the insertion on. If it is foo.1.bar, it will return d[“foo”][1]. If it is only foo, it will return d directly.

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.set_correct_schema_version_and_backend()

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition(profile=None, group_size=1000, delete_table=False)

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition_attributes(profile=None, group_size=1000, debug=False, delete_table=False)

Migrate the DbAttribute table into the attributes column of db_dbnode.

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition_config_files(profile=None)

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition_db(profile=None, group_size=1000, delete_table=False)

Migrate the attributes, extra, and some other columns to use JSONMigrate the attributes, extra, and some other columns to use JSONB column type.

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition_extras(profile=None, group_size=1000, delete_table=False)

Migrate the DbExtra table into the extras column for db_dbnode.

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition_json_column(profile=None)

Migrate the TEXT column containing JSON into JSON columns

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition_load_db_env(process=None, profile=None, *args, **kwargs)

aiida.backends.sqlalchemy.transition_06dj_to_07sqla.transition_settings(profile=None)

Migrate the DbAttribute table into the attributes column of db_dbnode.

aiida.backends.sqlalchemy.utils._load_dbenv_noschemacheck(process=None, profile=None, connection=None)

Load the SQLAlchemy database.

aiida.backends.sqlalchemy.utils.alembic_command(selected_command, *args, **kwargs)

This function calls the necessary alembic command with the provided arguments. :param selected_command: The command that should be called from the alembic commands. :param args: The arguments. :param kwargs: The keyword arguments. :return: Nothing.

aiida.backends.sqlalchemy.utils.check_schema_version(force_migration=False, alembic_cfg=None)

Check if the version stored in the database is the same of the version of the code.

Note:if the DbSetting table does not exist, this function does not fail. The reason is to avoid to have problems before running the first migrate call. Note:if no version is found, the version is set to the version of the code. This is useful to have the code automatically set the DB version at the first code execution. Raises:ConfigurationError – if the two schema versions do not match. Otherwise, just return.

aiida.backends.sqlalchemy.utils.delete_nodes_and_connections_sqla(pks_to_delete)

Delete all nodes corresponding to pks in the input. :param pks_to_delete: A list, tuple or set of pks that should be deleted.

aiida.backends.sqlalchemy.utils.dumps_json(d)

Transforms all datetime object into isoformat and then returns the JSON

aiida.backends.sqlalchemy.utils.get_alembic_conf()

This function returns the alembic configuration file contents by doing the necessary updates in the ‘script_location’ name. :return: The alembic configuration.

aiida.backends.sqlalchemy.utils.get_automatic_user()

aiida.backends.sqlalchemy.utils.get_daemon_user()

Return the username (email) of the user that should run the daemon, or the default AiiDA user in case no explicit configuration is found in the DbSetting table.

aiida.backends.sqlalchemy.utils.get_db_schema_version(config)

This function returns the current version of the database. :param config: The alembic configuration. :return: The version of the database.

aiida.backends.sqlalchemy.utils.get_migration_head(config)

This function returns the head of the migration scripts. :param config: The alembic configuration. :return: The version of the head.

aiida.backends.sqlalchemy.utils.get_pg_tc(links_table_name, links_table_input_field, links_table_output_field, closure_table_name, closure_table_parent_field, closure_table_child_field)

Return the transitive closure table template

aiida.backends.sqlalchemy.utils.install_tc(session)

Install the transitive closure table with SqlAlchemy.

aiida.backends.sqlalchemy.utils.load_dbenv(process=None, profile=None, connection=None)

Load the database environment (SQLAlchemy) and perform some checks.

Parameters:

• process – the process that is calling this command (‘verdi’, or ‘daemon’)
• profile – the string with the profile to use. If not specified, use the default one specified in the AiiDA configuration file.

aiida.backends.sqlalchemy.utils.loads_json(s)

Loads the json and try to parse each basestring as a datetime object

aiida.backends.sqlalchemy.utils.recreate_after_fork(engine)

Parameters:engine – the engine that will be used by the sessionmaker

Callback called after a fork. Not only disposes the engine, but also recreates a new scoped session to use independent sessions in the forked process.

aiida.backends.sqlalchemy.utils.reset_session(config)

Parameters:config – the configuration of the profile from the configuration file

Resets (global) engine and sessionmaker classes, to create a new one (or creates a new one from scratch if not already available)

aiida.backends.sqlalchemy.utils.set_daemon_user(user_email)

Return the username (email) of the user that should run the daemon, or the default AiiDA user in case no explicit configuration is found in the DbSetting table.