Updating AiiDA

Instructions

Warning

The following instructions are how to update from v0.12.* to v1.0.0. Each version increase may come with its own necessary migrations and you should only ever update the version by one at a time. To find the instructions for older versions, refer to the table below.

  1. Finish all running calculations. After migrating your database, you will not be able to resume unfinished calculations. Data of finished calculations will of course be automatically migrated.

  2. Finish all running legacy workflows. The legacy workflows are completely deprecated and all data will be removed from your database, so make sure to create a backup (see point 5).

  3. Enter the python environment where AiiDA is installed

  4. Stop the daemon using verdi daemon stop

  5. Create a backup of your database and repository

Warning

Once you have migrated your database, you can no longer go back to an older version of aiida-core, unless you restore your database and repository from a backup of course. In addition, the data migration can take quite some time depending on the size of your database, so please be patient. Big databases of multiple millions of nodes can take up to a few hours to migrate.

  1. Update your aiida-core installation

    • If you have installed AiiDA through pip simply run: pip install --upgrade aiida-core

    • If you have installed from the git repository using pip install -e ., first delete all the .pyc files (find . -name "*.pyc" -delete) before updating your branch.

  2. Finally, after having upgraded the installation, migrate your database with verdi -p <profile_name> database migrate

After the database migration finishes, you will be able to continue working with your existing data. However, backwards incompatible changes were introduced in the python API, so you probably will have to update your code and installed plugins.

Backwards incompatible changes

The following list covers the most important backward incompatible changes between aiida-core==0.12.* and aiida-core==1.0.0.

General

  • Reorganization of some second-level modules. Various classes and functions have been moved, renamed or replaced. See this page for the rules on the publicness of parts of the python API [#2357]

  • The module aiida.work has been renamed to aiida.engine and reorganized significantly [#2524]

  • The module aiida.scheduler has been renamed to aiida.schedulers [#2498]

  • The module aiida.transport has been renamed to aiida.transports [#2498]

  • The module aiida.utils has been removed and its contents have been placed in other places where appropriate [#2357]

  • Requirements were moved from setup_requirements.py to setup.json [#2307]

ORM

In order to define a clearer interface for users to the AiiDA entities, and allow (in the future) to swap between different profiles, the underlying hierarchy of nodes and links has been reworked (and a new paradigm has been implemented, with a “frontend” class that all users will interact with, and a “backend” class that users should never use directly, that implements the interaction to the database with a common interface, independent of the underlying ORM). The reorganisation of nodes and linktypes is mostly invisible to the user, but existing import statements will need to be updated.

  • Refactoring the ORM [#2190][#2210][#2227][#2225][#2481][#2544]

  • Renamed the node classes for the various process types

  • WorkCalculation to WorkChainNode [#2192]

  • FunctionCalculation to WorkFunctionNode [#2189]

  • JobCalculation to CalcJobNode [#2184] [#2201]

  • InlineCalculation to CalcFunctionNode [#2195]

  • Reorganization of the aiida.orm module, all node sub classes are now located under aiida.orm.nodes [#2402][#2497]

  • Make Code a real sub class of Data [#2193]

  • Implement new link types [#2220]

    • CREATE: CalculationNode -> Data

    • RETURN: WorkflowNode -> Data

    • INPUT_CALC: Data -> CalculationNode

    • INPUT_WORK: Data -> WorkflowNode

    • CALL_CALC: WorkflowNode -> CalculationNode

    • CALL_WORK: WorkflowNode -> WorkflowNode

  • Moved the plugin factories to aiida.plugins.factories [#2498]

  • Methods that operated on the repository of a Node instance have been moved to a Repository utility that is exposed through the Node.repository property [#2506]

  • Removed the Error data sub class and its entry point [#2529]

  • Removed the FrozenDict data sub class and its entry point [#2532]

  • Renamed the ParameterData data sub class to Dict [#2517]

QueryBuilder

  • Changed relationship indicator keywords, e.g. input_of is now with_outgoing. [#2224][#2278]

  • Changed type of UUIDs returned by the QueryBuilder to always be of type unicode [#2259]

Group

  • Change group type strings [#2329]

    • data.upf.family to data.upf

    • aiida.import to auto.import

    • autogroup.run to auto.run

    • custom ones to user

  • Remove Group.query and Group.group_query methods have been removed [#2329]

  • Renamed type column of Group database model to type_string [#2329]

  • Renamed name column of Group database model to label [#2329]

  • Class method Group.get_or_create has been removed, use the collection method Group.objects.get_or_create instead

  • Class method Group.get_from_string has been removed, use the class method Group.get instead

Node

  • The column type has been renamed to node_type [#2522]

  • The methods get_inputs, get_outputs, get_inputs_dict and get_outputs_dict have been removed and replace by get_incoming and get_outgoing [#2236]

  • Removed the link manager methods Node.inp and Node.out and the functionality has partially been replaced by: [#2569]

    • The link manager properties inputs and outputs for the CalculationNode and WorkflowNode classes.

    • Added the Data.creator property

    • Added the ProcessNode.caller property

    • Functionality to traverse the graph with tab completion when there is no uniqueness on the label is no longer supported and get_incoming and get_outgoing should be used instead.

  • The classes Node, ProcessNode can no longer be stored but only their sub classes [#2301]

Data

  • Kind.is_alloy() has been changed to a property Kind.is_alloy [#2374]

  • Kind.has_vacancies() has been changed to a property Kind.has_vacancies [#2374]

  • StructureData.is_alloy() has been changed to a property StructureData.is_alloy [#2374]

  • StructureData.has_vacancies() has been changed to a property StructureData.has_vacancies [#2374]

  • CifData._get_aiida_structure() has been renamed to CifData.get_structure(). [#2422]

  • CifData default library used in get_structure to convert to StructureData has been changed from ase to pymatgen [#1257]

  • SinglefileData the methods get_file_content, add_path and remove_path have been removed in favor of put_object_from_file and get_content [#2506]

  • ArrayData.iterarrays() has been renamed to ArrayData.get_iterarrays(). [#2422]

  • TrajectoryData._get_cif() has been renamed to TrajectoryData.get_cif(). [#2422]

  • TrajectoryData._get_aiida_structure() has been renamed to TrajectoryData.get_structure(). [#2422]

  • StructureData._get_cif() has been renamed to StructureData.get_cif(). [#2422]

  • Code.full_text_info() has been renamed to Code.get_full_text_info(). [#2422]

  • Code.is_hidden() has been renamed and changed to Code.hidden property. [#2422]

  • RemoteData.is_empty() has been changed to a property RemoteData.is_empty. [#2422]

  • The arguments stepids and cells of the TrajectoryData.set_trajectory() method are made optional which has implications on the ordering of the arguments passed to this method. [#2422]

  • The list of atomic symbols for TrajectoryData is no longer stored as array data but is now accessible through the TrajectoryData.symbols attribute. [#2422]

  • Removed deprecated methods BandsData._prepare_dat_1 and BandsData._prepare_dat_2 [#3114]

  • Removed deprecated method KpoinstData.bravais_lattice [#3114]

  • Removed deprecated method KpoinstData._set_bravais_lattice [#3114]

  • Removed deprecated method KpoinstData._get_or_create_bravais_lattice [#3114]

  • Removed deprecated method KpoinstData.set_kpoints_path [#3114]

  • Removed deprecated method KpoinstData._find_bravais_info [#3114]

  • Removed deprecated method KpoinstData.find_bravais_lattice [#3114]

  • Removed deprecated method KpoinstData.get_special_kpoints [#3114]

Process

  • Metadata inputs that used to start with an underscore (_label, _description and _options) no longer use an underscore and have moved within the metadata namespace [#1105]

  • Non-storable input ports are now markable as such through the non_db keyword [#1105]

Inline calculations

  • The make_inline and optional_inline decorators have been replaced by calcfunction. [#2203]

JobCalculation

In the new engine, it is not possible to launch calculation jobs by first creating an instance of the Calculation and then calling the calculation.use_xxx methods, as it was common in early versions of AiiDA. Instead, you need to pass the correct Calculation class to the run or submit function, passing the nodes to link as input as kwargs. For the past few versions, we have kept back-compatibility by supporting both ways of submitting. In version 1.0 we have decided to keep only one single way of submitting calculations for simplicity.

  • JobCalculation has been replaced by CalcJob process class [#2389]

  • Custom methods on the node class should now be implemented through a CalculationTools plugin [#2331]

  • Explicit set_ methods of the JobCalculation have been replaced with generic set_option method [#2361]

  • Explicit get_ methods of the JobCalculation have been replaced with generic get_option method [#1961]

  • New calculation job states have been introduced set as an attribute, only to be used for querying [#2389]

  • The DbCalcState table that recorded the old job state of JobCalculations has been removed [#2389]

Parser

  • parse_from_retrieved has been renamed to parse. In addition the arguments and return signatures have changed, for details see the PR [#2397]

WorkChain

  • The free function submit in any WorkChain should be replaced with self.submit.

  • The future returned by submit no longer has the pid attribute but rather pk.

  • The workfunction decorator can only be used for functions that return one of the inputs they receive, for all other use the calcfunction

  • The get_inputs_template class method has been replaced by get_builder. See the section on the process builder in the documentation on how to use it.

  • The input_group has been deprecated and been replaced by namespaces. See the section on ports in the documentation on how to use them.

  • The use of a . (period) in output keys is not supported in Process.out because that is now reserved to indicate namespaces.

Legacy workflows

  • Remove implementation of legacy workflows [#2379]

verdi

The verdi command line interface has been migrated over to a new system (called click), making the interface of all verdi commands consistent: now the way to specify a node (via a PK, a UUID or a LABEL) is the same for all commands, and command-line options that have the same meaning use the same flags in all commands. To make this possible, the interface of various verdi commands has been changed to ensure consistency. Also the output of most commands has been homogenised (e.g. to print errors or warnings always in the same style). Moreover, some of the commands have been renamed to be consistent with the new names of the classes in AiiDA.

  • Removed verdi data plugins in favor of verdi plugin list [#3114]

  • Removed verdi code rename in favor of verdi code relabel [#3114]

  • Removed verdi code update in favor of verdi code duplicate [#3114]

  • Removed verdi work in favor of verdi process [#2574]

  • Removed verdi calculation in favor of verdi process and verdi calcjob [#2574]

  • Removed verdi workflows [#2379]

  • Deprecated the commands to set and get config options verdi devel *property* in favor of verdi config [#2354]

  • verdi code show no longer shows number of calculations by default to improve performance, with --verbose flag to restore old behavior [#1428]

  • The tab-completion activation for verdi has changed, simply replace the eval "$(verdi completioncommand)" line in your activation script with eval "$(_VERDI_COMPLETE-source verdi)"

Daemon

  • Each profile now has its own daemon that can be run completely independently in parallel, so verdi daemon configureuser has been removed [#1217]

  • Replaced Celery with Circus as the daemonizer of the daemon [#1213]

Schedulers

  • Renamed aiida.daemon.execmanager.job_states to JOB_STATES, conforming to python conventions [#1799]

  • Abstract method aiida.scheduler.Scheduler._get_detailed_jobinfo_command() raises aiida.common.exceptions.FeatureNotAvailable (was NotImplemented).

  • Moved the SchedulerFactory to aiida.plugins.factories [#2498]

Transports

  • Moved the TransportFactory to aiida.plugins.factories [#2498]

Export import

  • New export archive format introduced v0.6. Older archives will automatically be converted when using verdi import, or alternatively can be manually exported using verdi export migrate

Older versions

To determine the current version of your installation use verdi --version. If the command does not exist, you have an older version of AiiDA, in which case you need to get it from the aiida.__init__.py file. Update instructions for older versions can be found in the documentation of the corresponding version: