Detailed Installation¶

Installing Prerequisites¶

AiiDA is designed to run on Unix operating systems and requires the following software:

bash or zsh (The shell)
python >= 3.5 (The programming language used by AiiDA)
python3-pip (Python 3 package manager)
postgresql (Database software, version 9.4 or higher)
RabbitMQ (A message broker necessary for AiiDA to communicate between processes)

PostgreSQL and RabbitMQ as services

PostgreSQL and RabbitMQ can also be configured to run as services that run on other machines. When setting up a profile, after having installed AiiDA, you can specify how to connect to these machines. With this setup, it is not necesseray to install PostgreSQL nor RabbitMQ on the machine where AiiDA is installed.

Depending on your set up, there are a few optional dependencies:

git (Version control system used for AiiDA development)
graphviz (For plotting AiiDA provenance graphs)

AiiDA has been tested on the following platforms:

Ubuntu 14.04, 16.04, 18.04
Mac OS X

We expect AiiDA to also run on:

Older and newer Ubuntu versions
Other Linux distributions
Windows Subsystem for Linux (WSL)

Below, we provide installation instructions for a number of operating systems.

Ubuntu¶

To install the prerequisites on Ubuntu and any other Debian derived distribution, you can use the apt package manager. The following will install the basic Python requirements and the git source control manager:

$ sudo apt-get install git python3-dev python3-pip

To install the requirements for the postgres database run the following:

$ sudo apt-get install postgresql postgresql-server-dev-all postgresql-client

Install the RabbitMQ message broker:

$ sudo apt-get install rabbitmq-server

This installs and adds RabbitMQ as a system service. To check whether it is running:

$ sudo rabbitmqctl status

If it is not running already, it should after a reboot.

Finally install the aiida-core Python environment:

$ sudo git python3-dev python3-pip
$ pip install aiida-core

Further Reading

For a more detailed description of database requirements and usage see the database topic.
For problems with installing RabbitMQ, refer to the detailed instructions provided on the RabbitMQ website for Debian based distributions.

Mac OS X (homebrew)¶

For Mac OS we recommend using the Homebrew package manager. If you have not installed Homebrew yet, see their installation guide.

After you have installed Homebrew, you can install the basic requirements as follows:

$ brew install postgresql rabbitmq git python

To start the postgres database server and rabbitmq service, execute:

$ brew services start postgresql
$ brew services start rabbitmq

You can check whether it is running by checking the status through the command:

$ /usr/local/sbin/rabbitmqctl status

Finally install the aiida-core python environment:

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ python get-pip.py
$ pip install aiida-core

Further Reading

For a more detailed description of database requirements and usage see the database topic.
For problems with installing RabbitMQ, refer to the detailed instructions provided on the RabbitMQ website for Homebrew based distributions.
For details on the installation of the pip package manager, refer to their documentation

Mac OS X (MacPorts)¶

Another package manager for MacOS is macports.

$ sudo port install postgresql postgresql-server rabbitmq-server git python

To start the postgres database server, run:

$ sudo su postgres
$ pg_ctl -D /opt/local/var/db/postgresql96/defaultdb start

To start the rabbitmq server, run:

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.rabbitmq-server.plist

You can check whether it is running as follows:

$ sudo rabbitmqctl status
  # this starts ``rabbitmq`` at system startup:
$ sudo port load rabbitmq-server

Finally install the aiida-core python environment:

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ python get-pip.py
$ pip install aiida-core

Trouble Installing RabbitMQ?

Be sure to install rabbitmq-server 3.7.9 or later. If rabbitmqctl status returns an error “Hostname mismatch”, the easiest solution can be to simply sudo port uninstall the package and install it again.

Further Reading

For a more detailed description of database requirements and usage see the database topic.
For details on the installation of the pip package manager, refer to their documentation

Gentoo Linux¶

To install RabbitMQ on a Gentoo distribution through the portage package manager run the following command:

$ emerge -av rabbitmq-server

To make sure that RabbitMQ is started at system boot, execute:

rc-update add rabbitmq

If you want to manually start the RabbitMQ server you can use:

/etc/init.d/rabbitmq start

Make sure that RabbitMQ is running with:

rabbitmqctl status

Trouble Installing RabbitMQ?

If you have encounter the following error
Argument '-smp enable' not supported."
Remove the mentioned option from the file /usr/libexec/rabbitmq/rabbitmq-env and restart the server. If you still have trouble getting RabbitMQ to run, please refer to the detailed instructions provided on the website of RabbitMQ itself for generic Unix systems.

Windows Subsystem for Linux (Ubuntu)¶

The guide for Ubuntu above can generally be followed, but there are a few things to note:

Tip

Installing Ubuntu instead of the version specific applications, will let you have the latest LTS version.

The Windows native RabbitMQ should be installed and started. (For WSL 2, this should not be necessary.)
Linux services under WSL are not started automatically. To start the PostgreSQL and RabbitMQ-server services, type the commands below in the terminal:
```
sudo service postgresql start
sudo service rabbitmq-server start
```
Tip

These services may be run at startup without passing a password in the following way:

Create a .sh file with the lines above, but without sudo. Make the file executeable, i.e., type:
```
$ chmod +x /path/to/file.sh
$ sudo visudo
```
And add the line:
```
<username> ALL=(root) NOPASSWD: /path/to/file.sh
```
Replacing <username> with your Ubuntu username. This will allow you to run only this specific .sh file with root access (without password), without lowering security on the rest of your system.
There is a known issue in WSL Ubuntu 18.04 where the timezone is not configured correctly out-of-the-box, which may cause problem for the database. The following command can be used to re-configure the time zone:
```
$ sudo dpkg-reconfigure tzdata
```
The file open limit may need to be raised using ulimit -n 2048 (default is 1024), when running tests. You can check the limit by using ulimit -n.

Hint

This may need to be run every time the system starts up.

It may be worth considering adding some of these commands to your ~/.bashrc file, since some of these settings may reset upon reboot.

Further Reading

For using WSL as a developer, please see the considerations made in our wiki-page for developers.

Using virtual environments¶

AiiDA depends on a number of third party python packages, and usually on specific versions of those packages. In order to not interfere with third party packages needed by other software on your system, we strongly recommend isolating AiiDA in a virtual Python environment, for example, by means of one of the methods described below.

Additional Information

A very good tutorial on Python environments is provided by realpython.com.

venv¶

The venv module for creating virtual environments ships directly with Python. To create a virtual environment, in a given directory, run:

$ python3 -m venv /path/to/new/virtual/environment/aiida

The command to activate the environment is shell specific (see the documentation). With bash the following command is used:

$ source /path/to/new/virtual/environment/aiida/bin/activate

To leave or deactivate the environment, simply run:

(aiida) $ deactivate

Update install tools

You may need to update pip and setuptools in your virtual environment, in case the system or user version of these tools is old.

(aiida) $ pip install -U setuptools pip

Conda¶

If you have Conda installed then you can directly create a new environment with aiida-core and (optionally) the Postgres and RabbitMQ services installed.

$ conda create -n aiida -c conda-forge python=3.7 aiida-core aiida-core.services pip
$ conda activate aiida
$ conda deactivate

Installing the aiida-core package¶

AiiDA can be installed either from the python package index PyPI, Anaconda Cloud (both good for general use) or directly from the aiida-core github repository (good for developers).

With your virtual environment active (see above), install the aiida-core python package from PyPI using:

$ pip install aiida-core

Installing AiiDA in your system environment

Consider adding the --user flag to avoid the need for administrator privileges.

Or, if using Conda, from the Anaconda Cloud:

$ conda install -c conda-forge aiida-core aiida-core.services

Note

aiida-core.services is an optional package to also install PostgreSQL and RabbitMQ.

Alternatively, you can create a directory where to clone the AiiDA source code and install AiiDA from source:

$ mkdir <your_directory>
$ cd <your_directory>
$ git clone https://github.com/aiidateam/aiida-core
$ pip install -e aiida-core

There are additional optional packages that you may want to install, which are grouped in the following categories:

atomic_tools: packages that allow importing and manipulating crystal structure from various formats

ssh_kerberos: adds support for ssh transport authentication through Kerberos

REST: allows a REST server to be ran locally to serve AiiDA data

docs: tools to build the documentation

notebook: jupyter notebook - to allow it to import AiiDA modules

tests: python modules required to run the automatic unit tests

pre-commit: pre-commit tools required for developers to enable automatic code linting and formatting

In order to install any of these package groups, simply append them as a comma separated list in the pip install command

$ pip install -e "aiida-core[atomic_tools,docs]"

Kerberos on Ubuntu

If you are installing the optional ssh_kerberos and you are on Ubuntu you might encounter an error related to the gss package. To fix this you need to install the libffi-dev and libkrb5-dev packages:

$ sudo apt-get install libffi-dev libkrb5-dev

AiiDA uses the reentry package to keep a fast cache of entry points for a snappy verdi cli. After installing AiiDA packages, always remember to update the reentry cache using:

$ reentry scan

Setting up the installation¶

AiiDA profile quicksetup¶

After successful installation, you need to create an AiiDA profile via AiiDA’s command line interface verdi.

Most users should use the interactive quicksetup:

$ verdi quicksetup

which leads through the installation process and takes care of creating the corresponding AiiDA database.

For maximum customizability, one can use verdi setup, that provides fine-grained control in configuring how AiiDA should connect to the required services. This is useful, for example, if PostgreSQL and or RabbitMQ are not installed and configured with default settings, or are run on a different machine from AiiDA itself.

Don’t forget to backup your data!

See the installation backup how-to.

Want to manage multiple profiles?

See the managing profiles how-to.

AiiDA profile custom setup¶

Creating the database¶

AiiDA uses a database to store the nodes, node attributes and other information, allowing the end user to perform fast queries of the results. Currently, the highly performant PostgreSQL database is supported as a database backend.

Find out more about the database

To manually create the database for AiiDA, you need to run the program psql to interact with postgres. On most operating systems, you need to do so as the postgres user that was created upon installing the software. To assume the role of postgres run as root:

$ su - postgres

(or, equivalently, type sudo su - postgres, depending on your distribution) and launch the postgres program:

$ psql

Create a new database user account for AiiDA by running:

CREATE USER aiida WITH PASSWORD '<password>';

replacing <password> with a password of your choice.

You will need to provide the password again when you configure AiiDA to use this database through verdi setup. If you want to change the password you just created use the command:

ALTER USER aiida PASSWORD '<password>';

Next, we create the database itself. We enforce the UTF-8 encoding and specific locales:

CREATE DATABASE aiidadb OWNER aiida ENCODING 'UTF8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE=template0;

and grant all privileges on this DB to the previously-created aiida user:

GRANT ALL PRIVILEGES ON DATABASE aiidadb to aiida;

You have now created a database for AiiDA and you can close the postgres shell by typing \q. To test if the database was created successfully, you can run the following command as a regular user in a bash terminal:

$ psql -h localhost -d aiidadb -U aiida -W

and type the password you inserted before, when prompted. If everything worked well, you should get no error and see the prompt of the psql shell.

If you use the same names as in the example commands above, then during the verdi setup phase the following parameters will apply to the newly created database:

$ Database engine: postgresql_psycopg2
$ Database host: localhost
$ Database port: 5432
$ AiiDA Database name: aiidadb
$ AiiDA Database user: aiida
$ AiiDA Database password: <password>

Don’t forget to backup your database!

See the Database backup how-to), and how to move your database.

Database setup using ‘peer’ authentication¶

On Ubuntu Linux, the default PostgreSQL setup is configured to use peer authentication, which allows password-less login via local Unix sockets. In this mode, PostgreSQL compares the Unix user connecting to the socket with its own database of users and allows a connection if a matching user exists.

Note

This is an alternative route to set up your database - the standard approach will work on Ubuntu just as well.

Below we are going to take advantage of the command-line utilities shipped on Ubuntu to simplify creating users and databases compared to issuing the SQL commands directly.

Assume the role of postgres:

$ sudo su postgres

Create a database user with the same name as the UNIX user who will be running AiiDA (usually your login name):

$ createuser <username>

replacing <username> with your username.

Next, create the database itself with your user as the owner:

$ createdb -O <username> aiidadb

Exit the shell to go back to your login user. To test if the database was created successfully, try:

$ psql aiidadb

During the verdi setup phase, use ! to leave host empty and specify your Unix user name as the AiiDA Database user.:

$ Database engine: postgresql_psycopg2
$ Database host: !
$ Database port: 5432
$ AiiDA Database name: aiidadb
$ AiiDA Database user: <username>
$ AiiDA Database password: ""

RabbitMQ configuration¶

In most normal setups, RabbitMQ will be installed and run as a service on the same machine that hosts AiiDA itself. In that case, using the default configuration proposed during a profile setup will work just fine. However, when the installation of RabbitMQ is not standard, for example it runs on a different port, or even runs on a completely different machine, all relevant connection details can be configured with verdi setup.

The following parameters can be configured:

Parameter	Option	Default	Explanation
Protocol	`--broker-protocol`	`amqp`	The protocol to use, can be either `amqp` or `amqps` for SSL enabled connections.
Username	`--broker-username`	`guest`	The username with which to connect. The `guest` account is available and usable with a default RabbitMQ installation.
Password	`--broker-password`	`guest`	The password with which to connect. The `guest` account is available and usable with a default RabbitMQ installation.
Host	`--broker-host`	`127.0.0.1`	The hostname of the RabbitMQ server.
Port	`--broker-port`	`5672`	The port to which the server listens.
Virtual host	`--broker-virtual-host`	`''`	Optional virtual host. If defined, needs to start with a forward slash.

verdi setup¶

After the database has been created, do:

$ verdi setup --profile <profile_name>

where <profile_name> is a profile name of your choosing. The verdi setup command will guide you through the setup process through a series of prompts.

The first information asked is your email, which will be used to associate the calculations to you. In AiiDA, the email is your username, and acts as a unique identifier when importing/exporting data from AiiDA.

Note

The password, in the current version of AiiDA, is not used (it will be used only in the REST API and in the web interface). If you leave the field empty, no password will be set and no access will be granted to the user via the REST API and the web interface.

Then, the following prompts will help you configure the database. Typical settings are:

$ Default user email: richard.wagner@leipzig.de
$ Database engine: postgresql_psycopg2
$ PostgreSQL host: localhost
$ PostgreSQL port: 5432
$ AiiDA Database name: aiida_dev
$ AiiDA Database user: aiida
$ AiiDA Database password: <password>
$ AiiDA repository directory: /home/wagner/.aiida/repository/
[...]
Configuring a new user with email 'richard.wagner@leipzig.de'
$ First name: Richard
$ Last name: Wagner
$ Institution: BRUHL, LEIPZIG
$ The user has no password, do you want to set one? [y/N] y
$ Insert the new password:
$ Insert the new password (again):

Don’t forget to backup your data!

See the installation backup how-to.

Starting the daemon¶

The AiiDA daemon process runs in the background and takes care of processing your submitted calculations and workflows, checking their status, retrieving their results once they are finished and storing them in the AiiDA database.

The AiiDA daemon is controlled using three simple commands:

verdi daemon start: start the daemon

verdi daemon status: check the status of the daemon

verdi daemon stop: stop the daemon

Note

While operational, the daemon logs its activity to a file in ~/.aiida/daemon/log/ (or, more generally, $AIIDA_PATH/.aiida/daemon/log). Get the latest log messages via verdi daemon logshow.

Final checks¶

Use the verdi status command to check that all services are up and running:

$ verdi status

✓ profile:     On profile quicksetup
✓ repository:  /repo/aiida_dev/quicksetup
✓ postgres:    Connected as aiida@localhost:5432
✓ rabbitmq:    Connected as amqp://127.0.0.1?heartbeat=600
✓ daemon:      Daemon is running as PID 2809 since 2019-03-15 16:27:52

In the example output, all service have a green check mark and so should be running as expected.

Are you in your virtual environment?

Remember that in order to work with AiiDA through for example the verdi command, you need to be in your virtual environment. If you open a new terminal for example, be sure to activate it first, e.g. for venv:

$ source ~/.virtualenvs/aiida/bin/activate

Want to manage your install?

See the Installation how-to

Configuring Updating Backing-up

What next?

Try out the Basic Tutorial.

Using AiiDA in Jupyter¶

Jupyter is an open-source web application that allows you to create in-browser notebooks containing live code, visualizations and formatted text.

Originally born out of the iPython project, it now supports code written in many languages and customized iPython kernels.

If you didn’t already install AiiDA with the [notebook] option (during pip install), run pip install jupyter inside the virtualenv, and then run from within the virtualenv:

$ jupyter notebook

This will open a tab in your browser. Click on New -> Python and type:

import aiida

followed by Shift-Enter. If no exception is thrown, you can use AiiDA in Jupyter.

If you want to set the same environment as in a verdi shell, add the following code to a .py file (create one if there isn’t any) in <home_folder>/.ipython/profile_default/startup/:

try:
    import aiida
except ImportError:
    pass
else:
    import IPython
    from aiida.tools.ipython.ipython_magics import load_ipython_extension

    # Get the current Ipython session
    ipython = IPython.get_ipython()

    # Register the line magic
    load_ipython_extension(ipython)

This file will be executed when the ipython kernel starts up and enable the line magic %aiida. Alternatively, if you have a aiida-core repository checked out locally, you can just copy the file <aiida-core>/aiida/tools/ipython/aiida_magic_register.py to the same folder. The current ipython profile folder can be located using:

$ ipython locate profile

After this, if you open a Jupyter notebook as explained above and type in a cell:

%aiida

followed by Shift-Enter. You should receive the message “Loaded AiiDA DB environment.” This line magic should also be enabled in standard ipython shells.

Using the Docker image¶

AiiDA maintains a Docker image on Docker Hub. This image contains a pre-configured AiiDA environment, and so is particularly useful for learning and testing purposes. It is a great way to quickly get started on the tutorials!

Follow Docker’s install guide to download Docker and start its daemon. Now you can pull the aiida-core image straight from Docker Hub, for a specific version.

$ docker pull aiidateam/aiida-core:1.2.1

We can start a container running by:

$ docker run -d --name aiida-container aiidateam/aiida-core:1.2.1

The container comes installed with all required services and, on start-up, will automatically start them and create an AiiDA profile (plus a localhost computer). To (optionally) wait for the services to start and inspect the start-up process, we can run:

$ docker exec -t aiida-container wait-for-services
$ docker logs aiida-container

The profile is created under the aiida username, so to execute commands use:

$ docker exec -t --user aiida aiida-container /bin/bash -l -c 'verdi status'
✓ config dir:  /home/aiida/.aiida
✓ profile:     On profile default
✓ repository:  /home/aiida/.aiida/repository/default
✓ postgres:    Connected as aiida_qs_aiida_477d3dfc78a2042156110cb00ae3618f@localhost:5432
✓ rabbitmq:    Connected as amqp://127.0.0.1?heartbeat=600
✓ daemon:      Daemon is running as PID 1795 since 2020-05-20 02:54:00

Or to enter into the container interactively:

$ docker exec -it --user aiida aiida-container /bin/bash

If you stop the container and start it again, any data you created will persist.

$ docker stop aiida-container
$ docker start aiida-container

But if you remove the container all data will be removed.

$ docker stop aiida-container
$ docker rm aiida-container

To store data and even share it between containers, you may wish to add a volume:

$ docker run -d --name aiida-container --mount source=my_data,target=/tmp/my_data aiidateam/aiida-core:1.2.1

Now anything that you save to the /tmp/my_data folder will be saved to the volume persistently. You can even add files directly to the folder outside of the container, by finding its mounting point:

$ docker volume inspect my_data
$ echo "hallo" | sudo tee -a /var/lib/docker/volumes/my_data/_data/hallo.txt  > /dev/null

What next?

Try out the Basic Tutorial.