AiiDA offers two avenues for sharing data with others: archive files and the REST API.
You have performed your calculations with AiiDA and you would like to share your AiiDA provenance graph, for example to make your scientific study reproducible.
Since AiiDA keeps track of the provenance of every computed result, this step is easy: Tell AiiDA the final results you would like to be reproducible, and AiiDA will automatically include their entire provenance using the Traversal Rules.
Let’s say the key results of your study are contained in three AiiDA nodes with PKs 12, 123, 1234. Exporting those results together with their provenance is as easy as:
12
123
1234
$ verdi export create my-calculations.aiida --nodes 12 123 1234
As usual, you can use any identifier (label, PK or UUID) to specify the nodes to be exported.
The resulting archive file my-calculations.aiida contains all information pertaining to the exported nodes. The default traversal rules make sure to include the complete provenance of any node specified and should be sufficient for most cases. See verdi export create --help for ways to modify the traversal rules.
my-calculations.aiida
verdi export create --help
Tip
If you want to make sure the archive includes everything you intended, you can create a new profile and import it:
$ verdi quicksetup --profile test-export # create new profile $ verdi --profile test-export import my-calculations.aiida
Now use, e.g. the QueryBuilder to query the database.
QueryBuilder
Please remember to use UUIDs when pointing your colleagues to data inside an AiiDA archive, since UUIDs are guaranteed to be universally unique (while PKs aren’t).
If the number of results to be exported is large, for example in a high-throughput study, use the QueryBuilder to add the corresponding nodes to a group my-results (see How to group nodes). Then export the group:
my-results
$ verdi export create my-calculations.aiida --groups my-results
AiiDA archive files can be published on any research data repository, for example the Materials Cloud Archive, Zenodo, or the Open Science Framework. When publishing AiiDA archives on the Materials Cloud Archive, you also get an interactive EXPLORE section, which allows peers to browse the AiiDA provenance graph directly in the browser.
Use verdi import to import AiiDA archives into your current AiiDA profile. verdi import accepts URLs, e.g.:
verdi import
$ verdi import "https://archive.materialscloud.org/record/file?file_id=2a59c9e7-9752-47a8-8f0e-79bcdb06842c&filename=SSSP_1.1_PBE_efficiency.aiida&record_id=23"
During import, AiiDA will avoid identifier collisions and node duplication based on UUIDs (and email comparisons for User entries). By default, existing entities will be updated with the most recent changes. Node extras and comments have special modes for determining how to import them - for more details, see verdi import --help.
User
verdi import --help
The AiiDA archive format has evolved over time, but you can still import archives created with previous AiiDA versions. If an outdated archive version is detected during import, the archive file will be automatically migrated to the newest version (within a temporary folder) and the import retried.
You can also use verdi export migrate to create updated archive files from existing archive files (or update them in place).
verdi export migrate
In order to get a quick overview of an archive file without importing it into your AiiDA profile, use verdi export inspect:
verdi export inspect
$ verdi export inspect sssp-efficiency.aiida -------------- ----- Version aiida 1.2.1 Version format 0.9 Computers 0 Groups 0 Links 0 Nodes 85 Users 1 -------------- -----
Note: For archive versions 0.2 and below, the overview may be inaccurate.
The AiiDA REST API allows to query your AiiDA database over HTTP(S) and returns results in JSON format.
Note
As of October 2020, the AiiDA REST API only supports GET methods (reading); in particular, it does not yet support workflow management. This feature is, however, part of the AiiDA roadmap.
GET
Start serving data from your default AiiDA profile via the REST API:
$ verdi restapi * REST API running on http://127.0.0.1:5000/api/v4 * Serving Flask app "aiida.restapi.run_api" (lazy loading) * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: off * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
The REST API is now running on port 5000 of your local computer.
5000
Like all verdi commands, you can select a different AiiDA profile via the -p PROFILE option:
verdi
-p PROFILE
verdi -p <another_profile> restapi
REST API version history:
aiida-core >= 1.0.0b6: v4. Simplified endpoints; only /nodes, /processes, /calcjobs, /groups, /computers and /servers remain. aiida-core >= 1.0.0b3, <1.0.0b6: v3. Development version, never shipped with a stable release. aiida-core <1.0.0b3: v2. First API version, with new endpoints added step by step.
aiida-core >= 1.0.0b6: v4. Simplified endpoints; only /nodes, /processes, /calcjobs, /groups, /computers and /servers remain.
aiida-core
v4
/nodes
/processes
/calcjobs
/groups
/computers
/servers
aiida-core >= 1.0.0b3, <1.0.0b6: v3. Development version, never shipped with a stable release.
v3
aiida-core <1.0.0b3: v2. First API version, with new endpoints added step by step.
v2
A URL to query the REST API consists of:
The base URL, by default:
http://127.0.0.1:5000/api/v4
Querying the base URL returns a list of all available endpoints.
The path defining the requested resource, optionally followed by a more specific endpoint. For example:
/nodes /nodes/page/2 /nodes/projectable_properties /nodes/<uuid> /nodes/<uuid>/links/outgoing
If no endpoint is appended, the API returns a list of objects of that resource. In order to request a specific object of a resource, append its UUID.
As usual, you can use partial UUIDs as long as they are unique.
In order to query by PK you need to use the id filter (see below). This also applies to User s, which don’t have UUIDs (but instead uses email).
id
(Optional) The query string for filtering, ordering and pagination of results. For example:
?limit=20&offset=35 ?id=200 ?node_type=like="data%"
Here are some examples to try:
http://127.0.0.1:5000/api/v4/users/ http://127.0.0.1:5000/api/v4/computers?scheduler_type="slurm" http://127.0.0.1:5000/api/v4/nodes/?id>45&node_type=like="data%"
The interactive EXPLORE sections on Materials Cloud are all powered by the AiiDA REST API and you can query the underlying API, either using your web browser or using a tool like curl:
curl
$ curl https://aiida-dev.materialscloud.org/2dstructures/api/v4/users
For an extensive user documentation of the endpoints, the query string as well as the format of the responses, see the AiiDA REST API reference.
The verdi restapi command runs the REST API through the werkzeug python-based HTTP server. In order to deploy production instances of the REST API for serving your data to others, we recommend using a fully fledged web server, such as Apache or NGINX, which then runs the REST API python application through the web server gateway interface (WSGI).
verdi restapi
werkzeug
One Apache/NGINX server can host multiple instances of the REST APIs, e.g. serving data from different AiiDA profiles.
A myprofile-rest.wsgi script for an AiiDA profile myprofile would look like this:
myprofile-rest.wsgi
myprofile
# -*- coding: utf-8 -*- # wsgi script for AiiDA profile 'myprofile' from aiida.restapi.run_api import configure_api from aiida.manage.configuration import load_profile load_profile('myprofile') api = configure_api() application = api.app
See the documentation of configure_api() for all available configuration options.
configure_api()
In the following, we explain how to run this wsgi application using Apache on Ubuntu.
Install and enable the mod_wsgi WSGI module module: $ sudo apt install libapache2-mod-wsgi-py3 $ sudo a2enmod wsgi Place the WSGI script in a folder on your server, for example /home/ubuntu/wsgi/myprofile-rest.wsgi. Configure apache to run the WSGI application using a virtual host configuration similar to: # Apache virtual host configuration file for AiiDA REST API # Copy to /etc/apache2/sites-enabled/aiida-rest.conf <VirtualHost *:80> LogLevel debug # Let the app do authorization WSGIPassAuthorization On # Require privileges on the wsgi directory <Directory "/home/ubuntu/wsgi"> Require all granted </Directory> # BEGIN SECTION for "myprofile" AiiDA profile # Use 5 threads and "aiida" virtual python environment WSGIDaemonProcess rest-myprofile \ user=ubuntu group=ubuntu \ threads=5 \ python-home=/home/ubuntu/.virtualenvs/aiida \ display-name=aiida-rest-myprofile # REST API will be served on <host>/myprofile/api/v4 WSGIScriptAlias /myprofile /home/ubuntu/wsgi/myprofile-rest.wsgi <Location /myprofile> WSGIProcessGroup myprofile </Location> # END SECTION for "myprofile" AiiDA profile </VirtualHost> Place this aiida-rest.conf file in /etc/apache2/sites-enabled Restart apache: sudo service apache2 restart.
Install and enable the mod_wsgi WSGI module module:
mod_wsgi
$ sudo apt install libapache2-mod-wsgi-py3 $ sudo a2enmod wsgi
Place the WSGI script in a folder on your server, for example /home/ubuntu/wsgi/myprofile-rest.wsgi.
/home/ubuntu/wsgi/myprofile-rest.wsgi
Configure apache to run the WSGI application using a virtual host configuration similar to:
# Apache virtual host configuration file for AiiDA REST API # Copy to /etc/apache2/sites-enabled/aiida-rest.conf <VirtualHost *:80> LogLevel debug # Let the app do authorization WSGIPassAuthorization On # Require privileges on the wsgi directory <Directory "/home/ubuntu/wsgi"> Require all granted </Directory> # BEGIN SECTION for "myprofile" AiiDA profile # Use 5 threads and "aiida" virtual python environment WSGIDaemonProcess rest-myprofile \ user=ubuntu group=ubuntu \ threads=5 \ python-home=/home/ubuntu/.virtualenvs/aiida \ display-name=aiida-rest-myprofile # REST API will be served on <host>/myprofile/api/v4 WSGIScriptAlias /myprofile /home/ubuntu/wsgi/myprofile-rest.wsgi <Location /myprofile> WSGIProcessGroup myprofile </Location> # END SECTION for "myprofile" AiiDA profile </VirtualHost>
Place this aiida-rest.conf file in /etc/apache2/sites-enabled
aiida-rest.conf
/etc/apache2/sites-enabled
Restart apache: sudo service apache2 restart.
sudo service apache2 restart
You should now be able to reach your REST API at localhost/myprofile/api/v4 (Port 80).
localhost/myprofile/api/v4