Automated parser tests

AiiDA testing facility can check for the proper functionality of parsers automatically. To facilitate the creation of new tests, we provide a simple tool to create a new parser test from a calculation that you already run in your AiiDA database, described below.

Test folders

Each folder inside the path aiida/backends/tests/parser_tests constitutes a single test. The naming convention for folders is the following:

  • it should contain only digits, letters and underscores, otherwise the folder will be ignored when running verdi devel tests db.parsers;
  • the folder name should start with test_;
  • the name should be followed by the parser plugin name, as returned by calculation.get_parser_name(), and with dots replaced with underscores;
  • it should be followed by an underscore;
  • finally it should be followed by a string that explains what is tested.

For instance, a valid name is test_quantumespresso_pw_vanderwaals. Note that the naming scheme is only a convention, and that the parser to use for the test is selected automatically.

Creation of a test from an existing calculation

In order to create the folder, you can open verdi shell while being in the folder aiida/backends/tests/parser_tests, import the following function:

from aiida.backends.tests.parsers import output_test

and then run it with the correct parameters. The documentation of the function can be found here.

An example call could be:

output_test(
    pk=21,
    testname='vanderwaals',
    skip_uuids_from_inputs=[
        'f579974c-6a9e-4eb4-9b41-e72486f86ac5',
        'ee0df234-955e-4f99-9808-17e168e6a769']
)

where:

  • 21 is the PK of the calculation that you want to export
  • vanderwaals is the name of the test: if for instance the node with pk=21 is a Quantum ESPRESSO pw.x calculation, the script will create a folder named test_quantumespresso_pw_vanderwaals
  • the (optional) skip_uuids_from_inputs is a list of UUIDs of input nodes that will not be exported.

The script will create a new folder, containing the exported content of the calculation, its direct inputs (except those listed in the skip_uuids_from_inputs list), and the output retrieved node. The format of the exported data is the same of the export files of AiiDA, but the folder is not zipped.

Note

The skip_uuids_from_inputs parameter is typically useful for input nodes containing large files that are not needed for parsing and would just create a large test; a typical example is given by pseudopotential input nodes for Quantum ESPRESSO.

After having run the command, the existence of the folder will only test that the parser is able to parse the calculation without errors. Typically, however, you will also want to check some parsed values.

In this case, you need to modify the _aiida_checks.json JSON file inside the folder. The syntax is the following:

  • each key represents an output node that should be generated by the parser;

  • each value is a dictionary with multiple keys (an empty dictionary will just check for the existence of the output node);

  • each key of the subdictionary is an attribute to check for. The value is a list of dictionaries, one for each test to perform on the given value; multiple tests are therefore possible. The dictionary should have at least have one key: “comparison”, a string to specifies the type of comparison. The other keys depend on the type of comparison, and typically there is at least a “value” key, the value to compare with. An example:

    {
      "output_parameters": {
        "energy": [
          {
            "comparison": "AlmostEqual",
            "value": -3699.26590536037
         }
        ],
        "energy_units": [
          {
            "comparison": "Equal",
            "value": "eV"
          }
        ]
      },
      "output_array": {
      }
    }
    

The list of valid comparisons is hardcoded inside the aiida.backends.tests.parsers module; if you need new comparison types, add them directly to the module.

Running tests

Finally, in order to run all tests contained in the folder aiida/backends/tests/parser_tests one can use the following verdi command:

verdi devel tests db.parsers

If no fail message appears it means that the test was successful.