Troubleshooting and tricks¶
When AiiDA tries to connect to the remote computer, it says
paramiko.SSHException: Server u'FULLHOSTNAME' not found in known_hosts
AiiDA uses the
paramikolibrary to establish SSH connections.
paramikois able to read the remote host keys from the
~/.ssh/known_hostsof the user under which the AiiDA daemon is running. You therefore have to make sure that the key of the remote host is stored in the file.
As a first check, login as the user under which the AiiDA daemon is running and run a:
FULLHOSTNAMEis the complete host name of the remote computer configured in AiiDA. If the key of the remote host is not in the
known_hostsfile, SSH will ask confirmation and then add it to the file.
If the above point is not sufficient, check the format of the remote host key. On some machines (we know that this issue happens at least on recent Ubuntu distributions) the default format is not RSA but ECDSA. However,
paramikois still not able to read keys written in this format.
To discover the format, run the following command:
ssh-keygen -F FULLHOSTNAME
that will print the remote host key. If the output contains the string
paramikowill not be able to use this key (see below for a solution). If instead
ssh-rsa, the key should be OK and paramiko will be able to use it.
In case your key is in ecdsa format, you have to first delete the key by using the command:
ssh-keygen -R FULLHOSTNAME
Then, in your
~/.ssh/configfile (create it if it does not exist) add the following lines:
Host * HostKeyAlgorithms ssh-rsa
(use the same indentation, and leave an empty line before and one after). This will set the RSA algorithm as the default one for all remote hosts. In case, you can set the
HostKeyAlgorithmsattribute only to the relevant computers (use
man ssh_configfor more information).
Then, run a:
command. SSH will ask confirmation and then add it to the file, but this time it should use the
ssh-rsaformat (it will say so in the prompt messsage). You can also double-check that the host key was correctly inserted using the
ssh-keygen -F FULLHOSTNAMEcommand as described above. Now, the error messsage should not appear anymore.
Increasing the debug level¶
By default, the logging level of AiiDA is minimal to avoid filling logfiles.
Only warnings and errors are logged (to the
~/.aiida/daemon/log/aiida_daemon.log file), while info and debug
messages are discarded.
If you are experiencing a problem, you can change the default minimum logging level of AiiDA messages (and celery messages – celery is the library that we use to manage the daemon process) using, on the command line, the two following commands:
verdi devel setproperty logging.celery_loglevel DEBUG verdi devel setproperty logging.aiida_loglevel DEBUG
After rebooting the daemon (
verdi daemon restart), the number of messages
logged will increase significantly and may help in understanding
the source of the problem.
In the command above, you can use a different level than
The list of the levels and their order is the same of the standard python
When the problem is solved, we suggest to bring back the default logging level, using the two commands:
verdi devel delproperty logging.celery_loglevel verdi devel delproperty logging.aiida_loglevel
to avoid to fill the logfiles.
Tips to ease the life of the hard drive (for large databases)¶
Those tips are useful when your database is very large, i.e. several hundreds of thousands of nodes and workflows or more. With such large databases the hard drive may be constantly working and the computer slowed down a lot. Below are some solutions to take care of the most typical reasons.
The backup of the repository takes an extensively long time if it is done through a standard rsync or backup software, since it contains as many folders as the number of nodes plus the number of workflows (and each folder can contain many files!). A solution is to use instead the incremental backup described in the repository backup section.
mlocate cron job¶
Under typical Linux distributions, there is a cron job (called
updatedb.mlocate) running every day to update a database of files and
folders – this is to be used by the
locate command. This might become
problematic since the repository contains many folders and
will be scanned everyday. The net effect is a hard drive almost constantly
To avoid this issue, edit as root the file
and put in
PRUNEPATHS the name of the repository folder.