Important Concepts¶

Python Packages¶

Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example the module name A.B designates a submodule named B in a package named A. [...]

The way that this behaviour is achieved in practice is by creating a folder structure like the following:

package/             Top-level package (import package)
   __init__.py       Marks the folder as a package and contains initialization
   moduleA.py        module (from package import moduleA)
   subpackage/       subpackage (from package import subpackage)
      __init__.py
      moduleB.py     module (from package.subpackage import moduleB)
      ...

The page linked on top of this section contains a more detailed examples as well as more information on how to write and use packages.

Python Distributions¶

The python community widely uses the term ‘package’ for both a package in the sense of a collection of subpackages and modules, as well as all the additional files necessary for building and installing a package. More about distributing packages here.

In this documentation we will refer to the latter as a distribution when we wish to distinguish between the two concepts.

Example:

distribution/        Distribution enclosing package and additional files
   package/          The package that the distribution installs
      __init__.py
      ...
   MANIFEST.in       (optional) lists non-python files to be installed
   README.rst        (optional) description to be used by github etc and PyPI
   setup.py          (required) contains requirements, metainformation, etc

Incidentally a distribution can contain and install more than one package at a time.

The most user-friendly way to distribute a package is to create such a distribution and uploading it to PyPI. Users then can simply install the package(s) by running pip <distribution-name>.

Installing a Package¶

What happens when pip is used to install a package is explained in detail in the python packaging user guide. However it is worth summarizing some points here.

the dependencies on other python packages as specified in setup.py are automatically resolved and installed;
a folder <distribution-name>.egg-info/ is created, containing metadata about the distribution;
if the -e option is given, a symbolic link is put into the python package search path, pointing to the distribution top level directory. This is where the .egg-info folder gets created. Changes to the source code will be picked up by python without reinstalling, however changes to the distribution metadata will not.

Entry Points¶

The setuptools package to which pip is a frontend has a feature called entry points. When a distribution which registers entry points is installed, the entry point specifications are written to a file inside the distribution’s .egg-info folder. setuptools provides a package pkg_resources which can find these entry points by distribution, group and/or name and load the data structure to which it points. This is the way AiiDA finds and loads classes provided by plugins.

There is a list of entry point groups defined by AiiDA on the page AiiDA Entry Points.