Packaging Finesse

Finesse is currently packaged for pip and conda. Finesse requires some system dependencies in addition to those defined in the setuptools configuration. See Requirements for building, running and developing Finesse for more information on Finesse dependencies.

Conda

We expect that conda packages will be the main way that users get hold of Finesse. The standard Anaconda channel does not contain all the packages we need. Instead we use Conda-Forge, which incidentally is also what IGWN as well as LIGO laboratory also uses to distribute and maintain their packages.

Conda-Forge packaging can be relatively straight forward however with the complicated Finesse compilation it can throw various issues. All the Conda-Forge package management happens on GitHub. When making a new package you have to work with the staging repository first to get the package initially made and signed off. Once that is done a “feedstock” is made. The main Finesse feedstock is also on Github. You must be made a maintainer to work with it and push new fixes.

The workflow with the feedstock is

  • Make a new release tag and push it to the Finesse

  • Push a new source to PyPi

  • The feedstock bot will see this sometime in the next day or so and a new pull-request will be made

  • Hopefully, this pull-request will just work and all the tests will pass

  • If new packages or build changes have been made the feedstock recipe/meta.yaml might need updating

  • If there are new package requirements this must be reflected in the meta.yaml file

It is very important you read the Conda-Forge packaging instructions first before doing much, as there are various ways things should be done. Overall the process isn’t too painful.

If a new PyPi package dependency is added to Finesse we must also ensure that the same package is available in Conda-Forge. It will not use PyPi as a dependency. For simple python packages you can use Grayskull to quickly make a conda package and get it approved.

Wheels

Platform-specific wheels are built for most Linux platforms (so-called manylinux wheels) and macOS. This procedure is automated as part of the continuous integration pipeline, but it is quite simple. The Red Hat based Docker image manylinux2014 provided by PyPA is used. The only upstream dependency is currently suitesparse-devel which provides the KLU C headers used by Finesse’s simulation module.

Building for ‘manylinux’

The procedure for building a bdist_wheel for the manylinux target is:

$ yum install -y suitesparse-devel
$ pip wheel . --no-deps -w wheelhouse
$ auditwheel repair wheelhouse/*.whl

These commands should be run from the source code root on the Docker image. The /path/to/pip parts should be replaced by the path to pip corresponding to the Python version being targeted.

Building for Mac OSX

The procedure to make a wheel for OSX is similar. The wheel must of course be built using OSX, and SuiteSparse must be installed by some means other than yum (e.g. with brew install suite-sparse). The tool for delocalising the produced wheel is also not auditwheel, but rather delocate-wheel provided by delocate:

$ pip wheel . --no-deps -w wheelhouse
$ delocate-wheel wheelhouse/*.whl