Releasing finesse

Finesse is released on two channels, pypi and conda-forge. Since the conda-forge release is based on the pypi release, we will describe the PyPi release first.

PyPi

The PyPi release is configured via the gitlab pipelines /.gitlab-ci.yml, in the deploy stage. You can trigger a release at any time by creating a tag on any branch, but preferably we tag a commit on develop that has already passed the tests and following pep 440. Since Finesse 3 is currently in alpha stage, the version follows 3.0aN, where N is an increasing integer.

The tag should be accompanied by a short description of user-facing changes since the last release.

The pypy job currently pushes a source distribution (a compressed tar.gz archive) and a binary distributions (wheels) for manylinux and MacOS arm64.

The source distribution is important because it is the basis for the conda-forge release and a fallback for platforms that we do not build wheels for. Installing finesse from PyPi on these platforms will trigger a build of the cython modules, requiring Cython and Suitesparse to be available on the system. This is similar to a Installing from source.

The PyPi release is usually fairly straightforward, since binary wheels are being created in every pipeline, catching any issues early during development. As of 3.0a23 Windows wheels are not being published.

Conda-forge

The conda-forge release is in principle automated, but can be more difficult to debug when issues arise. Much of the process is also documented here.

The conda-forge feedstock repository is where the release process is configured. To able to release finesse, your github account needs to be added to the maintainer list. New maintainers can be added by opening an issue in the feedstock with the title @conda-forge-admin, please add user @username. This creates a pull request which can be merged by existing maintainers.

Almost all behaviour is configured in the meta.yaml which is documented here.

Automatic releases

It runs a bot which watches the finesse PyPi repository and automatically creates a pull request when a new PyPi version is released. See an example here. The creation of the pull request triggers a series of builds on the conda-forge platform, building binary conda packages for Finesse on a variety of platforms and python versions. See an example here. This process is called ‘rendering’.

When issues inevitably arise, you will need to fix the configuration by committing to the branch the bot created, and adding a comment in the PR with the text @conda-forge-admin please rerender. When the builds for all platforms are successful the pull request should be merged to finalize the release.

Manual releases

If you do not want to wait for the bot to create the pull request, you can do so yourself. You will need to fork the feedstock and make the following changes in the meta.yaml file:

  • Reset the build->number to 0 (when not patching an existing release)

  • Update the source->sha256 to the hash of the corresponding source release on PyPi

  • Check if the requirements defined in the meta.yaml still match those defined in the pyproject.toml

  • See the current list of source->patches are functioning and whether any new patches are required.

Afterwards, you create a pull request of a branch in your fork into the original feedstock repository. You can then follow the flow described above.

Patches

Releasing for conda-forge unfortunately requires some patching of the finesse repository, mostly to the different files specifying versions. They are listed under source->patches and defined in the various .patch files in the recipe folder.

Patches failing usually results in a hunk fail in the logs of the rendering.

Updating requirements

The main source of problems is the different specifications for the package requirements. The meta.yaml file can conflict with the finesse pyproject.toml. For automatic releases the bot will perform a ‘Dependency Analysis’ that is meant to flag mismatches between the two specifications. See here for an example.

The requirements are spread between build, host and run sections. See here for a brief explanation.

Another possible source of failure is our dependencies themselves having misconfigured their conda feedstock, leading to version conflicts.

Chat channel

When getting stuck or confused, you can always try the asking questions in the chat channel