Continuous integration

The Finesse project makes use of GitLab’s continuous integration feature to automatically build, test and deploy Finesse on various environments.

The primary way in which continuous integration is used in the Finesse project is to run tests on different target platforms to catch bugs there, especially those that don’t appear in a developer’s editable install. Other uses are to build and deploy the documentation, and to generate statistics regarding the project such as its test coverage. The pipeline run history is also useful to help determine which changes to Finesse or its dependencies introduced some bug or change in behaviour.

The continuous integration pipeline runs on GitLab automatically whenever code is pushed to the upstream Finesse repository. If you have configured your account for it, GitLab will send an email whenever a pipeline triggered by your push fails. You can inspect the console output from the failed job by clicking the link in the email, or browsing to the CI/CD section of the GitLab project.

Configuring the pipeline

The .gitlab-ci.yml file in the project root defines the continuous integration pipeline. See the syntax reference for more information on how this must be formatted. This file can also be edited using the GitLab web interface.

A linting tool is provided as part of the GitLab project to ensure there are no syntax errors or issues before committing changes to this file.

Pushing code without triggering the pipeline

GitLab supports skipping the pipeline via git’s push options flag. This can be used to help save needless computation for minor changes that don’t need the full build and test suite to run:

$ git push -o=ci.skip

The pipeline will then appear in GitLab as “skipped”.

Wheels built by the pipeline

The pipeline contains build jobs that produce so-called wheels which make installation of Finesse straightforward on Linux and Mac OSX by bundling compiled C extensions. The process follows that outlined in the section on packaging but some caveats apply in certain situations, listed below.

Mac OSX on AMD64

Due to Finesse’s use of modern PEP 517 build isolation, pip wheel must normally be able to install all required build dependencies via PyPI. This is unfortunately not possible on OSX 11 (Big Sur) and AMD64 (also referred to as “M1” or “Apple Silicon”) as of writing because wheels are not available for scipy and h5py. This triggers pip to try to build the wheels locally, and the OSX runner then encounters errors due to lack of required build dependencies for those packages. The solution until wheels are available for these dependencies is to switch off isolated builds and instead build within a conda distribution. That’s why the CI job for OSX on AMD64 adds the --no-build-isolation flag to pip wheel.