SPEC 5 — CI Best Practices

Authors:
Tim Head <betatim@gmail.com>, Marta Iborra <martaiborra24@gmail.com>, Ryan May <rmay@ucar.edu>, Jarrod Millman <millman@berkeley.edu>, Brigitta Sipőcz <brigitta.sipocz@gmail.com>
Discussion:
https://discuss.scientific-python.org/t/spec-5-ci-best-practices/507
History:
https://github.com/scientific-python/specs/commits/main/spec-0005
Endorsed by:
Attention

This is a draft document.

Description#

Core Project Endorsement#

Ecosystem Adoption#

Implementation#

Build nightly wheels#

By building and uploading wheel for your project you make it easier for projects that depend on you to test against your latest changes. They can then report problems they find.

There are a few steps to implementing this for your project:

  1. Get access to the area that nightly wheels are uploaded to
  2. Setup a CI step that builds wheels for your project
  3. Setup a CI step that uploads wheels to https://anaconda.org/scientific-python-nightly-wheels/

For step (1) contact XXX. Deposit the token you get as a secret on your repository named UPLOAD_TOKEN.

The work for step (2) depends on your project. You are probably already doing this for your releases. The new thing to add is that building wheels is run on a schedule every night or once a week.

For step (3) there is a GitHub Action that you can use. You can find the action at https://github.com/scientific-python/upload-nightly-action. To use it in your “build wheels workflow” add the following lines as an additional step:

- name: Upload wheel
  uses: scientific-python/upload-nightly-action@main
  with:
    artifacts_path: dist/
    anaconda_nightly_upload_token: ${{ secrets.UPLOAD_TOKEN }}

Notes#

  • Recommend that CI running against PRs be expected to pass–if it fails, it should be important

  • Schedule regular runs of CI against nightlies/pre-releases

    • Good for other checks that don’t need to run on every PR
    • Could use for pins
    • Can use labels to trigger additional runs of “exotic” jobs
    • Can open an issue on failure rather than hiding in a UI somewhere–helps give notifications
    • Generate a badge for the workflow, that can be collected into a dashboard
  • GitHub actions security concerns

    • Need to trust action creators
    • Pin to hash vs. version/tag?
On this page