SQR-024: Enabling flake8 testing and Travis CI for existing DM repos

  • Jeff Carlin

Latest Revision: 2018-11-26

Note

This technote is not yet published.

This technote outlines the steps required to enable flake8 testing and Travis CI for existing DM repos for which this support has not been enabled.

1   Introduction

In this technote we describe how to enable flake8 testing and Travis CI for existing DM repos for which this support has not been enabled. The steps are outlined here in essentially a “recipe” form, and assume that one is working on the development version of the LSST Science Platform (lsp-dev). The steps should be the same on other environments, except that the paths to setup the LSST stack may differ.

2   Workflow

  1. Log onto lsp-dev, and open a terminal.

  1. Set up the LSST stack by typing the following in the terminal: source /opt/lsst/software/stack/loadLSST.bash ; setup lsst_distrib

  2. Make a place to put the repository (for example, travisifying/).

  3. Change directories to the folder you just created (cd travisifying/ in this example), and clone the repository that you wish to work on. For example, to clone this technote’s repo, type git clone https://github.com/lsst-sqre/sqr-024.git.

  4. Change directories into the top level of the cloned repository (e.g., cd sqr-024).

  5. Create a Jira ticket for this task (for our example, we’ll call it DM-99999). We will use that ticket for a new git branch.

  6. Create a branch in this repo: git checkout -b tickets/DM-99999 (replace 99999 with your ticket’s number).

  7. Set up the repository by typing setup -k -r .

  8. The flake8 testing requires a file called setup.cfg to be present in the top level of the repo. Put a template version of the setup.cfg in place by typing: curl -O https://raw.githubusercontent.com/lsst/templates/master/project_templates/stack_package/%7B%7Bcookiecutter.package_name%7D%7D/setup.cfg.

    Note: As of this note on 10/25/2018, if W504 isn’t included in the lists of tests to ignore in setup.cfg, then you should add it.

  9. Is there a tests/ directory in the repo? If so, are there any Python tests in that directory? If either of these is not true, then create them. An example test that simply imports a module and then confirms that it imported is located here:. You can copy this one and edit as needed. Note that some repos only contain configs. For these, you may want to simply create a dummy test that asserts some simple math.

  10. If there is no Sconscript file in tests/, then add one by doing the following: cd tests ; curl -O https://raw.githubusercontent.com/lsst/templates/master/project_templates/stack_package/%7B%7Bcookiecutter.package_name%7D%7D/tests/SConscript ; cd ..

  11. Make sure this SConscript file contains a line that reads: scripts.BasicSConscript.tests(pyList=[]) (scons needs to be told that it is allowed to discover files, and not just run flake8 on files in the tests/ directory).

  12. Run the flake8 linting tests by typing scons at the command line.

  13. Go through the list of linting errors (if any) and fix them, rerunning scons until there are no errors.

  14. Add a .travis.yaml file in the top level of the repository: curl -O https://raw.githubusercontent.com/lsst/templates/master/project_templates/stack_package/%7B%7Bcookiecutter.package_name%7D%7D/.travis.yml

  15. Commit your changes and push them to github.

  16. Go to https://travis-ci.org/lsst/<repo_name> and click the Activate button. (Note: you need to have proper permissions to do this, or ask somebody who has permissions to do it for you.)

  17. Make a commit and check (at the url given in step 18) that travis runs. (Note: sometimes it needs a nudge. Do a “git commit –amend” and “git push –force” to wake travis up...)

  18. Require travis to run by going to https://github.com/lsst/<repo_name> and navigate to Settings/Branches. Make sure both of these options are enabled: * Require branches to be up to date before merging * Continuous Integration Travis/CI (Note: you need to have proper (admin) permissions to do this, or ask somebody who does to do it for you.)

  19. Run the ticket branch through Jenkins: First, go to https://ci.lsst.codes/. On this page, search for “stack-os-matrix”, then click Run. Enter the ticket branch and repo names of your repo, then start the Jenkins run. You will be invited to the #dmj-stack-os-matrix Slack channel (if you are not already in it), where you will be notified when the job completes.

  20. If Jenkins successfully ran, then push your changes and merge to master. (If not, then fix whatever is causing Jenkins to fail before merging.)

  21. Change the Jira ticket status to “Done”.