CI testing in Travis and Jenkins
Any suite of tests running under 2.5 hours, using 2 cpus, and 6gb of memory executes on Travis. Travis tests every pull request by a trusted contributor. Larger test suites run on Jenkins instances. Examples include the DSP Workbench CI testing (swatomation), a nightly test of
develop on (a snapshot of) the $5 Genome WDL, and the Cromwell-Perf tests that call-cache thousands of jobs.
CD to CaaS DEV
One instance utilizing continuous delivery is the
develop branch to CaaS
DEV. While manual testing could occur on
DEV, users primarily test on
The Cromwell developers stage Terra and CaaS
PROD deployments. All other deployments are performed by respective groups, who decide when and how to redeploy. Some upgrade Cromwell, while others deploy completely new instances, including a new database schema.
Only Travis Pull Requests generate maximal code coverage reports. All other CI either doesn't report coverage, or under-reports due to skipped tests.
In collaboration with the DSP Information Security team, scans include but are not limited to:
- Committed Git secrets
- Vulnerable Java dependencies
- Penetration testing