Google Pipelines API Workflow Options

These workflow options provide Google-specific information for workflows running tasks on the Google PAPI backend.

Keys Possible Values Description
jes_gcs_root string Where outputs of the workflow will be written. Expects this to be a GCS URL (e.g. gs://my-bucket/workflows). If this is not set, this defaults to the value within backend.jes.config.root in the Configuration.
google_compute_service_account string Alternate service account to use on the compute instance (e.g. my-new-svcacct@my-google-project.iam.gserviceaccount.com). If this is not set, this defaults to the value within backend.jes.config.genomics.compute-service-account in the Configuration if specified or default otherwise.
google_project string Google project used to execute this workflow.
auth_bucket string A GCS URL that only Cromwell can write to. The Cromwell account is determined by the google.authScheme (and the corresponding google.userAuth and google.serviceAuth). Defaults to the the value in jes_gcs_root.
monitoring_script string Specifies a GCS URL to a script that will be invoked prior to the user command being run. For example, if the value for monitoring_script is "gs://bucket/script.sh", it will be invoked as ./script.sh > monitoring.log &. The value monitoring.log file will be automatically de-localized.
monitoring_image string Specifies a Docker image to monitor the task. This image will run concurrently with the task container, and provides an alternative mechanism to monitoring_script (the latter runs inside the task container). For example, one can use quay.io/broadinstitute/cromwell-monitor, which reports cpu/memory/disk utilization metrics to Stackdriver.
monitoring_image_script string Specifies a GCS URL to a script that will be invoked on the container running the monitoring_image. This script will be invoked instead of the ENTRYPOINT defined in the monitoring_image. Unlike the monitoring_script no files are automatically de-localized.
google_labels object An object containing only string values. Represent custom labels to send with PAPI job requests. Per the PAPI specification, each key and value must conform to the regex [a-z]([-a-z0-9]*[a-z0-9])?.
enable_ssh_access boolean If set to true, will enable SSH access to the Google Genomics worker machines. Please note that this is a community contribution and is not officially supported by the Cromwell development team.
delete_intermediate_output_files boolean Experimental: Any File variables referenced in call output sections that are not found in the workflow output section will be considered an intermediate File. When the workflow finishes and this option is set to true, all intermediate File objects will be deleted from GCS. Cromwell must be run with the configuration value system.delete-workflow-files set to true. The default for both values is false. NOTE: The behavior of this option on other backends is unspecified.
enable_fuse boolean Specifies if workflow tasks should be submitted to Google Pipelines with an additional ENABLE_FUSE flag. It causes container to be executed with CAP_SYS_ADMIN. Use it only for trusted containers. Please note that this is a community contribution and is not officially supported by the Cromwell development team.

Example

{
  "jes_gcs_root": "gs://my-bucket/workflows",
  "google_project": "my_google_project",
  "google_compute_service_account": "my-new-svcacct@my-google-project.iam.gserviceaccount.com",
  "auth_bucket": "gs://my-auth-bucket/private",
  "monitoring_script": "gs://bucket/monitoring_script.sh",
  "monitoring_image": "quay.io/broadinstitute/cromwell-monitor",
  "monitoring_image_script": "gs://bucket/monitoring_image_script.sh",
  "enable_ssh_access": false,
  "google_labels": {
    "custom-label": "custom-value"
  },
  "delete_intermediate_output_files": false,
  "enable_fuse": false
}