The 'localization_optional' Optimization
Available in Cromwell version 33 and higher.
Scope
The 'localization_optional' optimization can be applied to a task's individual input declarations containing files, specifically File
and File?
values and any complex types containing them.
It allows you to save time and money by identifying files which do not need to be localized for the task to succeed.
Condition
The optimization signals to Cromwell that a task has been written in such a way that:
- The task will work if Cromwell does localize the specified file inputs
- For example if a file is localized for a local dockerized execution environment.
And:
- The task will also work if Cromwell does not localize the same file input
- For example the file remains in a cloud object store and the command is constructed using its URL rather than a local path.
Effect on File Localization
If the backend has been set up to respect localization_optional
, Cromwell will
choose not to localize the appropriate file input.
Effect on Call Caching:
None!
Files marked for optional localization are still treated in exactly the same way as other File
inputs for call caching.
Language Support
WDL 1.0 (or later)
In a WDL 1.0 task
, this optimization is specified by adding a localization_optional
field to
an input's entry in the task's parameter_meta
section. Here's an example:
task nio_task {
input {
File foo_file
File bar_file
}
parameter_meta {
foo_file: {
description: "a foo file",
localization_optional: true
}
bar_file: {
description: "a bar file"
}
}
command <<<
# This tool must work for **BOTH** local file paths **AND** object store URL values:
java -jar my_tool_1.jar ~{foo_file}
# Because the optimization is not applied to 'bar_file' in parameter_meta, this file **WILL** be localized:
java -jar my_tool_2.jar ~{bar_file}
>>>
}
Backend Support
This optimization is currently only applied to localization in the Pipelines API (GCE) backends.