"
ResourceRequirement:
coresMax: 2
ramMax: 4000 # specified in mebibytes
```
------
# Task accelerators in a HealthOmics workflow definition
In the workflow definition, you can optionally specify the GPU accelerator-spec for a task. HealthOmics supports the following accelerator-spec values, along with the supported instance types:
| Accelerator spec | Healthomics instance types |
| --- | --- |
| nvidia-tesla-t4 | G4 |
| nvidia-tesla-t4-a10g | G4 and G5 |
| nvidia-tesla-a10g | G5 |
| nvidia-t4-a10g-l4 | G4, G5, and G6 |
| nvidia-l4-a10g | G5 and G6 |
| nvidia-l4 | G6 |
| nvidia-l40s | G6e |
If you specify an accelerator type that supports multiple instance types, HealthOmics selects the instance type based on available capacity. If both instance types are available, HealthOmics gives preference to the lower cost instance. The exception is for the nvidia-t4-a10g-l4 task accelerator which gives preference to the latest generation instance available.
For details about the instance types, see [Accelerated-computing instances](memory-and-compute-tasks.md#workflow-task-accelerated-computing-instances).
In the following example, the workflow definition specifies `nvidia-l4` as the accelerator:
------
#### [ WDL ]
```
task my_task {
runtime {
...
acceleratorCount: 1
acceleratorType: "nvidia-l4"
}
...
}
```
------
#### [ NextFlow ]
```
process my_task {
...
accelerator 1, type: "nvidia-l4"
...
}
```
------
#### [ CWL ]
```
cwlVersion: v1.2
class: CommandLineTool
requirements:
...
cwltool:CUDARequirement:
cudaDeviceCountMin: 1
cudaComputeCapability: "nvidia-l4"
cudaVersionMin: "1.0"
```
------
# WDL workflow definition specifics
The following topics provide details about types and directives available for WDL workflow definitions in HealthOmics.
**Topics**
+ [Implicit type conversion in WDL lenient](#workflow-wdl-type-conversion)
+ [Namespace definition in input.json](#workflow-wdl-namespace-defn)
+ [Primitive types in WDL](#workflow-wdl-primitive-types)
+ [Complex types in WDL](#workflow-wdl-complex-types)
+ [Directives in WDL](#workflow-wdl-directives)
+ [Task metadata in WDL](#workflow-wdl-task-metadata)
+ [WDL workflow definition example](#wdl-example)
## Implicit type conversion in WDL lenient
HealthOmics supports implicit type conversion in the input.json file and the workflow definition. To use implicit type casting, specify the workflow engine as WDL lenient when you create the workflow. WDL lenient is designed to handle workflows migrated from Cromwell. It supports customer Cromwell directives and some non-conformant logic.
WDL lenient supports type conversion for the following items in the list of WDL’s [limited exceptions](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-limited-exceptions):
+ Float to Int, where the coercion results in no loss of precision (such as 1.0 maps to 1).
+ String to Int/Float, where the coercion results in no loss of precision.
+ Map[W, X] to Array[Pair[Y, Z]], in the case where W is coercible to Y and X is coercible to Z.
+ Array[Pair[W, X]] to Map[Y, Z], in the case where W is coercible to Y and X is coercible to Z (such as 1.0 maps to 1).
To use implicit type casting, specify the workflow engine as WDL\$1LENIENT when you create the workflow or workflow version.
In the console, the workflow engine parameter is named **Language**. In the API, the workflow engine parameter is named **engine**. For more information, see [Create a private workflow](create-private-workflow.md) or [Create a workflow version](workflows-version-create.md).
## Namespace definition in input.json
HealthOmics supports fully qualified variables in input.json. For example, if you declare two input variables named number1 and number2 in workflow **SumWorkflow**:
```
workflow SumWorkflow {
input {
Int number1
Int number2
}
}
```
You can use them as fully qualified variables in input.json:
```
{
"SumWorkflow.number1": 15,
"SumWorkflow.number2": 27
}
```
## Primitive types in WDL
The following table shows how inputs in WDL map to the matching primitive types. HealthOmics provides limited support for type coercion, so we recommend that you set explicit types.
**Primitive types**
| WDL type | JSON type | Example WDL | Example JSON key and value | Notes |
| --- | --- | --- | --- | --- |
| Boolean | boolean | Boolean b | "b": true | The value must be lower case and unquoted. |
| Int | integer | Int i | "i": 7 | Must be unquoted. |
| Float | number | Float f | "f": 42.2 | Must be unquoted. |
| String | string | String s | "s": "characters" | JSON strings that are a URI must be mapped to a WDL file to be imported. |
| File | string | File f | "f": "s3://amzn-s3-demo-bucket1/path/to/file" | Amazon S3 and HealthOmics storage URIs are imported as long as the IAM role provided for the workflow has read access to these objects. No other URI schemes are supported (such as file://, https://, and ftp://). The URI must specify an object. It cannot be a directory meaning it can not end with a /. |
| Directory | string | Directory d | "d": "s3://bucket/path/" | The Directory type isn't included in WDL 1.0 or 1.1, so you will need to add version development to the header of the WDL file. The URI must be a Amazon S3 URI and with a prefix that ends with a '/'. All contents of the directory will be recursively copied to the workflow as a single download. The Directory should only contain files related to the workflow. |
## Complex types in WDL
The following table show how inputs in WDL map to the matching complex JSON types. Complex types in WDL are data structures comprised of primitive types. Data structures such as lists will be converted to arrays.
**Complex types**
| WDL type | JSON type | Example WDL | Example JSON key and value | Notes |
| --- | --- | --- | --- | --- |
| Array | array | Array[Int] nums | “nums": [1, 2, 3] | The members of the array must follow the format of the WDL array type. |
| Pair | object | Pair[String, Int] str\$1to\$1i | “str\$1to\$1i": \$1"left": "0", "right": 1\$1 | Each value of the pair must use the JSON format of its matching WDL type. |
| Map | object | Map[Int, String] int\$1to\$1string | "int\$1to\$1string": \$1 2: "hello", 1: "goodbye" \$1 | Each entry in the map must use the JSON format of its matching WDL type. |
| Struct | object | struct SampleBamAndIndex {
String sample_name
File bam
File bam_index
} SampleBamAndIndex b_and_i | "b_and_i": {
"sample_name": "NA12878",
"bam": "s3://amzn-s3-demo-bucket1/NA12878.bam",
"bam_index": "s3://amzn-s3-demo-bucket1/NA12878.bam.bai"
} | The names of the struct members must exactly match the names of the JSON object keys. Each value must use the JSON format of the matching WDL type. |
| Object | N/A | N/A | N/A | The WDL Object type is outdated and should be replaced by Struct in all cases. |
## Directives in WDL
HealthOmics supports the following directives in all WDL versions that HealthOmics supports.
### Configure GPU resources
HealthOmics supports runtime attributes **acceleratorType** and **acceleratorCount** with all supported [GPU instances](https://docs.aws.amazon.com/omics/latest/dev/task-accelerators.html). HealthOmics also supports aliases named **gpuType** and **gpuCount**, which have the same functionality as their accelerator counterparts. If the WDL definition contains both directives, HealthOmics uses the accelerator values.
The following example shows how to use these directives:
```
runtime {
gpuCount: 2
gpuType: "nvidia-tesla-t4"
}
```
### Configure task retry for service errors
HealthOmics supports up to two retries for a task that failed because of service errors (5XX HTTP status codes). You can configure the maximum number of retries (1 or 2) and you can opt out of retries for service errors. By default, HealthOmics attempts a maximum of two retries.
The following example sets `preemptible` to opt out of retries for service errors:
```
{
preemptible: 0
}
```
For more information about task retries in HealthOmics, see [Task Retries](monitoring-runs.md#run-status-task-retries).
### Configure task retry for out of memory
HealthOmics supports retries for a task that failed because it ran out of memory (container exit code 137, 4XX HTTP status code). HealthOmics doubles the amount of memory for each retry attempt.
By default, HealthOmics doesn't retry for this type of failure. Use the `maxRetries` directive to specify the maximum number of retries.
The following example sets `maxRetries` to 3, so that HealthOmics attempts a maximum of four attempts to complete the task (the initial attempt plus three retries):
```
runtime {
maxRetries: 3
}
```
**Note**
Task retry for out of memory requires GNU findutils 4.2.3\$1. The default HealthOmics image container includes this package. If you specify a custom image in your WDL definition, make sure that the image includes GNU findutils 4.2.3\$1.
### Configure return codes
The **returnCodes** attribute provides a mechanism to specify a return code, or a set of return codes, that indicates a successful execution of a task. The WDL engine honors the return codes that you specify in the **runtime** section of the WDL definition, and sets the tasks status accordingly.
```
runtime {
returnCodes: 1
}
```
HealthOmics also supports an alias named **continueOnReturnCode**, which has the same capabilities as **returnCodes**. If you specify both attributes, HealthOmics uses the **returnCodes** value.
## Task metadata in WDL
HealthOmics supports the following metadata options for WDL tasks.
### Disable task-level caching with the volatile attribute
The **volatile** attribute allows you to disable call caching for specific tasks in your WDL workflow. When a task is marked as volatile, it will always execute and never use cached results, even when caching is enabled for the run.
Add the **volatile** attribute to the **meta** section of your task definition:
```
task my_volatile_task {
meta {
volatile: true
}
input {
String input_file
}
command {
echo "Processing ${input_file}" > output.txt
}
output {
File result = "output.txt"
}
}
```
## WDL workflow definition example
The following examples show private workflow definitions for converting from `CRAM` to `BAM` in WDL. The `CRAM` to `BAM` workflow defines two tasks and uses tools from the `genomes-in-the-cloud` container, which is shown in the example and is publicly available.
The following example shows how to include the Amazon ECR container as a parameter. This allows HealthOmics to verify the access permissions to your container before it starts the run the run.
```
{
...
"gotc_docker":".dkr.ecr..amazonaws.com/genomes-in-the-cloud:2.4.7-1603303710"
}
```
The following example shows how to specify which files to use in your run, when the files are in an Amazon S3 bucket.
```
{
"input_cram": "s3://amzn-s3-demo-bucket1/inputs/NA12878.cram",
"ref_dict": "s3://amzn-s3-demo-bucket1/inputs/Homo_sapiens_assembly38.dict",
"ref_fasta": "s3://amzn-s3-demo-bucket1/inputs/Homo_sapiens_assembly38.fasta",
"ref_fasta_index": "s3://amzn-s3-demo-bucket1/inputs/Homo_sapiens_assembly38.fasta.fai",
"sample_name": "NA12878"
}
```
If you want to specify files from a sequence store, indicate that as shown in the following example, using the URI for the sequence store.
```
{
"input_cram": "omics://429915189008.storage.us-west-2.amazonaws.com/111122223333/readSet/4500843795/source1",
"ref_dict": "s3://amzn-s3-demo-bucket1/inputs/Homo_sapiens_assembly38.dict",
"ref_fasta": "s3://amzn-s3-demo-bucket1/inputs/Homo_sapiens_assembly38.fasta",
"ref_fasta_index": "s3://amzn-s3-demo-bucket1/inputs/Homo_sapiens_assembly38.fasta.fai",
"sample_name": "NA12878"
}
```
You can then define your workflow in WDL as shown in the following example.
```
version 1.0
workflow CramToBamFlow {
input {
File ref_fasta
File ref_fasta_index
File ref_dict
File input_cram
String sample_name
String gotc_docker = ".dkr.ecr.us-west-2.amazonaws.com/genomes-in-the-
cloud:latest"
}
#Converts CRAM to SAM to BAM and makes BAI.
call CramToBamTask{
input:
ref_fasta = ref_fasta,
ref_fasta_index = ref_fasta_index,
ref_dict = ref_dict,
input_cram = input_cram,
sample_name = sample_name,
docker_image = gotc_docker,
}
#Validates Bam.
call ValidateSamFile{
input:
input_bam = CramToBamTask.outputBam,
docker_image = gotc_docker,
}
#Outputs Bam, Bai, and validation report to the FireCloud data model.
output {
File outputBam = CramToBamTask.outputBam
File outputBai = CramToBamTask.outputBai
File validation_report = ValidateSamFile.report
}
}
#Task definitions.
task CramToBamTask {
input {
# Command parameters
File ref_fasta
File ref_fasta_index
File ref_dict
File input_cram
String sample_name
# Runtime parameters
String docker_image
}
#Calls samtools view to do the conversion.
command {
set -eo pipefail
samtools view -h -T ~{ref_fasta} ~{input_cram} |
samtools view -b -o ~{sample_name}.bam -
samtools index -b ~{sample_name}.bam
mv ~{sample_name}.bam.bai ~{sample_name}.bai
}
#Runtime attributes:
runtime {
docker: docker_image
}
#Outputs a BAM and BAI with the same sample name
output {
File outputBam = "~{sample_name}.bam"
File outputBai = "~{sample_name}.bai"
}
}
#Validates BAM output to ensure it wasn't corrupted during the file conversion.
task ValidateSamFile {
input {
File input_bam
Int machine_mem_size = 4
String docker_image
}
String output_name = basename(input_bam, ".bam") + ".validation_report"
Int command_mem_size = machine_mem_size - 1
command {
java -Xmx~{command_mem_size}G -jar /usr/gitc/picard.jar \
ValidateSamFile \
INPUT=~{input_bam} \
OUTPUT=~{output_name} \
MODE=SUMMARY \
IS_BISULFITE_SEQUENCED=false
}
runtime {
docker: docker_image
}
#A text file is generated that lists errors or warnings that apply.
output {
File report = "~{output_name}"
}
}
```
# Nextflow workflow definition specifics
HealthOmics suppports Nextflow DSL1 and DSL2. For details, see [Nextflow version support](workflows-lang-versions.md#workflows-lang-versions-nextflow).
Nextflow DSL2 is based on the Groovy programming language, so parameters are dynamic and type coercion is possible using the same rules as Groovy. Parameters and values supplied by the input JSON are available in the parameters (`params`) map of the workflow.
**Topics**
+ [Use nf-schema and nf-validation plugins](#schema-and-validation-plugins-nextflow)
+ [Specify storage URIs](#storage-uris-nextflow)
+ [Nextflow directives](#workflow-nexflow-directives)
+ [Export workflow-level content](#exporting-workflow-content-nextflow)
+ [Export task content](#exporting-task-content-nextflow)
## Use nf-schema and nf-validation plugins
**Note**
Summary of HealthOmics support for plugins:
v22.04 – no support for plugins
v23.10 – supports `nf-schema` and `nf-validation`
v24.10 – supports `nf-schema`
v25.10 – supports `nf-schema`, `nf-core-utils`, `nf-fgbio`, and `nf-prov`
HealthOmics provides the following support for Nextflow plugins:
+ For Nextflow v23.10, HealthOmics pre-installs the nf-validation@1.1.1 plugin.
+ For Nextflow v23.10 and later, HealthOmics pre-installs the nf-schema@2.3.0 plugin.
+ You cannot retrieve additional plugins during a workflow run. HealthOmics ignores any other plugin versions that you specify in the `nextflow.config` file.
+ For Nextflow v24 and higher, `nf-schema` is the new version of the deprecated `nf-validation` plugin. For more information, see [ nf-schema](https://github.com/nextflow-io/nf-schema) in the Nextflow GitHub repository.
## Specify storage URIs
When an Amazon S3 or HealthOmics URI is used to construct a Nextflow file or path object, it makes the matching object available to the workflow, as long as read access is granted. The use of prefixes or directories is allowed for Amazon S3 URIs. For examples, see [Amazon S3 input parameter formats](workflows-run-inputs.md#s3-run-input-formats).
HealthOmics partially supports the use of glob patterns in Amazon S3 URIs or HealthOmics Storage URIs. Use Glob patterns in the workflow definition for the creation of `path` or `file` channels. For the expected behavior and exact cases, see [Nextflow Handling of Glob pattern in Amazon S3 inputs](workflows-run-inputs.md#wd-nextflow-s3-formats).
## Nextflow directives
You configure Nextflow directives in the Nextflow config file or workflow definition. The following list shows the order of precedence that HealthOmics uses to apply configuration settings, from lowest to highest priority:
1. Global configuration in the config file.
1. Task section of the workflow definition.
1. Task-specific selectors in the config file.
**Topics**
+ [Task retry strategy using `errorStrategy`](#workflow-nextflow-errorStrategy)
+ [Task retry attempts using `maxRetries`](#workflow-nexflow-task-retry)
+ [Opt out of task retry using `omicsRetryOn5xx`](#workflow-nextflow-retry-5xx)
+ [Task duration using the `time` directive](#time-directive-nextflow)
### Task retry strategy using `errorStrategy`
Use the `errorStrategy` directive to define the strategy for task errors. By default, when a task returns with an error indication (a non-zero exit status), the task stops and HealthOmics terminates the entire run. If you set `errorStrategy` to `retry`, HealthOmics attempts one retry of the failed task. To increase the number of retries, see [Task retry attempts using `maxRetries`](#workflow-nexflow-task-retry).
```
process {
label 'my_label'
errorStrategy 'retry'
script:
"""
your-command-here
"""
}
```
For information about how HealthOmics handles task retries during a run, see [Task Retries](monitoring-runs.md#run-status-task-retries).
### Task retry attempts using `maxRetries`
By default, HealthOmics doesn't attempt any retries of a failed task, or attempts one retry if you configure `errorStrategy`. To increase the maximum number of retries, set `errorStrategy` to `retry` and configure the maximum number of retries using the `maxRetries` directive.
The following example sets the maximum number of retries to 3 in the global configuration.
```
process {
errorStrategy = 'retry'
maxRetries = 3
}
```
The following example shows how to set `maxRetries` in the task section of the workflow definition.
```
process myTask {
label 'my_label'
errorStrategy 'retry'
maxRetries 3
script:
"""
your-command-here
"""
}
```
The following example shows how to specify task-specific configuration in the Nextflow config file, based on the name or label selectors.
```
process {
withLabel: 'my_label' {
errorStrategy = 'retry'
maxRetries = 3
}
withName: 'myTask' {
errorStrategy = 'retry'
maxRetries = 3
}
}
```
### Opt out of task retry using `omicsRetryOn5xx`
For Nextflow v23 and later, HealthOmics supports task retries if the task failed because of service errors (5XX HTTP status codes). By default, HealthOmics attempts up to two retries of a failed task.
You can configure `omicsRetryOn5xx` to opt out of task retry for service errors. For more information about task retry in HealthOmics, see [Task Retries](monitoring-runs.md#run-status-task-retries).
The following example configures `omicsRetryOn5xx` in the global configuration to opt out of task retry.
```
process {
omicsRetryOn5xx = false
}
```
The following example shows how to configure `omicsRetryOn5xx` in the task section of the workflow definition.
```
process myTask {
label 'my_label'
omicsRetryOn5xx = false
script:
"""
your-command-here
"""
}
```
The following example shows how to set `omicsRetryOn5xx` as task-specific configuration in the Nextflow config file, based on the name or label selectors.
```
process {
withLabel: 'my_label' {
omicsRetryOn5xx = false
}
withName: 'myTask' {
omicsRetryOn5xx = false
}
}
```
### Task duration using the `time` directive
HealthOmics provides an adjustable quota (see [HealthOmics service quotas](service-quotas.md)) to specify the maximum duration for a run. For Nextflow v23 and later workflows, you can also specify maximum task durations using the Nextflow `time` directive.
During new workflow development, setting maximum task duration helps you catch runaway tasks and long-running tasks.
For more information about the Nextflow time directive, see [time directive](https://www.nextflow.io/docs/latest/reference/process.html#process-time) in the Nextflow reference.
HealthOmics provides the following support for the Nextflow time directive:
1. HealthOmics supports 1 minute granularity for the time directive. You can specify a value between 60 seconds and the maximum run duration value.
1. If you enter a value less than 60, HealthOmics rounds it up to 60 seconds. For values above 60, HealthOmics rounds down to the nearest minute.
1. If the workflow supports retries for a task, HealthOmics retries the task if it times out.
1. If a task times out (or the last retry times out), HealthOmics cancels the task. This operation can have a duration of one to two minutes.
1. On task timeout, HealthOmics sets the run and task status to failed, and it cancels the other tasks in the run (for tasks in Starting, Pending, or Running status). HealthOmics exports the outputs from tasks that it completed before the timeout to your designated S3 output location.
1. Time that a task spends in pending status does not count toward the task duration.
1. If the run is part of a run group and the run group times out sooner than the task timer, the run and task transition to failed status.
Specify the timeout duration using one or more of the following units: `ms`, `s`, `m`,`h`, or `d`.
The following example shows how to specify global configuration in the Nextflow config file. It sets a global timeout of 1 hour and 30 minutes.
```
process {
time = '1h30m'
}
```
The following example shows how to specify a time directive in the task section of the workflow definition. This example sets a timeout of 3 days, 5 hours, and 4 minutes. This value takes precedence over the global value in the config file, but doesn't take precedence over a task-specific time directive for `my_label` in the config file.
```
process myTask {
label 'my_label'
time '3d5h4m'
script:
"""
your-command-here
"""
}
```
The following example shows how to specify task-specific time directives in the Nextflow config file, based on the name or label selectors. This example sets a global task timeout value of 30 minutes. It sets a value of 2 hours for task `myTask` and sets a value of 3 hours for tasks with label `my_label`. For tasks that match the selector, these values take precedence over the global value and the value in the workflow definition.
```
process {
time = '30m'
withLabel: 'my_label' {
time = '3h'
}
withName: 'myTask' {
time = '2h'
}
}
```
## Export workflow-level content
For Nextflow v25.10, you can export files produced outside of individual tasks, such as provenance reports or pipeline DAGs. To export these files, write them to `/mnt/workflow/output/`. HealthOmics exports files placed in this directory to the `output/` prefix in your run's Amazon S3 output location.
The following example shows how to configure the `nf-prov` plugin to write a provenance report to `/mnt/workflow/output/`.
```
prov {
formats {
bco {
file = "/mnt/workflow/output/pipeline_info/manifest.bco.json"
}
}
}
```
You can also pass this path as a parameter in your run's input JSON. This approach is common with nf-core workflows that use `params.outdir`.
```
{
"outdir": "/mnt/workflow/output/"
}
```
## Export task content
For workflows written in Nextflow, define a **publishDir** directive to export task content to your output Amazon S3 bucket. As shown in the following example, set the **publishDir** value to `/mnt/workflow/pubdir`. To export files to Amazon S3, the files must be in this directory.
```
nextflow.enable.dsl=2
workflow {
CramToBamTask(params.ref_fasta, params.ref_fasta_index, params.ref_dict, params.input_cram, params.sample_name)
ValidateSamFile(CramToBamTask.out.outputBam)
}
process CramToBamTask {
container ".dkr.ecr.us-west-2.amazonaws.com/genomes-in-the-cloud"
publishDir "/mnt/workflow/pubdir"
input:
path ref_fasta
path ref_fasta_index
path ref_dict
path input_cram
val sample_name
output:
path "${sample_name}.bam", emit: outputBam
path "${sample_name}.bai", emit: outputBai
script:
"""
set -eo pipefail
samtools view -h -T $ref_fasta $input_cram |
samtools view -b -o ${sample_name}.bam -
samtools index -b ${sample_name}.bam
mv ${sample_name}.bam.bai ${sample_name}.bai
"""
}
process ValidateSamFile {
container ".dkr.ecr.us-west-2.amazonaws.com/genomes-in-the-cloud"
publishDir "/mnt/workflow/pubdir"
input:
file input_bam
output:
path "validation_report"
script:
"""
java -Xmx3G -jar /usr/gitc/picard.jar \
ValidateSamFile \
INPUT=${input_bam} \
OUTPUT=validation_report \
MODE=SUMMARY \
IS_BISULFITE_SEQUENCED=false
"""
}
```
For Nextflow v25.10, as an alternative to `publishDir`, you can use workflow outputs to export task content. The following example shows how to define a workflow `output` block that exports task results to Amazon S3.
```
process myTask {
input:
val data
output:
path 'result.txt'
script:
"""
echo ${data} > result.txt
"""
}
workflow {
main:
output_file = myTask('hello')
publish:
results = output_file
}
output {
results {
path '.'
}
}
```
For more information about workflow outputs, see [Workflow outputs](https://www.nextflow.io/docs/latest/workflow.html#workflow-output-def) in the Nextflow documentation.
# CWL workflow definition specifics
Workflows written in Common Workflow Language, or CWL, offer similar functionality to workflows written in WDL and Nextflow. You can use Amazon S3 or HealthOmics storage URIs as input parameters.
If you define input in a secondaryFile in a sub workflow, add the same definition in the main workflow.
HealthOmics workflows don't support operation processes. To learn more about operations processes in CWL workflows, see the [CWL documentation](https://www.commonwl.org/user_guide/topics/operations.html).
Best practice is to define a separate CWL workflow for each container that you use. We recommend that you don't hardcode the dockerPull entry with a fixed Amazon ECR URI.
**Topics**
+ [Convert CWL workflows to use HealthOmics](#workflow-cwl-convert)
+ [Opt out of task retry using `omicsRetryOn5xx`](#workflow-cwl-retry-5xx)
+ [Loop a workflow step](#workflow-cwl-loop)
+ [Retry tasks with increased memory](#workflow-cwl-out-of-memory-retry)
+ [Examples](#workflow-cwl-examples)
## Convert CWL workflows to use HealthOmics
To convert an existing CWL workflow definition to use HealthOmics, make the following changes:
+ Replace all Docker container URIs with Amazon ECR URIs.
+ Make sure that all the workflow files are declared in the main workflow as input, and all variables are explicitly defined.
+ Make sure that all JavaScript code is strict-mode complaint.
## Opt out of task retry using `omicsRetryOn5xx`
HealthOmics supports task retries if the task failed because of service errors (5XX HTTP status codes). By default, HealthOmics attempts up to two retries of a failed task. For more information about task retry in HealthOmics, see [Task Retries](monitoring-runs.md#run-status-task-retries).
To opt out of task retry for service errors, configure the `omicsRetryOn5xx` directive in the workflow definition. You can define this directive under requirements or hints. We recommend adding the directive as a hint for portability.
```
requirements:
ResourceRequirement:
omicsRetryOn5xx: false
hints:
ResourceRequirement:
omicsRetryOn5xx: false
```
Requirements override hints. If a task implementation provides a resource requirement in hints that is also provided by requirements in an enclosing workflow, the enclosing requirements takes precedence.
If the same task requirement appears at different levels of the workflow, HealthOmics uses the most specific entry from `requirements` (or `hints`, if there are no entries in `requirements`). The following list shows the order of precedence that HealthOmics uses to apply configuration settings, from lowest to highest priority:
+ Workflow level
+ Step level
+ Task section of the workflow definition
The following example shows how to configure the `omicsRetryOn5xx` directive at different levels of the workflow. In this example, the workflow-level requirement overrides the workflow level hints. The requirements configurations at the task and step levels override the hints configurations.
```
class: Workflow
# Workflow-level requirement and hint
requirements:
ResourceRequirement:
omicsRetryOn5xx: false
hints:
ResourceRequirement:
omicsRetryOn5xx: false # The value in requirements overrides this value
steps:
task_step:
# Step-level requirement
requirements:
ResourceRequirement:
omicsRetryOn5xx: false
# Step-level hint
hints:
ResourceRequirement:
omicsRetryOn5xx: false
run:
class: CommandLineTool
# Task-level requirement
requirements:
ResourceRequirement:
omicsRetryOn5xx: false
# Task-level hint
hints:
ResourceRequirement:
omicsRetryOn5xx: false
```
## Loop a workflow step
HealthOmics supports looping a workflow step. You can use loops to run workflow steps repeatedly until a specified condition is met. This is useful for iterative processes where you need to repeat a task multiple times or until a certain result is achieved.
**Note:** Loop functionality requires CWL version 1.2 or later. Workflows using CWL versions earlier than 1.2 do not support loop operations.
To use loops in your CWL workflow, define a Loop requirement. The following example shows the loop requirement configuration:
```
requirements:
- class: "http://commonwl.org/cwltool#Loop"
loopWhen: $(inputs.counter < inputs.max)
loop:
counter:
loopSource: result
valueFrom: $(self)
outputMethod: last
```
The `loopWhen` field controls when the loop terminates. In this example, the loop continues as long as the counter is less than the maximum value. The `loop` field defines how input parameters are updated between iterations. The `loopSource` specifies which output from the previous iteration feeds into the next iteration. The `outputMethod` field set to `last` returns only the final iteration's output.
## Retry tasks with increased memory
HealthOmics supports automatic retry of out-of-memory task failures. When a task exits with code 137 (out-of-memory), HealthOmics creates a new task with increased memory allocation based on the specified multiplier.
**Note**
HealthOmics retries out-of-memory failures up to 3 times or until the memory allocation reaches 1536 GiB, whichever limit is reached first.
The following example shows how to configure out-of-memory retry:
```
hints:
ResourceRequirement:
ramMin: 4096
http://arvados.org/cwl#OutOfMemoryRetry:
memoryRetryMultiplier: 2.5
```
When a task fails due to out-of-memory, HealthOmics calculates the retry memory allocation using the formula: `previous_run_memory × memoryRetryMultiplier`. In the example above, if the task with 4096 MB of memory fails, the retry attempt uses 4096 × 2.5 = 10,240 MB of memory.
The `memoryRetryMultiplier` parameter controls how much additional memory to allocate for retry attempts:
+ **Default value:** If you don't specify a value, it defaults to `2` (doubles the memory)
+ **Valid range:** Must be a positive number greater than `1`. Invalid values result in a 4XX validation error
+ **Minimum effective value:** Values between `1` and `1.5` are automatically increased to `1.5` to ensure meaningful memory increases and prevent excessive retry attempts
## Examples
The following is an example of a workflow written in CWL.
```
cwlVersion: v1.2
class: Workflow
inputs:
in_file:
type: File
secondaryFiles: [.fai]
out_filename: string
docker_image: string
outputs:
copied_file:
type: File
outputSource: copy_step/copied_file
steps:
copy_step:
in:
in_file: in_file
out_filename: out_filename
docker_image: docker_image
out: [copied_file]
run: copy.cwl
```
The following file defines the `copy.cwl` task.
```
cwlVersion: v1.2
class: CommandLineTool
baseCommand: cp
inputs:
in_file:
type: File
secondaryFiles: [.fai]
inputBinding:
position: 1
out_filename:
type: string
inputBinding:
position: 2
docker_image:
type: string
outputs:
copied_file:
type: File
outputBinding:
glob: $(inputs.out_filename)
requirements:
InlineJavascriptRequirement: {}
DockerRequirement:
dockerPull: "$(inputs.docker_image)"
```
The following is an example of a workflow written in CWL with a GPU requirement.
```
cwlVersion: v1.2
class: CommandLineTool
baseCommand: ["/bin/bash", "docm_haplotypeCaller.sh"]
$namespaces:
cwltool: http://commonwl.org/cwltool#
requirements:
cwltool:CUDARequirement:
cudaDeviceCountMin: 1
cudaComputeCapability: "nvidia-tesla-t4"
cudaVersionMin: "1.0"
InlineJavascriptRequirement: {}
InitialWorkDirRequirement:
listing:
- entryname: 'docm_haplotypeCaller.sh'
entry: |
nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv
inputs: []
outputs: []
```
# Example workflow definitions
The following example shows the same workflow definition in WDL, Nextflow, and CWL.
------
#### [ WDL ]
```
version 1.1
task my_task {
runtime { ... }
inputs {
File input_file
String name
Int threshold
}
command <<<
my_tool --name ~{name} --threshold ~{threshold} ~{input_file}
>>>
output {
File results = "results.txt"
}
}
workflow my_workflow {
inputs {
File input_file
String name
Int threshold = 50
}
call my_task {
input:
input_file = input_file,
name = name,
threshold = threshold
}
outputs {
File results = my_task.results
}
}
```
------
#### [ Nextflow ]
```
nextflow.enable.dsl = 2
params.input_file = null
params.name = null
params.threshold = 50
process my_task {
//
input:
path input_file
val name
val threshold
output:
path 'results.txt', emit: results
script:
"""
my_tool --name ${name} --threshold ${threshold} ${input_file}
"""
}
workflow MY_WORKFLOW {
my_task(
params.input_file,
params.name,
params.threshold
)
}
workflow {
MY_WORKFLOW()
}
```
------
#### [ CWL ]
```
cwlVersion: v1.2
class: Workflow
requirements:
InlineJavascriptRequirement: {}
inputs:
input_file: File
name: string
threshold: int
outputs:
result:
type: ...
outputSource: ...
steps:
my_task:
run:
class: CommandLineTool
baseCommand: my_tool
requirements:
...
inputs:
name:
type: string
inputBinding:
prefix: "--name"
threshold:
type: int
inputBinding:
prefix: "--threshold"
input_file:
type: File
inputBinding: {}
outputs:
results:
type: File
outputBinding:
glob: results.txt
```
------