executor
The executor scope controls various executor behaviors.
executor.account
Used only by the SLURM, LSF, PBS/Torque and PBS Pro executors.
The project or organization account that should be charged for running the pipeline jobs.
executor.cpus
Support for grid executors was added.
Used only by the local executor and grid executors.
The maximum number of CPUs that can be requested by submitted tasks. Pending tasks are not submitted until enough CPUs are available, and a task that requests more CPUs than this limit fails. A job array counts the CPUs of all its tasks. Only CPUs requested with the cpus directive are counted (not clusterOptions), and a task without it counts as 1 CPU. For the local executor, the default is the number of CPUs on the system. For grid executors, there is no limit by default.
executor.dumpInterval
Determines how often to log the executor status (default: 5 min).
executor.exitReadTimeout
Used only by grid executors.
Determines how long to wait for the .exitcode file to be created after the task has completed, before returning an error status (default: 270 sec).
executor.jobName
Used only by grid executors and Google Batch.
Determines the name of jobs submitted to the underlying cluster executor:
executor.jobName = { "$task.name - $task.hash" }
The job name should satisfy the validation constraints of the underlying scheduler.
executor.killBatchSize
Determines the number of jobs that can be killed in a single command execution (default: 100).
executor.memory
Support for grid executors was added.
Used only by the local executor and grid executors.
The maximum amount of memory that can be requested by submitted tasks. Pending tasks are not submitted until enough memory is available, and a task that requests more memory than this limit fails. A job array counts the memory of all its tasks. Only memory requested with the memory directive is counted (not clusterOptions). For the local executor, the default is the total memory on the system. For grid executors, there is no limit by default.
executor.name
The name of the executor to be used (default: local).
executor.perCpuMemAllocation
Used only by the SLURM executor.
When true, memory allocations for SLURM jobs are specified as --mem-per-cpu <task.memory / task.cpus> instead of --mem <task.memory>.
executor.onlyJobState
Used only by the SLURM executor.
Requires SLURM 24.05 or later.
When true, job status queries use squeue --only-job-state without partition (-p) or user (-u) filters. This can reduce the load on the SLURM controller, especially if your SLURM administrator has enabled SchedulerParameters=enable_job_state_cache in your SLURM configuration. See --only-job-state for more information (default: false).
executor.perJobMemLimit
Used only by the LSF executor.
Enables the per-job memory limit mode for LSF jobs.
executor.perTaskReserve
Used only by the LSF executor.
Enables the per-task memory reserve mode for LSF jobs.
executor.pollInterval
Determines how often to check for process termination. Default varies for each executor.
executor.queueGlobalStatus
Determines how job status is retrieved. When false only the queue associated with the job execution is queried. When true the job status is queried globally i.e. irrespective of the submission queue (default: false).
executor.queueSize
The number of tasks the executor will handle in a parallel manner. A queue size of zero corresponds to no limit. Default varies for each executor.
executor.queueStatInterval
Used only by grid executors.
Determines how often to fetch the queue status from the scheduler (default: 1 min).
executor.submitRateLimit
Determines the max rate of job submission per time unit, for example '10sec' (10 jobs per second) or '50/2min' (50 jobs every 2 minutes) (default: unlimited).
executor.retry.delay
Used only by grid executors.
Delay when retrying failed job submissions (default: 500ms).
executor.retry.jitter
Used only by grid executors.
Jitter value when retrying failed job submissions (default: 0.25).
executor.retry.maxAttempts
Used only by grid executors.
Max attempts when retrying failed job submissions (default: 3).
executor.retry.maxDelay
Used only by grid executors.
Max delay when retrying failed job submissions (default: 30s).
executor.retry.reason
This option was renamed from executor.submit.retry.reason to executor.retry.reason.
Used only by grid executors.
Regex pattern that when verified causes a failed submit operation to be re-tried (default: Socket timed out).
Executor-specific defaults
Some executor settings have different default values depending on the executor.
| Executor | queueSize | pollInterval |
|---|---|---|
| AWS Batch | 1000 | 10s |
| Azure Batch | 1000 | 10s |
| Google Batch | 1000 | 10s |
| Grid Executors | 100 | 5s |
| Kubernetes | 100 | 5s |
| Local | N/A | 100ms |
Executor-specific configuration
Executor config settings can be applied to specific executors by prefixing the executor name with the symbol $ and using it as special scope. For example:
// block syntax
executor {
$sge {
queueSize = 100
pollInterval = '30sec'
}
$local {
cpus = 8
memory = '32 GB'
}
}
// dot syntax
executor.$sge.queueSize = 100
executor.$sge.pollInterval = '30sec'
executor.$local.cpus = 8
executor.$local.memory = '32 GB'