containerOptions directive bug with AWS Batch and "--shm-size=some_number"

[mriffle]

Bug report

When specifying when using AWS Batch:

containerOptions '--shm-size=1000000000'

An unhelpful error message will be produced when running the workflow:

ERROR ~ Error executing process > 'MY_PROCESS_NAME'

Caused by:
  For input string: "true"



 -- Check '.nextflow.log' file for details

Note that this only happens when using AWS Batch, other executors work correctly and set the shm-size as expected.

This can be fixed by removing the =, such as:
containerOptions '--shm-size 1000000000'

However, there is no indication whatsoever in the error message that this is the problem. It took me some hours to figure this out.

I assume this is related to special syntax required for AWS Batch and the parsing of this specific container option when Nextflow is run.

Expected behavior and actual behavior

The expected behavior is that the shm-size variable will be set appropriately, independent of which executor is being used.

Steps to reproduce the problem

Put this in a process and run using AWS Batch:

containerOptions '--shm-size=1000000000'

Program output

ERROR ~ Error executing process > 'MY_PROCESS_NAME'

Caused by:
  For input string: "true"



 -- Check '.nextflow.log' file for details

Environment

• Nextflow version: [24.04.3]
• Java version: [openjdk version "11.0.24" 2024-07-16]
• Operating system: [Linux]
• Bash version: (GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu))

[bentsherman]

See also #5174

Need to see if we can refactor the GNU args parsing to also support --option=value, assuming that is actually "GNU-compliant". Otherwise we can try to improve the error message.

Also, this is not documented but containerOptions can also be a map:

containerOptions 'shm-size': 1000000000

Maybe it would make more sense to emphasize this form in the docs, since there is no need to go through the extra step of users coding their options as CLI options and Nextflow trying to parse them

[pditommaso]

Do you mean that -shm-size 1000000000 works?

[bentsherman]

Yes that also works. The = is the problem

[pditommaso]

I see, then at least we can a warning in the docs

[bentsherman]

@pditommaso I'm concerned that the regex for parseGnuArgs is just throwing out the parts that don't match, especially without giving a warning. I think we could improve the parsing to handle things like =.

Otherwise, I would be fine with just adding a note to the docs and encouraging users to use a map instead.