Multi-node Execution Support in Sarek on HPC Systems

[anya-shm]

Hello Sarek team,

I am currently running the nf-core Sarek pipeline (v3.5.0) on an HPC system and encountered an issue when attempting to utilize multiple nodes. I launched the pipeline with the following PBS settings:

#PBS -l nodes=3:ppn=8

This should provide a total of 24 CPUs (3 nodes × 8 CPUs per node). However, I received the following error during execution:

Process requirement exceeds available CPUs -- req: 24; avail: 8

It appears that the pipeline is only recognizing the CPUs on a single node rather than across all allocated nodes. I would like to ask:

• Does Sarek support multi-node execution when run on HPC systems?
• If supported, are there specific configurations or additional parameters required to enable multi-node execution?

Any guidance or documentation on how to resolve this issue would be greatly appreciated.

Thank you for your help!

Best regards,
Anna.

[FriederikeHanssen]

Do you mean with multi-node execution that a single task is run on two nodes? If so, then I don't believe nextflow and by extension sarek supports it.

The above error indicates that you are requesting 24CPUs for a single task but there are only nodes with 8 available. To avoid this error you can use [resourceLimits](https://www.nextflow.io/docs/latest/reference/process.html#resourcelimits) and set them to the maximal allowed resources an individual task can consume,

[anya-shm]

Thank you for the quick reply and clarification!

Yes, I meant multi-node for a single process. I will try with resourceLimits, as you suggested.

Do you mean with multi-node execution that a single task is run on two nodes? If so, then I don't believe nextflow and by extension sarek supports it.

The above error indicates that you are requesting 24CPUs for a single task but there are only nodes with 8 available. To avoid this error you can use [resourceLimits](https://www.nextflow.io/docs/latest/reference/process.html#resourcelimits) and set them to the maximal allowed resources an individual task can consume,