Azure, WARN: Failed to cleanup work dir

[des2037]

Hi NextFlow devs/community,

I'm running into an error when I try to clean my working directories (running on Microsoft Azure). Specifically, in my nextflow.config, I have (as the last line)

cleanup = true

The error I receive is:

WARN: Unable to start plugin 'nf-azure' required by az://<my_container_name>/work/2e/8cd...
WARN: Failed to cleanup work dir: /my_container_name/work

Here is my log for the run:
nextflow.log

WIth the exception of not cleaning the working directory, the job runs as expected (files are read from and created in my Azure blob containers, so I don't think blob permissions the problem).

Any help with this error is greatly appreciated! Please let me know if I can provide any additional information that might be helpful.

Best,
Derek

[abhi18av]

Hi @des2037 ,

Thanks for letting us know about this issue. I noticed that the nf-azure plugin version is 0.11.2

Could you please try again with the latest plugin?

You'd need to update your nextflow.config with the following snippet (from docs)

plugins {
  id 'nf-azure@0.12.0'
}

If it fails again, please share the .nextflow.log file with us.

[des2037]

Hi @abhi18av

Thanks very much for the fast reply and suggestion.

I've updated the nf-azure plugin to 0.12.0, but sadly I'm still getting the same error.

Here is my nextflow.log:
nextflow.log

Thanks for the help!

[abhi18av]

Thanks @des2037 , I'll look into this soon.

[pditommaso]

The problem here is that the cleanup happens after the Azure plugin shutdown :/

Feb-28 16:04:09.986 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-azure
Feb-28 16:04:10.052 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
Feb-28 16:04:10.054 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Feb-28 16:04:10.054 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Feb-28 16:04:10.055 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Feb-28 16:04:10.057 [main] WARN  nextflow.file.FileHelper - Unable to start plugin 'nf-azure' required by az://hwlab-nextflow-temp/work/cf/f5868bd15108a917675cba34e5ecde

nextflow/modules/nextflow/src/main/groovy/nextflow/Session.groovy

Line 1111 in 0045834

log.warn("Failed to cleanup work dir: $workDir")

[mike-miroliubov]

Hi @pditommaso, is there any news on this issue? Was this addressed? We are seeing the same behavior with AWS S3, Nextflow version 21.10.6 build 5660:

Jul-12 18:44:22.918 [main] WARN  nextflow.file.FileHelper - Unable to start plugin 'nf-amazon' required by s3://***
java.lang.NullPointerException: null
        at nextflow.plugin.PluginsFacade.isStarted(PluginsFacade.groovy:290)
        at nextflow.plugin.PluginsFacade.startIfMissing(PluginsFacade.groovy:382)
        at nextflow.plugin.Plugins.startIfMissing(Plugins.groovy:73)
        at nextflow.file.FileHelper.autoStartMissingPlugin(FileHelper.groovy:284)
        at nextflow.file.FileHelper.asPath(FileHelper.groovy:264)
        at nextflow.Session$_cleanup_closure22.doCall(Session.groovy:1089)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
        at groovy.lang.Closure.call(Closure.java:412)
        at nextflow.CacheDB.eachRecord(CacheDB.groovy:291)
        at nextflow.Session.cleanup(Session.groovy:1085)
        at nextflow.script.ScriptRunner.terminate(ScriptRunner.groovy:223)
        at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:122)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:308)
        at nextflow.cli.Launcher.run(Launcher.groovy:480)
        at nextflow.cli.Launcher.main(Launcher.groovy:639)
Jul-12 18:44:22.919 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
Jul-12 18:44:22.919 [main] WARN  nextflow.Session - Failed to cleanup work dir: /***

Thanks!

[mike-miroliubov]

Upd: same issue with version 22.04.4, plugin nf-amazon@1.7.2

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[pditommaso]

Related to #3645