Open
Description
I had run out of space on my test VM, and trying to
eb --fetch QuantumESPRESSO-7.4-foss-2024a.eb
and it correctly display the fail, and it's easy to see it failed to due disk space in the log
$ eb --fetch QuantumESPRESSO-7.4-foss-2024a.eb
== Temporary log file in case of crash /dev/shm/eb-t6ieyp0y/easybuild-sgcd3k5r.log
== processing EasyBuild easyconfig /apps/easybuild-easyconfigs/easybuild/easyconfigs/q/QuantumESPRESSO/QuantumESPRESSO-7.4-foss-2024a.eb
== building and installing QuantumESPRESSO/7.4-foss-2024a...
>> installation prefix: /apps/Arch/software/QuantumESPRESSO/7.4-foss-2024a
== fetching files and verifying checksums...
ERROR: Shell command failed!
full command -> git clone --no-checkout https://github.com/wannier-developers/wannier90.git
exit code -> 128
called from -> 'get_source_tarball_from_git' function in /apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/filetools.py (line 2719)
working directory -> /dev/shm/eb-t6ieyp0y/tmpu1t3yx6s
output (stdout + stderr) -> /dev/shm/eb-t6ieyp0y/run-shell-cmd-output/git-3p2mjjk_/out.txt <-- I can see the error easily
interactive shell script -> /dev/shm/eb-t6ieyp0y/run-shell-cmd-output/git-3p2mjjk_/cmd.sh
but when I did this via --job it crashes hard
$ eb --job QuantumESPRESSO-7.4-foss-2024a.eb
== Temporary log file in case of crash /dev/shm/eb-uk5cm_ru/easybuild-3hy06wec.log
Traceback (most recent call last):
File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 801, in <module>
main_with_hooks()
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 787, in main_with_hooks
main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 742, in main
do_cleanup = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 555, in process_eb_args
submit_jobs(ordered_ecs, eb_go.generate_cmd_line(), testing=testing)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 159, in submit_jobs
return build_easyconfigs_in_parallel(command, ordered_ecs, prepare_first=prepare_first)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 97, in build_easyconfigs_in_parallel
prepare_easyconfig(easyconfig)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 214, in prepare_easyconfig
easyblock_instance.fetch_step(skip_checksums=True)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/easyblocks/generic/configuremake.py", line 231, in fetch_step
super(ConfigureMake, self).fetch_step(*args, **kwargs)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 2526, in fetch_step
self.fetch_sources(self.cfg['sources'], checksums=self.cfg['checksums'])
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 532, in fetch_sources
src_spec = self.fetch_source(source, checksum=checksum)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 488, in fetch_source
path = self.obtain_file(filename, extension=extension, download_filename=download_filename,
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 930, in obtain_file
return get_source_tarball_from_git(filename, targetdir, git_config)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/filetools.py", line 2719, in get_source_tarball_from_git
run_shell_cmd(' '.join(clone_cmd), hidden=True, verbose_dry_run=True, work_dir=tmpdir)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 192, in cache_aware_func
res = func(cmd, *args, **kwargs)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 627, in run_shell_cmd
raise_run_shell_cmd_error(res)
File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 175, in raise_run_shell_cmd_error
raise RunShellCmdError(cmd_res, caller_info)
easybuild.tools.run.RunShellCmdError: Shell command 'git' failed!
so the preparation that --job does to download the sources isn't handling run_shell_cmd failures the same way. I think it should.