Skip to content

Stacktrace without error message when git fails during --job #4750

Open
@Micket

Description

@Micket

I had run out of space on my test VM, and trying to

eb --fetch QuantumESPRESSO-7.4-foss-2024a.eb

and it correctly display the fail, and it's easy to see it failed to due disk space in the log

$ eb --fetch QuantumESPRESSO-7.4-foss-2024a.eb 
== Temporary log file in case of crash /dev/shm/eb-t6ieyp0y/easybuild-sgcd3k5r.log
== processing EasyBuild easyconfig /apps/easybuild-easyconfigs/easybuild/easyconfigs/q/QuantumESPRESSO/QuantumESPRESSO-7.4-foss-2024a.eb
== building and installing QuantumESPRESSO/7.4-foss-2024a...
  >> installation prefix: /apps/Arch/software/QuantumESPRESSO/7.4-foss-2024a
== fetching files and verifying checksums...

ERROR: Shell command failed!
    full command              ->  git clone --no-checkout https://github.com/wannier-developers/wannier90.git
    exit code                 ->  128
    called from               ->  'get_source_tarball_from_git' function in /apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/filetools.py (line 2719)
    working directory         ->  /dev/shm/eb-t6ieyp0y/tmpu1t3yx6s
    output (stdout + stderr)  ->  /dev/shm/eb-t6ieyp0y/run-shell-cmd-output/git-3p2mjjk_/out.txt  <-- I can see the error easily
    interactive shell script  ->  /dev/shm/eb-t6ieyp0y/run-shell-cmd-output/git-3p2mjjk_/cmd.sh

but when I did this via --job it crashes hard

$ eb --job QuantumESPRESSO-7.4-foss-2024a.eb 
== Temporary log file in case of crash /dev/shm/eb-uk5cm_ru/easybuild-3hy06wec.log
Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 801, in <module>
    main_with_hooks()
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 787, in main_with_hooks
    main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 742, in main
    do_cleanup = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 555, in process_eb_args
    submit_jobs(ordered_ecs, eb_go.generate_cmd_line(), testing=testing)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 159, in submit_jobs
    return build_easyconfigs_in_parallel(command, ordered_ecs, prepare_first=prepare_first)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 97, in build_easyconfigs_in_parallel
    prepare_easyconfig(easyconfig)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 214, in prepare_easyconfig
    easyblock_instance.fetch_step(skip_checksums=True)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/easyblocks/generic/configuremake.py", line 231, in fetch_step
    super(ConfigureMake, self).fetch_step(*args, **kwargs)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 2526, in fetch_step
    self.fetch_sources(self.cfg['sources'], checksums=self.cfg['checksums'])
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 532, in fetch_sources
    src_spec = self.fetch_source(source, checksum=checksum)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 488, in fetch_source
    path = self.obtain_file(filename, extension=extension, download_filename=download_filename,
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 930, in obtain_file
    return get_source_tarball_from_git(filename, targetdir, git_config)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/filetools.py", line 2719, in get_source_tarball_from_git
    run_shell_cmd(' '.join(clone_cmd), hidden=True, verbose_dry_run=True, work_dir=tmpdir)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 192, in cache_aware_func
    res = func(cmd, *args, **kwargs)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 627, in run_shell_cmd
    raise_run_shell_cmd_error(res)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 175, in raise_run_shell_cmd_error
    raise RunShellCmdError(cmd_res, caller_info)
easybuild.tools.run.RunShellCmdError: Shell command 'git' failed!

so the preparation that --job does to download the sources isn't handling run_shell_cmd failures the same way. I think it should.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions