Dockerfile fails to build container - module `mako` not found

[zerofeerouting]

Issue and Steps to Reproduce

Fetch v0.12.1 from Github and try spinning up a Docker container via Dockerfile

[...]
CC: cc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -Wno-maybe-uninitialized -std=gnu11 -g -fstack-protector-strong -Og -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/x86_64-linux-gnu/libbacktrace-build -I external/libsodium/src/libsodium/include -I external/libsodium/src/libsodium/include/sodium -I external/x86_64-linux-gnu/libsodium-build/src/libsodium/include -I . -I/usr/local/include  -I/usr/include/postgresql    -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DBUILD_ELEMENTS=1  -c -o
LD: cc   -Og  config.vars  -Lexternal/x86_64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -lsodium -L/usr/local/include -Wl,-dn -lgmp -lsqlite3 -lz -Wl,-dy -lm -lpthread -ldl  -L/usr/lib/x86_64-linux-gnu -lpq -o
wiregen hsmd/hsmd_wiregen.h
cc ccan/ccan/cdump/tools/cdump-enumstr.c
cc ccan/ccan/cdump/cdump.c
Traceback (most recent call last):
  File "/opt/lightningd/tools/generate-wire.py", line 27, in <module>
    from mako.template import Template
ModuleNotFoundError: No module named 'mako'
make: *** [Makefile:331: hsmd/hsmd_wiregen.h] Error 1
make: *** Waiting for unfinished jobs....
ERROR: Service 'cln' failed to build: The command '/bin/sh -c ./configure --prefix=/tmp/lightning_install --enable-static && make -j3 DEVELOPER=${DEVELOPER} && make install' returned a non-zero code: 2

getinfo output

v0.12.1

[zerofeerouting]

Additional info - ran into this error a couple of weeks ago but assumed a mistake on my end (since I was trying to run it on a newly set up VPS and didn't have the time to look into it).

It looks like mako is being installed by the Dockerfile process before compiling but still it runs into this error.

Other versions run into this error too (tried with v0.11.2).

[vincenzopalazzo]

Stupid question, what docker file are you running? we have a few in the repository

[zerofeerouting]

Using the default: https://github.com/ElementsProject/lightning/blob/master/Dockerfile

[jamaljsr]

We are also facing this same issue in jamaljsr/polar#602.

Just to add more info, about a month ago I built v0.12.0 using the same Dockerfile above with no issues. Maybe something changed in the remote dependencies.

[zerofeerouting]

Yes. I assume it has something to do with the base image debian:bullseye-slim or which version of python is used to compile.

I was unable to build with the exact same Dockerfile that has worked before. So this is definitely something that was introduced by one of the dependencies.

[cdecker]

I have updated the Dockerfile for the v0.12.1 release in this branch: https://github.com/ElementsProject/lightning/tree/docker-v0.12.1

It just switches from being poetry back to pip, which seems to be more stable. We can't change the tag on the v0.12.1 release, but using that tiny patch should get it working again. FWIW I also updated the v0.12.1 and latest tags on Docker Hub:

https://hub.docker.com/r/elementsproject/lightningd/tags

Let me know if this works as a basis for you guys

[vincenzopalazzo]

there is some consideration to improve the usage of poetry instead to change it? like #5606

I think poetry fix some bugs in the last release that impact our usage

[cdecker]

No, I want to get rid of poetry as much as possible, and since we can export the requirements.txt file and just use pip (reusing the cached dependency resolution from poetry) I think that already solves 99% of the reason we switched to poetry in the first place.

• poetry for developers that need to occasionally update dependencies or add new ones
• pip for anybody else (including automated builds)

[jamaljsr]

Let me know if this works as a basis for you guys

Confirmed, this fixes the build for me. Thanks!

[NicolasDorier]

just to mention I ran into this issue, ended up doing a pip install mako before ./configure, it worked. 🤢