[Questions] Why escript folder contains many identical files?

[inikulshin]

I noticed a big difference in RabbitMQ assets for Windows:

rabbitmq-server-3.13.7.exe is 15.9 MB
rabbitmq-server-windows-3.13.7.zip is 30.7 MB

The difference originates from different compression algorithms: lzma in NSIS installer and deflate(?) for creating zip file. The former compresses identical (binary same) files effectively, while the latter compresses each file independently.

But my question is why RabbitMQ installation (on Windows only?) requires so many files with identical content? Can't they be optimized?

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I agree to provide all relevant information (versions, logs, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

3.12.x or older

Erlang version used

26.2.x

Operating system (distribution) used

Windows

How is RabbitMQ deployed?

Windows installer

[michaelklishin]

@inikulshin you know very well that 3.12 is out of community support.

You haven't mentioned what exactly are those "identical files", which is not helping us help you.

Most CLI tools (except for rabbitmq-server and its Windows service counterpart) is a single executable Erlang/BEAM binary (an escript) that exposes a certain set of commands — a "scope" in CLI source code terms — depending on the name of the executable.

This binary is (way) smaller than 1 MiB, so duplicating it N times does not meaningfully change the side of the distribution. There are no plans to change this, this has been the case since 2016 and this is the 2nd time that I recall someone even noticing.

[inikulshin]

@michaelklishin I just downloaded and installed RabbitMQ 4.1.0.

rabbitmq-server-4.1.0.exe is 25.1 MB
rabbitmq-server-windows-4.1.0.zip is 88.6 MB

Those identical escript files are now 76 MiB out of 100 MiB RabbitMQ installation requires.

I know it's a very minor issue and it's not about few MBs, but it's still pretty annoying for people with various disorders.

[gomoripeti]

I guess an escript is large because it contains the whole Elixir standard library. How Elixir is built has changed in 4.1, that could be the reason why each escript grown substantially between 4.0 and 4.1 (good catch)

Im not familiar how the escripts are assembled, but probably could be optimised so that each shell script in sbin uses the same escript.

Unfortunately these are not high priority issues, but interesting if it can be tweaked.

[michaelklishin]

RabbitMQ CLI tools have grown to some 70 commands plus several new dependencies (such as observer_cli) compared to just a few years ago.

Each binary is a copy because it is much easier than dealing with symbolic links. The important part is the name, it controls the scope (the set of available commands).

CLI tools will likely change by 4.3 (unlikely earlier) and much of their code will move to the server.

[michaelklishin]

I can also imagine that erlang.mk's Elixir compilation support may be omitting certain settings that can optimize the size of the binary.

@lhoguin would know what the defaults are and why. And maybe the response is "it's all defaults unless you override them".

[lhoguin]

I'll check.

[lhoguin]

The difference is that the beam files now include debug info, while they were stripped before. We could do the same but it requires additional Erlang.mk work because we don't compile Elixir so we need to strip the beam files before generating the escript. It's not too difficult but not trivial either.

[inikulshin]

Another minor issue(?): redbug plugin was downgraded from redbug-2.1.0 in RabbitMQ 4.0.7 to redbug-2.0.7 in RabbitMQ 4.1.0

Maybe it's intentional, but maybe just a missing merge between branches.

[michaelklishin]

This was not intentional and most likely a side effect of one of the two build system migrations in the 4.1.0 cycle.