More memory-efficient parallel encoding

[RunDevelopment]

Right now, parallel encoding is implemented by collecting the encoded coded fragments into one large Vec<Vec<u8>>. This means that the entire encoded surface is stored in memory, which is just unnecessary.

Let n be the number of CPU cores, then we only need to store n encoded fragments in memory at a time. The basic idea is to process them in parallel, but commit them to disk in order. Like this:

4 CPU cores

Core 1: <- encode frag 1, commit -> <- encode frag 5, commit -> ...
Core 2: <- encode frag 2, (wait) commit -> <- encode frag 6, commit -> ...
Core 3: <- encode frag 3, (    wait   ) commit -> <- encode frag 7, commit -> ...
Core 4: <- encode frag 4, (       wait       ) commit -> <- encode frag 8, commit -> ...
time -->

Commits go to a single writer, so they lock on a mutex waiting for their turn.

This also means that if frag_time < commit_time * (n-1) (where frag_time is the time it takes to encode one fragment and commit_time is the time it takes to write one fragment to disk), we will unnecessarily block rayon's thread pool. This is only a problem for others though. It doesn't make parallel encoding any slower.

Note that this approach is not faster than the current approach. They will both be equally fast (ignoring synchronization overhead). The difference is that this approach uses less memory, which will make it possible to encode certain large images in parallel that the current approach cannot (due to OOM crash).

[fintelia]

If the full uncompressed input is passed in via memory, then storing the compressed output in memory will be only a fraction of that size. There isn't anything wrong with avoiding this memory overhead, but it is unlikely to make a huge difference either. In other encoders, I've debated whether to try to avoid buffering the full output in memory and opted not to bother implementing it.

[RunDevelopment]

If the full uncompressed input is passed in via memory, then storing the compressed output in memory will be only a fraction of that size.

Depends. Encoding a Grayscale U8 images as BC{2,3,5,6,7} or BC{1,4} will result in a 1:1 or 1:2 compression ratio. So the ratio can be close to 1.

Of course, the situation is a lot better for the common case of RGB and RGBA, so you're probably right and this isn't an issue. If it ever does turn out to be an issue, we can fix it then.

Thank you for the input @fintelia!