GPU cache optimization ideas

[glennw]

A couple of ideas for optimizing the GPU cache:

• Much of the data we use is integer based and fits within a u16. For example, texture coordinates, render task rects are suitable to store as u16. Since they are natively integer this would also remove int <-> float conversions on the CPU. This could save some CPU time, and also memory transfer size.

• GPU cache addresses are currently a (u, v) pair. This is necessary because the GPU cache packs variable sized data within a single texture page. Since it uses a slab allocator, we could potentially do something like:

  • Switch the GPU cache to be backed by an array texture.
  • Each texture layer has a fixed slab size.
  • The correct layer to sample from can then be determined implicitly, since the shader knows the # of GPU cache blocks that are allocated.
  • This would mean a GPU cache address can become a single u16, which would significantly reduce the size of each instance structure.

[kvark]

A couple of ideas for optimizing the GPU cache:

Is GPU cache showing up in any profile data these days? I've only seen the updates of it taking 3% of the compositor time, roughly.

Much of the data we use is integer based and fits within a u16.

We could use RGBA32UI format and cast some of the fields to float with uintBitsToFloat (available from GLES 3.0 and up).

Switch the GPU cache to be backed by an array texture.

I'm concerned that to blow up both the total VRAM allocated size of the GPU cache and the need to resize it (causing serious hitches).

The correct layer to sample from can then be determined implicitly, since the shader knows the # of GPU cache blocks that are allocated.

Could you explain this part a bit more?

This would mean a GPU cache address can become a single u16, which would significantly reduce the size of each instance structure.

Pretty sure we can reduce the current pointer size to at least 24: each dimension is to be capped by 4K. Surely, u16 would be nicer in a sense of less bit mangling. Do you have a breakdown on what our instance sizes are at the moment, and (more importantly) how does uploading instances fare in terms of time spent versus everything else we are doing on the compositor? I'd be surprised if we notice that time.

[nical]

I thought I had replied to this thread a while back but we should pass device space coordinates as 16 bits unorms instead f32. It would free at least 64 bits per primitive that has uv coordinates (and since I heard 80% of the primitives tend to be glyphs that's a lot of them).
Edit: hm, actually I suppose glyphs share the glyph resources, but still that's quite a few uv rects with images and masks.