Cache higher order objects in HPACK's dynamic table.

[buchgr]

In #2217, @carl-mastrangelo had the idea to cache high order objects in HPACK's dynamic table. Carl mentioned the example of instead of parsing a jwt token on every request, we could cache an object that already contains the parsed representation, reducing garbage creation and parsing overhead.

Additionally, since Metadata uses String for names, we could for header names cache the String representation.

This would mostly require changes to Netty's HPACK implementation. We need some way to attach additional headers. I suggested to modify the StaticTable, Decoder and HuffmanDecoder to take a HeaderLiteralFactory as an optional ctor argument.

interface HeaderLiteralFactory {
  CharSequence newKey(byte[] bytes);
  CharSequence newValue(CharSequence key, byte[] bytes);
}

Netty's default implementation would simply return AsciiString objects. The gRPC implementation, could use String objects for keys. For values, we could special case certain header names and return holder objects e.g.

class JwtHolder implements CharSequence {
  JwtObject jwt;
  byte[] bytes;
}

Risks
The decoder has no power over what gets cached and what doesn't - it's up to the encoder. We might introduce overhead for headers that don't get cached. Also, a (name, value) is treated as one header entry in HPACK's dynamic table. Say if a users use custom headers, with lots of different values, then caching in the dynamic table might not be very effective, as entries might get evicted before they can be reused (if ever).

[ejona86]

Also, a (name, value) is treated as one header entry in HPACK's dynamic table.

That's true, although the name can be copied from other entries. See RFC 7541 §6.2.1.

If the header field name matches the header field name of an entry stored in the static table or the dynamic table, the header field name can be represented using the index of that entry.

[buchgr]

@ejona86

Good point I didn't know that. I guess it depends on the implementation of the remote endpoints encoder. Also, it might help slightly but we might still end up caching many unique values that aren't re-used and take up space in the dynamic table. Just wanted to point that out.