Stream error when hitting /pyroscope/render endpoint

[Sreenitti]

I was trying to hit the endpoint /pyroscope/render with some load and got the following errors:

1. {"code":"unknown","message":"canceled: stream error: stream ID 47061; CANCEL"}
2. {"code":"unknown","message":"canceled: stream error: stream ID 23551; INTERNAL_ERROR; i/o timeout"}
3. {"code":"unknown","message":"deadline_exceeded: i/o timeout"}

This is the curl command i used
curl --get \ --data-urlencode "query=process_cpu:cpu:nanoseconds:cpu:nanoseconds{service_name=\"test-app\"}" \ --data-urlencode "from=now-1h" \ http://pyroscope-query-frontend.pyroscope-test.svc.cluster.local/.:4040/pyroscope/render

When I checked the logs in the query frontend pod, I was able to find the following errors were numerous times:

1. ts=2025-04-09T06:35:35.420270927Z caller=tcp_transport.go:322 level=error component="memberlist TCPTransport" msg="unknown message type" msgType=G remote=10.227.16.242:45894

2. ts=2025-04-09T06:35:32.457089689Z caller=frontend_scheduler_worker.go:360 level=debug component=frontend msg="stream context finished" err="context canceled"