Open
Description
What would you like to be added:
Install the grafana dashboard for inference engines:
- vllm: https://docs.vllm.ai/en/latest/getting_started/examples/prometheus_grafana.html
- sglang: https://github.com/sgl-project/sglang/tree/main/examples/monitoring
- llama.cpp
- TGI
Why is this needed:
Completion requirements:
This enhancement requires the following artifacts:
- Design doc
- API change
- Docs update
The artifacts should be linked in subsequent comments.