Bad Response error when using query_prometheus tool

[luthfihizb]

I'm experiencing issue when calling query_prometheus tool

Parameters:

{
  "datasourceUid": "source_a",
  "expr": "rate(transaction_count[2m])",
  "queryType": "range",
  "startRfc3339": "2025-03-14T11:22:35Z",
  "endRfc3339": "2025-03-14T11:52:35Z",
  "stepSeconds": 30
}

Result:

{"error":"MCP error -32603: querying Prometheus range: bad_response: readObjectStart: expect { or n, but found c, error found in #1 byte of ...|cannot pars|..., bigger context ...|cannot parse \"\" to a valid timestamp|..."}

Is that related to timestamp format? I have tested the query directly in Grafana explorer and it's working fine. What's possibly wrong with the prompt?

[csmarchbanks]

Hello, all the inputs look correct to me, and I verified that they work against a local instance. The error I am seeing is that for some reason the Prometheus datasource you are querying is not returning a JSON payload, instead something starting with "c". A couple of questions:

1. What is the config you are using to point mcp-grafana to your Grafana instance?
2. Do you have a proxy or anything in front of either Grafana or Prometheus? Something that might return just text for an error instead of json?

You can actually find this error in the client_golang tests: https://github.com/prometheus/client_golang/blob/331dfab0cc853dca0242a0d96a80184087a80c1d/api/prometheus/v1/api_test.go#L1366, in their example a server is returning with a 422 and just "bad json" as the response data instead of valid json.

[csmarchbanks]

Actually the full message is cannot parse \"\" to a valid timestamp, which comes from Prometheus (or loki) code, not sure why you would get that considering we already parse your arguments before passing them along... Even if we did send an empty request to a prometheus instance it would return json:

$ curl https://prometheus.demo.prometheus.io/api/v1/query_range
{"status":"error","errorType":"bad_data","error":"invalid parameter \"start\": cannot parse \"\" to a valid timestamp"}

[luthfihizb]

I'm using mcp.json in Cursor

The connection is just fine because other tools like search_dashboards or list_datasources are working properly. Meanwhile it only happened to query_prometheus.

I tried to call it with instant type but got other error:

{
  "datasourceUid": "source_a",
  "expr": "rate(transaction_count[2m])",
  "queryType": "instant",
  "startRfc3339": "2025-03-01T00:00:00Z"
}

[sd2k]

Those parameters all look fine to me. This is a bit of a hunch but I wonder if the Prometheus URL in your Grafana instance has a trailing / (or is missing one)? The reason I ask is that our Prometheus client will send query / query_range requests as POST requests with the body encoded as JSON. If the URL is slightly misconfigured and returns a redirect (probably a 301) then the client will (probably) re-send the request as a GET without the body, so Prometheus would receive empty inputs for expr, start, etc.

I'm not even sure where you'd need to look in the logs to find these redirects though. Is your Prometheus datasource's URL pointing to a reverse proxy or anything like that?

[gemiit]

I'm experiencing issue when calling query_prometheus tool

Parameters:

{
  "datasourceUid": "source_a",
  "expr": "rate(transaction_count[2m])",
  "queryType": "range",
  "startRfc3339": "2025-03-14T11:22:35Z",
  "endRfc3339": "2025-03-14T11:52:35Z",
  "stepSeconds": 30
}

Result:

{"error":"MCP error -32603: querying Prometheus range: bad_response: readObjectStart: expect { or n, but found c, error found in #1 byte of ...|cannot pars|..., bigger context ...|cannot parse \"\" to a valid timestamp|..."}

Is that related to timestamp format? I have tested the query directly in Grafana explorer and it's working fine. What's possibly wrong with the prompt?

---

try:

{
  "datasourceUid": "source_a",
  "expr": "rate(transaction_count[2m])",
  "queryType": "range",
  "startRfc3339": "2025-03-14T11:22:35+08:00",
  "endRfc3339": "2025-03-14T11:52:35+08:00",
  "stepSeconds": 30
}