add route payload to deploy Inference Endpoints #3013
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
src/huggingface_hub/hf_api.py
Outdated
| domain (`str`, *optional*): | ||
| The custom domain for the Inference Endpoint (e.g. `"my-new-domain.cool-website.woof"`). | ||
| path (`str`, *optional*): | ||
| The custom path for the Inference Endpoint, should start with a `/` (e.g. `"/models/google-bert/bert-base-uncased"`). |
There was a problem hiding this comment.
What does it means exactly? Is it that the endpoint should be served under endpoints-(...)-url.co/models/google-bert/bert-base-uncased or is it that the URL endpoints-(...)-url.co/ points to the /models/google-bert/bert-base-uncased path of the docker image? (worth some clarification in the docstring IMO)
There was a problem hiding this comment.
IIRC @oOraph mentioned this is applicable for anyone with a custom domain registered as well, so I'd consider this to be generic.
There was a problem hiding this comment.
Yes, worth having the parameter supported in huggingface_hub! Just wanted to be sure what it meant exactly (The custom path for the Inference Endpoint is not explicit enough IMO)
There was a problem hiding this comment.
updated here: 54572d5
| The custom domain for the Inference Endpoint (e.g. `"my-new-domain.cool-website.woof"`). | ||
| path (`str`, *optional*): | ||
| The custom path for the Inference Endpoint, should start with a `/` (e.g. `"/models/google-bert/bert-base-uncased"`). | ||
| cache_http_responses (`bool`, *optional*): |
There was a problem hiding this comment.
(link this private thread to avoid forgetting about it - if this attribute is HF-only, let's remove it)
| cache_http_responses (`bool`, *optional*): | ||
| Whether to cache HTTP responses from the Inference Endpoint. Defaults to `False`. | ||
| tags (`List[str]`, *optional*): | ||
| A list of tags to associate with the Inference Endpoint. |
There was a problem hiding this comment.
Out of curiosity, how do we play with tags once set? (e.g. can we list endpoints based on tags? or is it a UI change?)
There was a problem hiding this comment.
yeah! you can list all endpoints deployed with a certain specific tag
…uggingface_hub into vb/upd-inference-endpoints
|
CI failure is unrelated, merging it now (code quality is ✔️ though) |
| cache_http_responses (`bool`, *optional*): | ||
| Whether to cache HTTP responses from the Inference Endpoint. |
There was a problem hiding this comment.
So we're okay with keeping this?
There was a problem hiding this comment.
yes I think so in the end. It's also publicly documented here: https://api.endpoints.huggingface.cloud/#post-/v2/endpoint/-namespace-. This method is already not much used so if it can make HF's people life easier, let's go for it
ref: https://huggingface.slack.com/archives/C016D661PAN/p1744900723918749?thread_ts=1742997838.592509&cid=C016D661PAN