Skip to content

Issues: InftyAI/llmaz

Milestone v0.2.0
#259 opened Jan 27, 2025 by kerthcet
Open 1
Benchmark toolkit support
#66 opened Aug 6, 2024 by kerthcet
Open 5
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
40 Open 354 Closed
40 Open 354 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

fix: add ut for backend runtime. cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#428 opened May 22, 2025 by X1aoZEOuO Loading…
13
feat: support runai streamer for vllm feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#423 opened May 19, 2025 by cr7258 Loading…
1
3
[Umbrella] Metrics Aggregator Implementation feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#421 opened May 17, 2025 by kerthcet
2 tasks
@kerthcet
1
Enable envoy token rate limiting by configuration feature Categorizes issue or PR as related to a new feature. help wanted Extra attention is needed needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#412 opened May 14, 2025 by kerthcet
1 of 3 tasks
@googs1025
4
Add T/$ as indicator to measure the cost efficiency feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#401 opened May 6, 2025 by kerthcet
3 tasks
Manage the ai gateway resources automatically feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#387 opened Apr 30, 2025 by kerthcet
3 tasks done
1
open-webui support namespaces feature Categorizes issue or PR as related to a new feature. help wanted Extra attention is needed needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#386 opened Apr 29, 2025 by kerthcet
3 tasks
1
[Umbrella] grafana support with inference engines feature Categorizes issue or PR as related to a new feature. help wanted Extra attention is needed needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#385 opened Apr 29, 2025 by kerthcet
1 of 7 tasks
1
chore: add ci to test deploy with helm cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#377 opened Apr 24, 2025 by liangyuanpeng Loading…
14
[Umbrella] advanced traffic load balancing algorithms feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#376 opened Apr 24, 2025 by kerthcet
2 of 7 tasks
[Umbrella] inference engine metrics installation feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#375 opened Apr 24, 2025 by kerthcet
1 of 7 tasks
v0.2.0
1
Envoy gateway plugin support with random selection feature Categorizes issue or PR as related to a new feature. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#371 opened Apr 23, 2025 by kerthcet
3 tasks done
@googs1025
11
[OSPP] KEDA-based Serverless Elastic Scaling for llmaz feature Categorizes issue or PR as related to a new feature. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#362 opened Apr 22, 2025 by pacoxu
3 tasks
3
[OSPP]Enabling Efficient Model and Container Image Distribution in LLMaz with Dragonfly feature Categorizes issue or PR as related to a new feature. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#361 opened Apr 22, 2025 by pacoxu
3 tasks
Support runai model streamer for fast model loading feature Categorizes issue or PR as related to a new feature. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#352 opened Apr 18, 2025 by kerthcet
1 of 3 tasks
v0.2.0
1
@cr7258
5
Can this initcontainer image be configurable? needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#350 opened Apr 17, 2025 by pacoxu
6
after backend runtime update, should we re-create the playground pod? bug Categorizes issue or PR as related to a bug. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#332 opened Mar 26, 2025 by pacoxu
3
Proposal for LoRA autoscaler approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#313 opened Mar 13, 2025 by kerthcet Loading…
11
Lora Autoscaler feature Categorizes issue or PR as related to a new feature. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#287 opened Feb 28, 2025 by kerthcet
3 tasks done
v0.3.0
@kerthcet
3
Add popular open source models as in-tree support cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#268 opened Feb 14, 2025 by kerthcet
3 tasks
v0.2.0
8
Milestone v0.2.0 needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#259 opened Jan 27, 2025 by kerthcet
2 tasks
1
vllm only has /health endpoint needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#241 opened Jan 17, 2025 by kerthcet
3 tasks
2
Able to set the toRender parameters dynamically needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#239 opened Jan 16, 2025 by kerthcet
1 of 3 tasks
2
Unify the chat api for all inference servers feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#218 opened Dec 10, 2024 by kerthcet
3 tasks
2
Serverless support needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#192 opened Oct 29, 2024 by kerthcet
3 tasks done
v0.3.0
3
ProTip! What’s not been updated in a month: updated:<2025-04-22.