OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
OpenLLM scores 5.5/10 (Solid) in Gateways & Routers.
Language Python License Apache-2.0 Stars 12.4k★ Type service Self-host Yes Repo bentoml/OpenLLM Homepage https://github.com/bentoml/OpenLLM
Quality
Solid 5.5
Adoption 52
Activity 29
Maturity 97
Community 49
Capability 50
Show the math
Overall 5.5/10
value = 5.5/10
score = value
Adoption 5.7/10
value = 52.3/100
score = 1 + 9 * value / 100
Activity 3.6/10
value = 28.6/100
score = 1 + 9 * value / 100
Maturity 9.7/10
value = 97/100
score = 1 + 9 * value / 100
Community 5.4/10
value = 48.9/100
score = 1 + 9 * value / 100
Capability 5.5/10
value = 50/100
score = 1 + 9 * value / 100
Key metrics
12.4k★ GitHub stars
1/90d Recent commits
1 devs/90d Recent contributors
3.2y Project age
active Last commit
Gotchas
No gotchas documented yet. Contribute one if you know a constraint we missed.
Labels
- Self-hosted
Self-hosted
You deploy and operate it yourself; there is no hosted option here.
- Managed option
Managed option
Available as a hosted/managed service in addition to self-hosting.
serving openai-compatible self-hosted bentoml inference