Xinference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
Xinference scores 7.1/10 (Strong) in Inference & Serving.
Language Python License Apache-2.0 Stars 9.4k★ Type service Self-host Yes Repo xorbitsai/inference Homepage https://inference.readthedocs.io
Quality
Strong 7.1
Adoption 49
Activity 82
Maturity 96
Community 49
Capability 50
Show the math
Overall 7.1/10
value = 7.1/10
score = value
Adoption 5.4/10
value = 49.3/100
score = 1 + 9 * value / 100
Activity 8.3/10
value = 81.5/100
score = 1 + 9 * value / 100
Maturity 9.7/10
value = 96.3/100
score = 1 + 9 * value / 100
Community 5.4/10
value = 48.9/100
score = 1 + 9 * value / 100
Capability 5.5/10
value = 50/100
score = 1 + 9 * value / 100
Key metrics
9.4k★ GitHub stars
239/90d Recent commits
16 devs/90d Recent contributors
3.1y Project age
active Last commit
Gotchas
No gotchas documented yet. Contribute one if you know a constraint we missed.
Labels
- Self-hosted
Self-hosted
You deploy and operate it yourself; there is no hosted option here.
- Managed option
Managed option
Available as a hosted/managed service in addition to self-hosting.
distributed embeddings rerank multimodal openai-compatible