ExLlamaV2
A fast inference library for running LLMs locally on modern consumer-class GPUs
ExLlamaV2 scores 4.4/10 (Emerging) in Inference & Serving.
Language Python License MIT Stars 4.6k★ Type library Self-host Yes Repo turboderp-org/exllamav2 Homepage https://github.com/turboderp-org/exllamav2
Quality
Emerging 4.4
Adoption 42
Activity 11
Maturity 87
Community 39
Capability 50
Show the math
Overall 4.4/10
value = 4.4/10
score = value
Adoption 4.7/10
value = 41.5/100
score = 1 + 9 * value / 100
Activity 2/10
value = 11/100
score = 1 + 9 * value / 100
Maturity 8.8/10
value = 87.1/100
score = 1 + 9 * value / 100
Community 4.5/10
value = 39.2/100
score = 1 + 9 * value / 100
Capability 5.5/10
value = 50/100
score = 1 + 9 * value / 100
Key metrics
4.6k★ GitHub stars
0/90d Recent commits
0 devs/90d Recent contributors
2.8y Project age
4mo ago Last commit
Gotchas
No gotchas documented yet. Contribute one if you know a constraint we missed.
Labels
- Self-hosted
Self-hosted
You deploy and operate it yourself; there is no hosted option here.
quantization exl2 gpu fast local