Scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Scrapy scores 8.4/10 (Strong) in Crawl & Search.
Language Python License BSD-3-Clause Stars 62.9k★ Type library Self-host Yes Repo scrapy/scrapy Homepage https://scrapy.org
Quality
Strong 8.4
Adoption 70
Activity 82
Maturity 100
Community 78
Capability 100
Show the math
Overall 8.4/10
value = 8.4/10
score = value
Adoption 7.3/10
value = 70/100
score = 1 + 9 * value / 100
Activity 8.4/10
value = 82/100
score = 1 + 9 * value / 100
Maturity 10/10
value = 100/100
score = 1 + 9 * value / 100
Community 8/10
value = 78.3/100
score = 1 + 9 * value / 100
Capability 10/10
value = 100/100
score = 1 + 9 * value / 100
Key metrics
62.9k★ GitHub stars
145/90d Recent commits
28 devs/90d Recent contributors
62.3k stars GitHub stars (popularity proxy) Popularity proxy (not a capability benchmark): Scrapy GitHub repo shows 62.3k stars - the most-starred general-purpose Python web crawling/scraping framework. Source: https://github.com/scrapy/scrapy
16.4y Project age
active Last commit
Gotchas
No gotchas documented yet. Contribute one if you know a constraint we missed.
Labels
- Self-hosted
Self-hosted
You deploy and operate it yourself; there is no hosted option here.
crawler scraper python framework