Trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
Trafilatura scores 6.7/10 (Solid) in Crawl & Search.
Language Python License Apache-2.0 Stars 6.2k★ Type library Self-host Yes Repo adbar/trafilatura Homepage https://trafilatura.readthedocs.io
Quality
Solid 6.7
Adoption 45
Activity 64
Maturity 100
Community 41
Capability 88
Show the math
Overall 6.7/10
value = 6.7/10
score = value
Adoption 5/10
value = 44.9/100
score = 1 + 9 * value / 100
Activity 6.8/10
value = 63.9/100
score = 1 + 9 * value / 100
Maturity 10/10
value = 100/100
score = 1 + 9 * value / 100
Community 4.7/10
value = 40.8/100
score = 1 + 9 * value / 100
Capability 8.9/10
value = 88.3/100
score = 1 + 9 * value / 100
Key metrics
6.2k★ GitHub stars
27/90d Recent commits
9 devs/90d Recent contributors
F1 0.883 Web content extraction F1 (best overall mean) Bevendorff et al. 2023 (ACM SIGIR) / contextractor analysis of the same benchmark: Trafilatura had the best overall mean F1 = 0.883 across 8 evaluation datasets (best all-around, vs Readability's higher article-only median). Source: https://dl.acm.org/doi/pdf/10.1145/3539618.3591920 ; https://www.contextractor.com/trafilatura-vs-readability-vs-newspaper/
7.2y Project age
active Last commit
Gotchas
No gotchas documented yet. Contribute one if you know a constraint we missed.
Labels
- Self-hosted
Self-hosted
You deploy and operate it yourself; there is no hosted option here.
extraction boilerplate-removal python text metadata