by discourselab
AI-powered web scraping CLI. Describe what you want, get a production-ready Scrapy spider. Write once, reuse forever.
# Add to your Claude Code skills
git clone https://github.com/discourselab/scrapai-cliYou: "Add https://bbc.co.uk to my news project"
Minutes later you have a tested, production-ready scraper stored in a database. No Python, no CSS selectors, no Scrapy knowledge. The AI agent analyzes the site, writes extraction rules, verifies quality, and saves a reusable config. Run it tomorrow or next year. Same command, no AI costs.
Built by DiscourseLab. Used in production across 500+ websites.
<p align="center"> <img src="demo.svg" alt="ScrapAI Demo" width="800"> </p>No comments yet. Be the first to share your thoughts!
Good fit:
Not a good fit:
See COMPARISON.md for a detailed comparison with Scrapling and crawl4ai.
We needed data for our work. Hundreds of websites, scraped regularly, structured consistently. We got sick of building and maintaining fleets of scrapers.
There are great crawling frameworks out there. Scrapy, crawl4ai, and Scrapling are our favourites, and ScrapAI is built on top of Scrapy. But even with great frameworks, you hi...