scrapy
https://github.com/scrapy/scrapy
Python
Scrapy, a fast high-level web crawling & scraping framework for Python.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported164 Subscribers
Add a CodeTriage badge to scrapy
Help out
- Issues
- Improve statscollector.py along with test_stats.py and test_link.py
- Addressing issues identified in issue #5944
- Add Content-Encoding header in response flag
- DownloaderMiddleware should be able to return a request generator from process_request
- Adding support for windows absolute pathlib.Path objects in FeedExporter
- Integration with DI frameworks
- Change redirect text from Response.request docs
- Docs involving Request.cookies are inaccurate
- _store_in_thread must not receive a closed file when using postprocessing plugins
- test_s3_export fails on Windows/asyncio
- Docs
- Python not yet supported