node-crawler
https://github.com/bda-research/node-crawler
JavaScript
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
JavaScript not yet supported0 Subscribers
Add a CodeTriage badge to node-crawler
Help out
- Issues
- Returning data from callback
- Update documentation to reflect that node-crawler is based on request
- Integrate with ProxyCrawl crawler
- pause / resume?
- Documentation unclear; CRAWLER Error Error: incorrect header check when fetching
- jsdom.env is not defined
- Add ciphers into request options
- Passing a crt with the proxy
- Catch an error: "invalid markup"
- A way to set maximum number of links to follow
- Docs
- JavaScript not yet supported