Scrapy Python Crawler: Crawlpy

Python web spider/crawler based on scrapy with support for POST/GET login, variable level of recursions/depth and optionally save to disk. Requirements python 2.7 lxml pip pip install Scrapy Features POST/GET Login prior crawling Can handle logins that requires dynamic CSRF token Variable level o...