Scrapy-splash is a library which provides Scrapy and JavaScript integration. In affected versions users who use HttpAuthMiddleware
(i.e. the http_user
and http_pass
spider attributes) for Splash authentication will have any non-Splash request expose your credentials to the request target. This includes robots.txt
requests sent by Scrapy when the ROBOTSTXT_OBEY
setting is set to True
. Upgrade to scrapy-splash 0.8.0 and use the new SPLASH_USER
and SPLASH_PASS
settings instead to set your Splash authentication credentials safely. If you cannot upgrade, set your Splash request credentials on a per-request basis, using the splash_headers
request parameter, instead of defining them globally using the HttpAuthMiddleware
. Alternatively, make sure all your requests go through Splash. That includes disabling the robots.txt middleware.
CPE | Name | Operator | Version |
---|---|---|---|
scrapy-splash | eq | 0.6 | |
scrapy-splash | eq | 0.3 | |
scrapy-splash | eq | 0.7.1 | |
scrapy-splash | eq | 0.7 | |
scrapy-splash | eq | 0.1.1 | |
scrapy-splash | eq | 0.6.1 | |
scrapy-splash | eq | 0.7.2 | |
scrapy-splash | eq | 0.4 | |
scrapy-splash | eq | 0.2 | |
scrapy-splash | eq | 0.1 |