Nmap NSE net: http-robots.txt

Checks for disallowed entries in '/robots.txt' on a web server. The higher the verbosity or debug level, the more disallowed entries are shown. SYNTAX: http.pipeline: If set, it represents the number of HTTP requests that'll be pipelined ie, sent in a single request. This can be set low to make...