DATABASE RESOURCES PRICING ABOUT US

{"id": "KITPLOIT:1538150805868015968", "vendorId": null, "type": "kitploit", "bulletinFamily": "tools", "title": "Evine - Interactive CLI Web Crawler", "description": "[](<https://1.bp.blogspot.com/-jfhKR5_hRuc/Xy3zhGv5kZI/AAAAAAAATWk/jz7MXNpgD-IUHMGO-OYE3dgsz1-yB4OYACNcBGAsYHQ/s1600/evine_4_screen.png>)\n\n \nEvine is a simple, fast, and interactive web [crawler](<https://www.kitploit.com/search/label/Crawler> \"crawler\" ) and web scraper written in Golang. Evine is useful for a wide range of purposes such as [metadata](<https://www.kitploit.com/search/label/Metadata> \"metadata\" ) and data extraction, data mining, [reconnaissance](<https://www.kitploit.com/search/label/Reconnaissance> \"reconnaissance\" ) and testing. \n \n\n\n[](<https://asciinema.org/a/351624>)\n\n \nFollow the project on [Twitter](<https://twitter.com/EvineProject> \"Twitter\" ). \n \n**Install** \n \n**From Binary** \nPre-build [binary releases](<https://github.com/saeeddhqan/evine/releases> \"binary releases\" ) are also available. \n \n**From source** \n\n \n \n go get github.com/saeeddhqan/evine\n \"$GOPATH/bin/evine\" -h\n\n \n**From GitHub** \n\n \n \n git clone https://github.com/saeeddhqan/evine.git\n cd evine\n go build .\n mv evine /usr/local/bin\n evine --help\n\nNote: golang 1.13.x required. \n \n**Commands &amp; Usage** \nKeybinding | Description \n---|--- \nEnter | Run crawler (from URL view) \nEnter | Display response (from Keys and Regex views) \nTab | Next view \nCtrl+Space | Run crawler \nCtrl+S | Save response \nCtrl+Z | Quit \nCtrl+R | Restore to default values (from Options and Headers views) \nCtrl+Q | Close response save view (from Save view) \n \n \n evine -h\n\nIt will displays help for the tool: \nflag | Description | Example \n---|---|--- \n-url | URL to crawl for | evine -url toscrape.com \n-url-exclude string | Exclude URLs maching with this regex (default \".*\") | evine -url-exclude ?id= \n-domain-exclude string | Exclude in-scope domains to crawl. Separate with comma. default=root domain | evine -domain-exclude host1.tld,host2.tld \n-code-exclude string | Exclude HTTP status code with these codes. Separate whit '|' (default \".*\") | evine -code-exclude 200,201 \n-delay int | Sleep between each request(Millisecond) | evine -delay 300 \n-depth | Scraper depth search level (default 1) | evine -depth 2 \n-thread int | The number of concurrent goroutines for resolving (default 5) | evine -thread 10 \n-header | HTTP Header for each request(It should to separated fields by \\n). | evine -header KEY: VALUE\\nKEY1: VALUE1 \n-proxy string | Proxy by scheme://ip:port | evine -proxy [http://1.1.1.1:8080](<http://1.1.1.1:8080/> \"http://1.1.1.1:8080\" ) \n-scheme string | Set the scheme for the requests (default \"https\") | evine -scheme http \n-timeout int | Seconds to wait before timing out (default 10) | evine -timeout 15 \n-keys string | What do you want? write here(email,url,query_urls,all_urls,phone,media,css,script,cdn,comment,dns,network,all, or a file extension) | evine -keys urls,pdf,txt \n-regex string | Search the Regular Expression on the page contents | evine -regex 'User.+' \n-max-regex int | Max result of regex search for regex field (default 1000) | evine -max-regex -1 \n-robots | Scrape [robots.txt](<https://www.kitploit.com/search/label/Robots.txt> \"robots.txt\" ) for URLs and using them as seeds | evine -robots \n-sitemap | Scrape sitemap.xml for URLs and using them as seeds | evine -sitemap \n \n**VIEWS** \n\n\n * URL: In this view, you should enter the URL string.\n * Options: This view is for setting options.\n * Headers: This view is for setting the HTTP Headers.\n * Keys: This view is used after the [crawling](<https://www.kitploit.com/search/label/Crawling> \"crawling\" ) web. It will be used to extract the data(docs, URLs, etc) from the web pages that have been crawled.\n * Regex: This view is useful to search the Regexes in web pages that have been crawled. Write your Regex in this view and press Enter.\n * Response: All of the results write in this view\n * Search: This view is used to search the Regexes in the Response content.\n \n**TODO** \n\n\n * Archive crawler as seeds\n * JSON output\n \n**Bugs or Suggestions** \nBugs or suggestions? Create an [issue](<https://github.com/saeeddhqan/evine/issues> \"issue\" ). \nevine is heavily inspired by [wuzz](<https://github.com/asciimoo/wuzz> \"wuzz\" ). \n \n \n\n\n**[Download Evine](<https://github.com/saeeddhqan/evine> \"Download Evine\" )**\n", "published": "2020-08-09T12:30:00", "modified": "2020-08-09T12:30:03", "cvss": {"score": 0.0, "vector": "NONE"}, "cvss2": {}, "cvss3": {}, "href": "http://www.kitploit.com/2020/08/evine-interactive-cli-web-crawler.html", "reporter": "KitPloit", "references": ["https://github.com/saeeddhqan/evine/issues", "https://github.com/saeeddhqan/evine/releases", "https://github.com/saeeddhqan/evine", "https://github.com/asciimoo/wuzz"], "cvelist": [], "immutableFields": [], "lastseen": "2022-04-07T12:02:43", "viewCount": 67, "enchantments": {"dependencies": {}, "score": {"value": 0.5, "vector": "NONE"}, "backreferences": {}, "exploitation": null, "vulnersScore": 0.5}, "_state": {"dependencies": 1659880087, "score": 1659828793}, "_internal": {"score_hash": "4455d054d7dff1cb3caea50052db7ce3"}, "toolHref": "https://github.com/saeeddhqan/evine"}

{}