Q&A: Web Application Security Scanning with Netsparker

ID N0WHERE:155973
Type n0where
Reporter N0where
Modified 2017-02-02T19:57:20


Q&A with Huseyin Tufekcilerli,

the lead developer of Netsparker Desktop web application security scanner

More than 70% of all cyber breaches involve web applications, and almost 90% organizations believe their application security programs need to be improved. Web application security has risen to the top of most organizations’ concern list, and for good reason: web applications are very easy targets for cyber attackers.

What types of risks do web applications bring into the business and which of those risk Netsparker see as the most common ones?

The web applications of today are quite complex and dynamic compared to web applications of the old days. There are dozens of endpoints where users send data to and each of these endpoint is a potential attack surface for a malicious user. If not tested well, these endpoints can be used to manipulate the data and also extract the data from the target web application. The most common risks we see these days are stealing of user databases from various popular web websites via web application vulnerabilities such as SQL Injection .

How can enterprises protect data access through web applications and safeguard them against incursions via the web [According to Netsparker]?

The first step to start protecting your web applications from users with malicious intent is to use a software stack with up-to-date versions. Companies or open source software maintainers try to react as fast as they can to any security issues reported and patch their software with security updates. As a web application owner, you should know the whole stack your web application runs on and make sure you are receiving these critical software updates.

The next most important thing is to follow the best practices of the software development environment you are running on. I.e. if your web application is running SQL queries against a relational database, you must not create those query statements by concatenating strings along with the user input and start using the secure recommended way of performing queries against database (parameterized queries, prepared statements, etc.). Another important thing is to never trust the user data and always do the necessary sanitization. Last but not least, you should use tools to scan your website for vulnerabilities, such as a web application security scanner.

What are the most important things to keep in mind when testing web applications for security flaws with Netsparker ?

The most important thing to keep in mind is to make sure your web application is thoroughly covered by the Netsparker crawler. Without a good crawler coverage, portions of your web application will not be tested against security flaws by Netsparker. We are trying to keep Netsparker as frictionless as possible when performing scans against web applications, and also to make it a _ point and shoot _ web application security scanner but there are cases where your attention is needed. There are many dials and knobs on Netsparker to configure such as:

  • Form based authentication, to make sure the restricted sections of your web application which needs some sort of user credentials are crawled and scanned,
  • Include/exclude rule and scope configuration, to be able to guide the crawler which parts it is allowed to penetrate and not,
  • Link importing, to instruct the crawler about parts of the web application that is not linked on the public surface area of the application (i.e. if you have an administration area which is not linked from anywhere)

To be able to configure these settings, one should first know the web application very well, what are the common areas of the web application, is it doing any out-of-band AJAX calls to the backend, what is the format of the payload used when it sends data to the server (JSON? XML?), etc.

What are the biggest misconceptions when it comes to web application security testing [According to Netsparker] ?

Web application security scanners have improved quite a lot over the years, especially when compared to what was on the market five to ten years ago. Potential users of the scanners on the other hand, underestimate their power and do not really know the capabilities of the scanners. A website developer may think that all of his website’s resources are behind a login form, therefore the scanner has no chance to access that part of the website. But this is wrong, Netsparker has a very good success rate of filling the authentication forms and detecting any log outs.

Another misconception is that many website owners think that scanners are not able to automatically find vulnerabilities in websites built with modern technologies such as HTML5, REST APIs, Single Page Applications (SPA) etc. Today’s scanners do a bunch of operations behind the scenes to make sure the target web site is really covered and penetrated and most of them are able to easily crawl and scan such type of modern and custom built websites.

Why web application security scanners typically generate false positives?

The main reason behind false positives is that the patterns the scanners use to detect vulnerabilities, for example by matching a particular string in web server responses, are not well built. The web application security scanners which are designed poorly are mostly making very simple text matching in HTTP responses to determine if there is a vulnerability or not, and therefore reporting vulnerabilities which are not there, thus being inaccurate.

What steps Netsparker does to provide false positive free web application scanning?

When Netsparker detects a vulnerability, it tries to confirm it by exploiting it in a read only and safe way. If it exploits it, Netsparker also generates a proof of exploit. All this is done automatically with the proof-based vulnerability scanning technology . For example in case of a SQL Injection vulnerability, Netsparker will extract the software version of the target database backend, or in the case of an LFI (Local File Inclusion) vulnerability , Netsparker will retrieve a well known system file from the target web server. By giving such a proof, there is no doubt that the vulnerability does not exist.

Apart from guaranteeing false positive free web security scans, are there any other advantages in having an exploitation engine in Netsparker?

The most important role of having an exploitation engine in Netsparker is that it reveals the real impact of the vulnerability. Netsparker first identifies a possible vulnerability, confirms it by extracting data from the target system and also presents a few proofs that the vulnerability is real. Netsparker does not stop there. It gives the user the chance to further exploit that vulnerability on his website. At that point, the user can really see what could go wrong with this vulnerability in the hands of an evildoer, like extracting the users table of the backend database.

Why is Netsparker so dependent on .NET and why there’s no *NIX [Linux,UNIX] version of the package?

Netsparker is built on top of Microsoft .NET framework. Since the .NET framework is so dependent on Microsoft Windows operating system, it is quite a job to port it to Linux. But this has been changing recently with the open source movement in Microsoft. It is not there yet but there is an extensive amount of work done by Microsoft and open source contributors to make .NET run on non-Windows systems. As Netsparker, we currently do not have a road map of Netsparker on Linux but that is definitely something we might be considering some day.

How does Netsparker approach web authentication ( considering the diversity of authentication models )?

Netsparker supports three different authentication mechanisms. These are Form authentication, which is the most common, Basic, NTLM/Kerberos authentication and Client Certificate authentication. It was trivial to implement the latter two, but the Form authentication took us a few iterations to get it right. In its latest incarnation, we tried to keep as easy as possible to configure the form authentication. The user simply has to enter three pieces of data: the login form URL, username and password. Then Netsparker does its magic by locating the web form on target site, filling it with credentials and submitting it. While implementing this, we have aggressively tested it with real world login forms and had a success rate of 80-90%. We have also integrated a simple custom scripting environment for cases where the automatic form authentication may not work for your site.

What are the pros and cons of using Netsparker Cloud vs. Netsparker Desktop for web application security testing (or Why move to the Cloud) ?

At their core, Netsparker Desktop and Netsparker Cloud both have the same vulnerability scanning engine and both will detect same set of vulnerabilities. But they both have different roles in the arsenal of security professionals.

For example Netsparker Desktop is more like a single user tool and:

  • You can install it on a PC on your local network and scan your websites which are not publicly accessible.
  • Netsparker Desktop has several exploitation utilities and controlled scan features which you can use to do fine grain tests.
  • The manual crawling feature of Netsparker Desktop allows the user to use the scanner as a proxy for your web browser and you can manually crawl your website.

Netsparker Cloud is a multi-user enterprise level solution with built-in workflow tools. With Netsparker Cloud:

  • There is no need for any hardware investment from your side.
  • It has a flexible licensing model and you pay as you go.
  • It is web based, therefore if you are a mac or linux shop and don’t have Windows computers, you can scan your websites just fine.
  • If you need to scale and perform lots of concurrent scans, Netsparker Cloud is the way to go.
  • The built-in workflow tools are specifically designed to help organizations ensure the long term security of their web applications.

Partial and incremental scans are particularity interesting. Does it means that Netsparker see itself more as a part of SDLC, rather than penetration tester’s tool?

Since the early days of Netsparker, we have positioned it to be as a vital part of the Software Development Life Cycle. It fits so well and it would be a big mistake if we haven’t positioned Netsparker like that. Think about it; you have a web application project that several developers are continuously pushing code to. You have a goal of performing continuous deployments to your web server, therefore it is quite natural to put Netsparker as a step right before the deployment step. By doing so you can have a quick security check of the latest snapshot of your web application.

How does Netsparker compare to other Web Application Security scanners on the market?

Number one feedback that we are getting from our users, who also have experiences with other scanner products is the better usability of Netsparker. Since day one, usability is one of our main concerns while designing the product. We are trying to keep the learning curve of Netsparker as shallow as possible. By its very nature, web application security scanners tend to have lots of configuration options. The real problem here is to having the best default values of any of these options to cover most of the web applications.

In terms of vulnerability scanning technology, we are happy to say that for the last few years Netsparker has led the pack when independent web scanner comparisons were done .

Considering how many big companies are turning more and more to open source solutions, including Microsoft, why did Netsparker decide to kill CE version?

The CE version of Netsparker was a real success. It allowed us to reach many security professional that we would have not had the chance to reach otherwise. And we are grateful to these people because their feedback helped us shape Netsparker and determine a correct direction to take.

Though it also gave users a false sense of security. The Community Edition’s scanning engine was limited and it was not scanning for all the different web application vulnerabilities out there.So when people used it and it did not find any vulnerabilities on their website, they were misled to believe that their website is secure. Though web application security is not just about SQL Injections, there are many other web application vulnerabilities that their website could be vulnerable to, and that could be exploited by malicious attackers.

Though it is not all doom and gloom. To counteract for this we are now giving away free Netsparker Cloud accounts to developers of open source web applications, so if you are one please get in touch.

What can we expect from Netsparker in the future?

The web is evolving even faster than before with the new release of a library or a new protocol every other day. And also, the web applications of current day are getting quite complex. At Netsparker we are trying to do best to support these new technologies as soon as they appear. We are also trying to enhance our scanner to be able to perform faster web vulnerability scans which reports useful information on shorter durations. While doing these, we are going to make sure that Netsparker is still the easiest to use product on market in terms of usability.

Q&A: Web Application Security Scanning with Netsparker