Up to 30X Faster PHP Malware Scans with Wordfence CLI 4.0.1

Most of our customers scan a single site or a small number of sites for PHP malware using the Wordfence Plugin, and they coordinate scanning across multiple sites with Wordfence Central. If you are responsible for securing a large hosting provider network as part of an operations or security team, you know that scanning terabytes of data and millions of PHP files can be a resource intensive task.

Today we're announcing Wordfence CLI 4.0.1 which includes a massive scan speed-up of 6X to 30X depending on your workload. This speed increase gives enterprise teams the ability to perform scans for PHP malware on huge volumes of data with the highest available performance. We can confidently say that Wordfence CLI is now the fastest and most thorough PHP malware scanner in the world.

How Did We Achieve Up To 30X Speed Increase?

At Wordfence, we use a pattern matching system called regular expressions to recognize malware. Our plugin is installed on over 5 million websites and this provides us with a huge surface area to collect malicious samples from our users performing malware scans. We also provide a site-cleaning service via our Wordfence Care and Wordfence Response services, and our analysts are able to collect the newest malware samples from the work they do remediating infected sites.

The PHP malware samples we collect are all ingested into our internal threat intelligence platform where we identify emerging threats and develop regular expressions to recognize those threats. These regular expressions (regexes) are deployed to the Wordfence Plugin where the plugin's malware scanner uses these "scan signatures" to detect the newest malware targeting WordPress. The regexes are also deployed to Wordfence CLI which uses them to detect PHP malware from the command line.

Approximately two years ago we recognized that an Intel produced product that was open-source at the time, called Hyperscan, could provide a massive speed-up in scan times. Hyperscan supported a subset of the PCRE (Perl compatible regular expressions) syntax we use, so we had to do a significant amount of work to port our PHP malware signatures over to be Hyperscan compatible. We completed that work about a year and a half ago.

Unfortunately Intel announced that they were changing the license on Hyperscan starting with version 5.5 to essentially make it closed source. Hyperscan also has the limitation that it only runs on Intel CPUs.

Thankfully a team of developers forked the open source version of Hyperscan to create Vectorscan which will continue to develop and release an open source high performance regex scan engine. Additionally, Vectorscan runs on x86 and ARM CPU architectures. Huge kudos to the Vectorscan team for this achievement. Wordfence CLI can use Vectorscan or Hyperscan, depending on which you have installed on your server.

Both Vectorscan and the open source version of Hyperscan are available as packages on most popular Linux systems and we include instructions on how to install these dependencies to get the incredible high performance malware scanning that Wordfence CLI provides. If for some reason you're unable to install Vectorscan or Hyperscan, you can revert to using the PCRE engine, but unfortunately you won't benefit from a huge performance boost.

Your Server And Network Configuration Matters

With great performance comes one or two caveats - mainly that your network stack may not be able to keep up. Wordfence CLI with Vectorscan or Hyperscan is blazingly fast and with the speed increase version 4.0.1 provides, it is no longer the bottleneck in many scans.

We've found that scanning on a server with local SSD drives provides excellent performance. Scanning a server over a network via an NFS mount is perhaps the worst case scenario when it comes to scan bottlenecks. NFS performance can be quite awful and does not do a great job of keeping up with Wordfence CLI performance. So we recommend as far as possible to run your scans on the server where the data resides, allowing your fast CPU to access fast hard-drives via the PCIe bus which is an ideal performance scenario.

Note that if you have a fleet of say 100 servers all hosting PHP websites, we suggest parallelizing scan operations by having a scheduled job that launches Wordfence CLI on each individual server where it can run locally and access local disk. This gives you the benefit of high performance and parallelization.

If you'd like to try out Wordfence CLI on your local machine, I've found that I get excellent performance running Ubuntu Linux in a virtual machine on my M1 Macbook, on a large set of test PHP files. Even running within the hypervisor, Wordfence CLI is able to access local disk, and is able to use Vectorscan on the M1 CPU architecture, and provides an unbelievable speedup when you compare it to the PCRE regular expression engine.

How To Install Wordfence CLI And Invoke High Speed Scans

You can find detailed instructions to install Wordfence CLI on our product page. Some of our packages will automatically install Vectorscan or Hyperscan for you, but in some cases you'll need to manually install libhyperscan5 or libvectorscan5. On Ubuntu you can do this with the command:

apt install libhyperscan5(or libvectorscan5)

To invoke a scan using Vectorscan/Hyperscan with Wordfence CLI you can use a command like this to scan /usr/local/wordpress/ and all files and directories under that path, recursively:

wordfence malware-scan -­-progress -­-match-engine vectorscan /usr/local/wordpress/

The new -­-match-engine parameter is where you specify "vectorscan". Note that by default Wordfence CLI still uses the PCRE engine, so you must specify this parameter to activate fast scanning.

Performance Matters at Scale

Wordfence CLI is designed for medium to large hosting providers who are hosting WordPress websites and other PHP applications. With this massive performance increase, we're hoping to make it feasible to increase your scan frequency while reducing the impact of any scan on your operations. Wordfence CLI is free and open source, and a paid license will give you access to our newest PHP malware scan signatures.

Thanks for choosing Wordfence!

Mark Maunder - Chief Technology Officer at Defiant Inc.

The post Up to 30X Faster PHP Malware Scans with Wordfence CLI 4.0.1 appeared first on Wordfence.