On WatchGuard Firebox and XTM appliances, an unauthenticated user can execute arbitrary code, aka FBX-22786. This vulnerability impacts Fireware OS before 12.7.2_U2, 12.x before 12.1.3_U8, and 12.2.x through 12.5.x before 12.5.9_U2.

Recent assessments:

jbaines-r7 at March 25, 2022 4:05pm UTC reported:

Edit on March 28, 2022: On March 27, 2022 a proof of concept was posted to GitHub. I’m leaving the original analysis below and will add a reply comment with some additional information.

Introduction

On March 17, 2022, GreyNoise published WatchGuard CVE-2022-26318 RCE Detection, IOCs, and Prevention for Defenders. The writeup details GreyNoise’s observation of in-the-wild exploitation of CVE-2022-26318. This is particularly interesting for a few reasons:

1. The affected WatchGuard products, FireBox and XTM, are firewall/VPN solutions. Making them obvious and delicious targets for attackers.

2. There are a couple hundred thousand of these on Shodan.

3. There is no public exploit for CVE-2022-26318. GreyNoise’s limited description is, by far, the most detailed publicly available description of what an exploit for this vulnerability might look like.

4. CVE-2022-26318 appears to be related to Cyclops Blinked, Sandworm’s VPNFilter 2.0 which was recently unmasked by CISA, NSA, NCSC UK, and the FBI. Neither WatchGuard nor the various government agencies explicitly say the vulnerability is related, but it was the only unauthenticated, remote code execution issue patched at the same time as the Cyclops Blinked publication so it seems like a reasonable assumption.

The lack of proof of concept, or really any useful information about the vulnerability, is unfortunate. The CVE description provides no information about the attack vector or even what the vulnerability class might be. NVD currently has the CWE assigned as “NVD-CWE-noinfo” or “Insufficient Information”:

Patching is not always an option. Being able to mitigate an attack using a firewall is sometimes the only solution. Being able to use network signatures to block exploitation attempts, while not ideal, is sometimes all a defender has. So when NSCS UK only publishes YARA rules (which is useless on a system that blocks access to the filesystem), CISA publishes no mitigation guidance, and WatchGuard only publishes a log parser, defenders are left in a tough spot when attempting to understand their exposure and mitigation options.

Thankfully GreyNoise shared some information. But there are still questions. Is port 4117 the only affected port? Or is the default administrative webui affected as well (hint: I think it is)? Is /agent/login the only vulnerable URI (hint: it isn’t). But there is no official information either way, so who knows.

I figured I could get the answers myself. I could quickly diff the changes between XTMv 12.1.3u7 and 12.1.3u8 and bingo bango you got an exploit. Dear reader. I’ve failed you. I looked at the diffs and I haven’t the faintest idea what this vulnerability is. So what is this write up? Well, just because I failed doesn’t mean you will. I’m going to lay out what I saw and maybe you will have better luck than me.

Getting Started

To start, we need binaries to diff. I snagged the XTMv VMWare Operating System Files here. The most current version, at the time of writing, is 12.1.3 Update 8, which is the first patched version. You can download the last unpatched version, 12.1.3 Update 7, by slightly tweaking the URL for 12.1.3 Update 8.

Once you have the XTMv ova file, go ahead and import it and install it. You don’t need any type of license or key to get a base installation going. From there, snag the underlying .vmdk file and extract the file system using 7zip (e.g. 7z x ./xtmv_12_1_3_U8-disk1.vmdk). You should then get a handful of mountable .img files that contain the XTMv system.

What Now?

Having the entire filesystem is nice. But where do we start? GreyNoise’s writeup says they were seeing the vulnerability being exploited over the HTTP endpoint /agent/login on port 4117. Well, hunting in the filesystem we can find the nginx configuration for the port 4117 server:

albinolobster@ubuntu:/media/albinolobster/5d0ede31-24e7-49b6-b114-abed98c4830f$ cat ./etc/nginx/http-server-wgagent
server {
    listen              4117 ssl;
    listen              [::]:4117 ssl;

    include             fastcgi_params;
    fastcgi_param       SCRIPT_NAME     $fastcgi_script_name;
    fastcgi_param       SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param       WG_SSL_SERVER_CERT /var/run/nginx/server.pem;
    fastcgi_request_buffering off;

    if ($request_method !~ ^(GET|HEAD|POST)$) { return 444; }

    location /agent/ {
        fastcgi_pass    unix:/usr/share/web/upload/tmp/wgagent;
        # /agent/file_action can take a while, e.g. backup
        fastcgi_read_timeout    10m;
    }
    location /login {   # no trailing slash
        fastcgi_pass    unix:/usr/share/web/upload/tmp/wgagent;
    }
    location /logout {
        fastcgi_pass    unix:/usr/share/web/upload/tmp/wgagent;
    }
    location /ping {    # no trailing slash
        fastcgi_pass    unix:/usr/share/web/upload/tmp/wgagent;
    }
    location /cluster/ {
        fastcgi_pass    unix:/usr/share/web/upload/tmp/wgagent;
    }
}

This points to the wgagent binary being the binary of interest. It can be found in /usr/bin/.

What Should the Exploit Look Like?

Based on my understanding of GreyNoise’s writeup and the logic contained in wgagent the exploit occurs during authentication. Therefore the exploit is likely to look like a variation on a valid login request like the following:

import requests
import zlib
import binascii
from urllib3.exceptions import InsecureRequestWarning

# Suppress only the single warning from urllib3 needed.
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)


url = 'https://10.0.0.11:4117/agent/login'
login_data = b'<methodCall><methodName>login</methodName><params><param><value><struct><member><name>password</name><value><string>readwrite</string></value></member><member><name>user</name><value><string>admin</string></value></member><member><name>domain</name><value><string>Firebox-DB</string></value></member><member><name>uitype</name><value><string>2</string></value></member></struct></value></param></params></methodCall>'

extra_headers = { 'Content-Encoding': 'gzip' }
comp = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -15)
compressed_login_data = b'\x1f\x8b\x08\x07\x41\x41\x41\x41\x41\x41' 
compressed_login_data += comp.compress(login_data)
compressed_login_data += comp.flush()

print('Uncompressed data length: %u' % len(login_data))
print('Compressed data length: %u' % len(compressed_login_data))

x = requests.post(url, data = compressed_login_data, headers = extra_headers, verify=False)

print(x.status_code)
print(x.text)

There are a couple of interesting tidbits about this though:

1. Compression isn’t required for a login request. wgagent only tries to remove the compression if Content-Encoding: gzip is included (shocking, I know). But GreyNoise noted that the exploits they saw in the wild uses compression… So maybe that’s required for exploitation? Maybe not, who knows!

2. This request also works on the default administrative webui port (8080). Which implies, to me, that 4117 is not the only vulnerable endpoint. Considering that the default webui is far more likely to be exposed to the internet, it’s a pretty important thing to keep in mind. E.g. You aren’t safe just because you blocked port 4117. Maybe you need to block 8080 as well. Who knows though! (Probably CISA and WatchGuard).

Regardless, the above code hits, at least, part of the desired code path. So now we need to hunt for the vulnerable code. Hopefully aided by binary diffing!

Ch-Ch-Ch-Ch-Changes

There are a large number of differences between 12.1.3u7 and 12.1.3u8. Unfortunately (or fortunately), WatchGuard fixed five other CVEs and a number of other issues with the update 8 release. Looking at the changes, I had a very hard time determining if some of them were because “Well, we’re in here so might as well tweak this” or actually related to a vulnerability.

Unfortunately, I don’t currently have access to diaphora. This task probably would have been easier with it, but radiff2 + Ghidra is tolerable. The problem is, I couldn’t pinpoint any one thing that looked obviously exploitable… or even “maybe this is exploitable”. The changes seemed to fit into two categories:

Perplexing. Why was this here in the first place? Why did WatchGuard remove it now? A great example, mentioned below, is the !ENTITY logic.
Unnecessary. Why did zlib get updated? Why was password hashing standardized?

The “scary” part is both categories just show my ignorance and, worse, lack of imagination. About 90% of my vulnerability analysis skill is a weird mix of intuition+experience. And, for whatever reason, this diff has me feeling like Gandalf in Moria.

So, I can’t provide useful context to any of this, but I can point out a couple of changes that occurred along the likely exploitation path. Maybe that will spark something in someone else’s brain.

Removal of !ENTITY logic

In the following screenshot you’ll see the unpatched version of wgagent previously search for the string “!ENTITY” in the HTTP POST payload. If found, it walks backwards through the HTTP payload updating the contents to use the local locale for… reasons. Note that this is done before decompression. Odd. But not obviously exploitable (to me).

Update of zlib

WatchGuard XTMv was using a fairly old version of zlib to inflate the compressed HTTP payload. The zlib version being used in the unpatched version, 1.2.8, just so happens to be the same version analyzed by Trail of Bits and TrustInSoft for Mozilla back in 2016. The findings were very theoretical as far as actual exploitation goes, and, to my knowledge, zlib hasn’t had any CVE since that report.

But WatchGuard upgraded to zlib 1.2.11 with this update. Did they need to do that in order to fix this vulnerability? I doubt it. But worth noting.

Add “struct” to the state machine

wgagent uses libxml2 to parse the XML payload. But it does so, in part, by using the push parser which allows them to do some custom and stateful parsing. I very much assume the vulnerability lies in one of these functions. But the only major change was the addition of extraction and cleanup of the “<struct>``</struct>” tags (cleanup not pictured, and see the payload above to see how <struct> is used).

I cannot actually link this logic to anything useful. The only notable thing is that the XML payload now must have the struct tags, whereas before it was optional. Meaning, in 12.1.3u7 a request without the struct tags is accepted, but in 12.1.3u8 the request is rejected. But, again, I can’t link the added logic to any vulnerability or faulty logic… but this addition does suggest something?. Probably.

Update to standardize password hashing

After the XML parsing, the contents are obviously used for something. In this case, authentication. One change in the authentication world was the change highlighted below. FUN_004153e2 previously converted the provided user’s password into a UTF-16 representation and then MD4 hashed it. This function was deprecated in favor of a new and slightly different version of this functionality in libwgcrypto.so.

Once again, I don’t see anything obviously wrong in FUN_004153e2. Certainly, it makes sense to use a standard implementation in a single library but why make the change in this update?

There are a slew of other changes, but mostly they are related to the memory corruption issues in the upgrade process (I believe). And of course, I was only looking at wgagent and associated libraries. The reality is that the vulnerability could exist elsewhere. But if you are going down this journey yourself, maybe it will help.

Final Thoughts

One thing that should be pointed out about the Cyclops Blink malware, is that the NCSC UK write up only seems to address a PowerPC variant. However, the virtualized version that I tested (XTMv) is x86 based. Does that mean NCSC UK thinks only PowerPC versions of WatchGuard appliances are affected? Or maybe Sandworm only cared about PowerPC? Or NCSC UK only saw PowerPC versions in the wild? It’s these types of questions that a well thought out report would have addressed.

Finally, as I mentioned in a Twitter thread with @wvuuuuuuuuuuuuu and @_darrenmartyn, the GreyNoise observed exploit is sort of mystifying. Why would an attacker that had achieved RCE on a system exfiltrate an entire configuration file? Especially, when the exfiltrated file isn’t the one with usernames and password hashes? Exfiltration via TFTP is just an odd choice. And I’m not sure what to make of the exploit name (test.py). All of that would suggest an unsophisticated attacker to me… but then they’re using a vulnerability I can’t figure out. So who’s the unsophisticated one?

Assessed Attacker Value: 5
Assessed Attacker Value: 5Assessed Attacker Value: 3