Joyent SmartOS Hyprlofs FS IOCTL Add Entries 32-bit File System Denial of Service Vulnerability(CVE-2016-9040)

Summary

An exploitable denial of service exists in the the Joylent SmartOS OS 20161110T013148Z Hyprlofs file system. The vulnerability is present in the Ioctl system call with the command HYPRLOFSADDENTRIES when used with a 32 bit model. An attacker can cause a buffer to be allocated and never freed. When repeatedly exploit this will result in memory exhaustion, resulting in a full system denial of service.

Tested Versions

Joyent SmartOS 20161110T013148Z

Product URLs

https://www.joyent.com/smartos

CVSSv3 Score

6.2 - CVSS:3.0/AV:L/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H

CWE

CWE-400: Uncontrolled Resource Consumption (‘Resource Exhaustion’)

Details

Joyent SmartOS is an operating system deployed by Joyent to be used as a hypervisor like solution meaning virtual machines will run on top of the system itself. SmartOS is unique in the fact that it is based on a fork of Opensolaris. This leaves many vulnerabilities in the kernel due to the fact that it is not as actively developed as other operating systems. Hyprlofs is a file system specifically designed for SmartOS which allows the creation of new virtual file systems quickly and easily. This was developed and designed to help make their product, Manta, possible.
Most of the controls for Hyprlofs go through the Ioctl calls. An Ioctl is a control function that operates on various streams in this case a file descriptor to the file system. Looking further into that code we can spot the vulnerability. The beginning of the function is shown below.

    illumos-joyent-master/usr/src/uts/common/fs/hyprlofs/hyprlofs_vnops.c

            static int
134         hyprlofs_ioctl(vnode_t *vp, int cmd, intptr_t data, int flag,
                cred_t *cr, int *rvalp, caller_context_t *ct)
            {
                ...

                if (cmd == HYPRLOFS_ADD_ENTRIES || cmd == HYPRLOFS_RM_ENTRIES) {
                    ...

225                 e32 = kmem_alloc(len, KM_SLEEP);
                    if (copyin((void *)(unsigned long)(ebuf32.hle_entries),
                        e32, len)) {
                        kmem_free(e32, len);
                        return (EFAULT);
                    }

                    for (i = 0; i < cnt; i++) {
                        if (e32[i].hle_nlen == 0 ||
                            e32[i].hle_nlen > MAXPATHLEN)
235                         return (EINVAL);

                        ...

                            if (e32[i].hle_plen == 0 ||
                                e32[i].hle_plen > MAXPATHLEN)
248                             return (EINVAL);

The code at [1] shows the allocation of a buffer with a partially controlled size. At the code marked [2], we see that this function can return without freeing the allocated buffer. If this IOCTL is repeatedly called this will lead to a denial of service.

Exploit Proof-of-Concept

Attached is a C file that works as a POC. Simply compile this on SmartOS and mount a hyprlofs file system and it will trigger the vulnerability.

Timeline

• 2016-12-01 - Vendor Disclosure
• 2016-12-12 - Public Release