Multiple applications fd_set structure bitmap array index overflow

Type securityvulns
Reporter Securityvulns
Modified 2005-01-24T00:00:00


Issue: Multiple applications fd_set structure bitmap array index overflow Type: remote Date: December, 12 2004 Original URL: Author: 3APA3A URL: Affected: gnugk 2.2.0 (confirmed, fixed by vendor) gnugk is OpenH323 Gatekeeper - The GNU Gatekeeper

    jabber 1.4.1 (tested, confirmed)
    jabber is "the Linux of instant messaging" -- an open,
    secure, ad-free alternative to consumer IM services
    like AIM, ICQ, MSN, and Yahoo

    bnc 2.8.4 (tested, confirmed, DoS only)
    BNC is an IRC (Internet Relay Chat) proxying server

    socks5 1.0r1 (untested)
    socks5 is SOCKS v5 application layer gateway and clients
    socks5 is unsupported, contact your package distributor

    citadel 6.27 (untested)
    Citadel is flexible, powerful, community-oriented groupware

    dante 1.1 (tested, confirmed)
    Dante is a circuit-level firewall/proxy (socks implemented)

    rinetd 0.62 (untested)
    rinetd is a simple TCP port redirector
    is unsupported, contact your package distributor

    bld 0.3 (limited, untested)
    bld is a blacklisting daemon

    3proxy 0.4 (limited, tested, fixed by vendor)
    3Proxy is a really tiny cross-platform (Win32&Unix) proxy
    servers set


Actiaully, different advisories should be created for every product, but I do not like idea to flood security lists with similar advisories.

Vulnerability was discovered for 3proxy during stress-testing and was found to apply to different products. Many other products should be tested as well.


fd_set overflow vulnerability is not new. It was already discussed back in 2002 in (1). For reason I don't understand NetBSD reported this vulnerability as a local one and it was never discussed as remotely exploitable vulnerability. It is.


fd_set structure is defined to be used with select() (man 2 select) function. fd_set is used in select() and few special macros (FD_SET, FD_CLR, FD_ISSET, FD_CLEAR). For all POSIX compatible operations systems fd_set is defined as a bitmask array with a socket number as an array index.


define FD_SETSIZE 1024


define NBBY 8 / number of bits in a byte /

typedef long fd_mask;

define NFDBITS (sizeof (fd_mask) * NBBY) / bits per mask /

define howmany(x,y) (((x)+((y)-1))/(y))

typedef struct _types_fd_set { fd_mask fds_bits[howmany(FD_SETSIZE, NFDBITS)]; } _types_fd_set;

define fd_set _types_fd_set

A call to FD_SET sets a bit to 1 using socket number as an index:

define FD_SET(n, p) ((p)->fds_bits[(n)/NFDBITS] |= (1L << ((n) % NFDBITS)))

select() clears all used fd_set bits and sets a bits for sockets with requested activity type. Select calculates length of the bitmask array from the first argument, which must be above the largest number of the socket used.

Neither FD_SET nor select() do not control socket to be below FD_SETSIZE. For select() it's not possible in current select() implementation.

A control for socket number to be below FD_SETSIZE is left for programmer.


If programmer fails to check socket number before using select() or fd_set macros, it's possible to overwrite memory behind fd_set structure. Very few select() based application actually check FD_SETSIZE value. A simpliest example of vulnerable function may be:

int waitsockdata(SOCKET sock, int timeosec, int timeousec){ fd_set fds; struct timeval tv; int res;

tv.tv_sec = timeosec; tv.tv_usec = timeousec; FD_ZERO(&fds); FD_SET(sock, &fds); if ((res = select (sock+1, &fds, NULL, NULL, &tv))!=1) return EOF; return(sock); }

in this example, if waitsockdata() is called with large socket number, some data on the stack, for example saved EIP may be overwriten, but attacker can control only one bit of data. In different situatiion, if multiple sockets are placed on the fd_set attacker may have more control.

Vulnerable application:

There can be few types of select() based Unix servers:

  1. One process per client model (server fork()s for every client connection)
  2. Single application with single thread (finite state machine architecture)
  3. Threaded server (server creates a thread for every client)

Different models can be mixed. Models 2 and 3 could be vulnerable to this kind of attacks if number of clients for each process is not limit or limits allow large number (under FD_SETSIZE) of files or sockets to be open in a single process. Model 1 is safe from this kind of vulnerability (very limited in performance though).

Vulnerable applications examples:

See begining of the article. "untested" means code was audited and problem was found but no testing performed. You can identify vulnerable piecies of code by yourself.


Depending on vulnerable application it's possible to overwrite portions of memory. Impact is close to off-by-one overflows, code execution doesn't seems exploitable.

Mitigating factors:

For all tested Linux distributions default ulimits for open descriptors (1024) are equal to FD_SETSIZE (1024) preventing exploitation in defaul configuration.

For Windows fd_set is a sockets array, not bitmask and FD_SETSIZE defines maximum number of sockets in this array. So, Windows application may be vulnerable only if it places a large number of sockets into same fd_set structure (finite state machine architecture).

In case of BNC it's possible to overwrite variable with addrlen accept() parameter effectively preventing new connections from being accepted and almost fully control stack content. But vulnerable function never returns, so code execution is probably impossible.

Elevating factors:

For FreeBSD and probably many more systems there is no default ulimits.

For Windows default FD_SETSIZE is 64 and select() is only POSIX-complatible function to wait on socket imput (there is no poll(), but there are Windows specific functions).

For Cygwin FD_SETSIZE erroneously defined as 64, probably for Windows compatibility, but fd_set is defined as a bitmask. It makes Cygwin very sensible to this kind of attacks.

Dante probably have some additional bugs, because in tests it crashed long before reaching FD_SETSIZE of connections.


In most cases to exploit this vulnerability to DoS all you need is to establish large number of concurent connections. In some cases you need additionally handshake client protocol. Exploit code is trivial and will not be published.


Set ulimits for all network daemon accounts below FD_SETSIZE. Do not use network daemons compiled with Cygwin.

Vendors and solutions:

3proxy 0.5b fixes the problem, download available from 3proxy still in beta development stage.

All vendors were contacted with contacts listed in product packages. Only Jan Willamowius from GNU h.323 Gatekeeper team replied to bug report.

gnugk 2.2.1 fixes the problem, download available from


  1. NetBSD Security Advisory 2002-014: fd_set overrun in mbone tools and pppd

Thanks to:

Ilya Anfimov and Kirill Lopuchov for help in identifying this vulnerability, Greg MacManus from iDefense for historic information on this issue and help with impact identification (NO, it was not contributed to iDefense, but discussed with Greg), Phiby for here love and patience.