RUS-CERT Advisory 2002-08:02: Flaw in calloc and similar routines

Flaw in calloc and similar routines

Integer overflow can occur during the computation of the
memory region size by calloc and similar functions. As a
result, the function returns a buffer which is too small,
possibly resulting in a subsequent buffer overflow.

Who Should Read This Document

This advisory is targeted at C, C++ and Ada compiler
implementors and programmers.

Systems Affected

RUS-CERT has verified that the following products are affected
by this defect:

 * C run-time libraries:
      + GNU libc 2.2.5
      + dietlibc CVS as of 2002-08-01
      + Microsoft Visual C++ 4.0
      + Microsoft Visual C++ 6.0

 * language-specific allocators:
      + GNU C++ Compiler (GCC 2.95, 3.0, 3.1.1)
      + GNU Ada Compiler (GNAT 3.14p, GCC 3.1.1)
      + Microsoft Visual C++ 6.0 (C++ new allocator)

Probably many more products are affected. (If a product is not
listed above, it has not been examined.)

Attack Requirements And Impact

Attack requirements and impact depend on the application using
these interfaces.

Description

Many memory allocation interfaces exhibit the same erratic
behavior as the xdr_array Sun RPC function which has recently
been published by ISS (CAN-2002-0391). All these interfaces
take a (somtimes implicit) argument, the storage size of the
element type, and the number of elements. To compute the size
of the memory area which is needed, both numbers are
multipliedA. If the result cannot be represented in a machine
word, it can happen that the allocation routine returns a
pointer to an allocated area which is too small (instead of
signalling an error condition using the appropriate mechanism
defined by the programming language). As a result, the
application might overflow this buffer. The defect discovered
in the xdr_array function mentioned above shows that errors in
this class do have security implications.

Typical code fragments which might lead to vulnerable
applications are listed below.

 * C: pointer = calloc(sizeof(element_t), count);
 * C++: pointer = new ElementType[count];
 * Ada: Array_Access := new Element_Type (1 .. Count);

How To Detect The Defect

In the calloc case, the source code should be examined.
Constructs like "size = count * element_size;" without any
overflow checks are problematic (and, similarly, expressions
like "size *= nelems").

In the C++ and Ada cases, the compiler can emit machine code
instructions to calculate the total size in place. A small
test program like the following can be compiled:

typedef struct {
char data[0x10];
} DATA10;

void allocate(unsigned size)
{
DATA10 *x = new DATA10[size];
}

After that, the generated machine code has to be examined for
overflow checking. In the Ada case, the following procedure
can be used:

procedure Test (Size : Positive) is

subtype Data10 is String (1 … 16#0000_0010#);
type Data10_Array is array (Positive range <>) of Data10;

type Pointer is access Data10_Array;

V : Pointer;

begin
V := new Data10_Array (1 … Size);
end;

For both languages, the size might be computed using a library
routine. In this case, the source code of the library routine
has to be examined.

Countermeasures

The GNU libc CVS repository contains a patch to add overflow
detection to calloc.

Although in most cases, the present overflows can be detected
after they occured because only unsigned types are involved,
programmers should be aware that overflow checking in C is not
straightforward (see RUS-CERT Advisory 2002-08:01,
http://cert.uni-stuttgart.de/advisories/c-integer-overflow.php&#41;.

About RUS-CERT

RUS-CERT (http://CERT.Uni-Stuttgart.DE/&#41; is the Computer
Emergency Response Team located at the Computing Center (RUS)
of the University of Stuttgart, Germany.

Revision History

This document was published on 2002-08-05.

URL For This Document

http://CERT.Uni-Stuttgart.DE/advisories/calloc.php

–
Florian Weimer [email protected]
University of Stuttgart http://CERT.Uni-Stuttgart.DE/people/fw/
RUS-CERT fax +49-711-685-5898