fs/exec.c in the Linux kernel before 2.6.37 does not enable the OOM Killer to assess use of stack memory by arrays representing the (1) arguments and (2) environment, which allows local users to cause a denial of service (memory consumption) via a crafted exec system call, aka an “OOM dodging issue,” a related issue to CVE-2010-3858.
git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git%3Ba=commit%3Bh=3c77f845722158206a7209c45ccddc264d19319c
grsecurity.net/~spender/64bit_dos.c
linux.derkeiler.com/Mailing-Lists/Kernel/2010-11/msg13278.html
lkml.org/lkml/2010/8/27/429
lkml.org/lkml/2010/8/29/206
lkml.org/lkml/2010/8/30/138
lkml.org/lkml/2010/8/30/378
openwall.com/lists/oss-security/2010/11/22/15
openwall.com/lists/oss-security/2010/11/22/6
secunia.com/advisories/42884
secunia.com/advisories/46397
www.exploit-db.com/exploits/15619
www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.37
www.redhat.com/support/errata/RHSA-2011-0017.html
www.securityfocus.com/archive/1/520102/100/0/threaded
www.securityfocus.com/bid/45004
www.vmware.com/security/advisories/VMSA-2011-0012.html
bugzilla.redhat.com/show_bug.cgi?id=625688
exchange.xforce.ibmcloud.com/vulnerabilities/64700