mm/huge_memory.c in the Linux kernel before 2.6.38-rc5 does not prevent creation of a transparent huge page (THP) during the existence of a temporary stack for an exec system call, which allows local users to cause a denial of service (memory consumption) or possibly have unspecified other impact via a crafted application.
git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git%3Ba=commit%3Bh=a7d6e4ecdb7648478ddec76d30d87d03d6e22b31
openwall.com/lists/oss-security/2011/02/17/3
openwall.com/lists/oss-security/2011/02/17/6
www.kernel.org/pub/linux/kernel/v2.6/testing/ChangeLog-2.6.38-rc5
www.securityfocus.com/bid/46442
bugzilla.redhat.com/show_bug.cgi?id=678209
exchange.xforce.ibmcloud.com/vulnerabilities/65535