Oracle OIT IX SDK libvs_pdf Root xref Denial of Service Vulnerabiity(CVE-2016-3577)

ID SSV:96700
Type seebug
Reporter Root
Modified 2017-10-16T00:00:00



A stack overflow leading to a crash due to unbounded recusive function call is present in the PDF file format parsing code of the IX SDK.


Oracle Outside In IX sdk 8.5.1



While parsing a malformed PDF file which contains a reference to the Root element with malformed or missing an xref table a recursive call to a function is made each time with the same parameters eventualy leading to a crash due to process stack exhaustion.

Technical information below:

During a call to VwStreamOpen function in library, code dealing with Root element is reached (image base is at 0xB74BF000): .text:B74ED100 loc_B74ED100: .text:B74ED100 lea ebp, [esp+6BCh+var_BC] .text:B74ED107 cld .text:B74ED108 mov ecx, 8 .text:B74ED10D xor eax, eax .text:B74ED10F mov edi, ebp .text:B74ED111 rep stosd .text:B74ED113 lea ecx, [esp+6BCh+var_34] .text:B74ED11A mov eax, [esp+6BCh+arg_10] .text:B74ED121 mov [esp+6BCh+s], eax .text:B74ED124 lea edx, (aRoot - 0B74F6998h)[ebx] ; "Root" .text:B74ED12A mov eax, esi .text:B74ED12C call sub_B74D653E .text:B74ED131 mov edx, eax .text:B74ED133 test ax, ax .text:B74ED136 jnz loc_B74E

Function sub_B74D653E in turn calls a function sub_B74D5EEC in which the unbounded recursive call can happen: .text:B74D6095 lea edx, [esp+5ACh+var_14] .text:B74D609C lea eax, [esp+5ACh+var_C0] .text:B74D60A3 mov ecx, ebp .text:B74D60A5 call sub_B74D5EEC .text:B74D60AA test ax, ax .text:B74D60AD jnz short loc_B74

The supplied minimized testcase triggers the recursive call and leads to a crash due to stack exhaustion. The sample program ixsample supplied with the SDK can be used to reproduce the crash.


  • 2016-04-12 - Vendor Notification
  • 2016-07-19 - Public Disclosure