Oracle OIT IX SDK libvs_pdf Root xref Denial of Service Vulnerabiity(CVE-2016-3577)

2017-10-16T00:00:00
ID SSV:96700
Type seebug
Reporter Root
Modified 2017-10-16T00:00:00

Description

DESCRIPTION

A stack overflow leading to a crash due to unbounded recusive function call is present in the PDF file format parsing code of the IX SDK.

TESTED VERSIONS

Oracle Outside In IX sdk 8.5.1

PRODUCT URLs

http://www.oracle.com/technetwork/middleware/content-management/oit-all-085236.html

DETAILS

While parsing a malformed PDF file which contains a reference to the Root element with malformed or missing an xref table a recursive call to a function is made each time with the same parameters eventualy leading to a crash due to process stack exhaustion.

Technical information below:

During a call to VwStreamOpen function in libvs_pdf.so library, code dealing with Root element is reached (image base is at 0xB74BF000): .text:B74ED100 loc_B74ED100: .text:B74ED100 lea ebp, [esp+6BCh+var_BC] .text:B74ED107 cld .text:B74ED108 mov ecx, 8 .text:B74ED10D xor eax, eax .text:B74ED10F mov edi, ebp .text:B74ED111 rep stosd .text:B74ED113 lea ecx, [esp+6BCh+var_34] .text:B74ED11A mov eax, [esp+6BCh+arg_10] .text:B74ED121 mov [esp+6BCh+s], eax .text:B74ED124 lea edx, (aRoot - 0B74F6998h)[ebx] ; "Root" .text:B74ED12A mov eax, esi .text:B74ED12C call sub_B74D653E .text:B74ED131 mov edx, eax .text:B74ED133 test ax, ax .text:B74ED136 jnz loc_B74E

Function sub_B74D653E in turn calls a function sub_B74D5EEC in which the unbounded recursive call can happen: .text:B74D6095 lea edx, [esp+5ACh+var_14] .text:B74D609C lea eax, [esp+5ACh+var_C0] .text:B74D60A3 mov ecx, ebp .text:B74D60A5 call sub_B74D5EEC .text:B74D60AA test ax, ax .text:B74D60AD jnz short loc_B74

The supplied minimized testcase triggers the recursive call and leads to a crash due to stack exhaustion. The sample program ixsample supplied with the SDK can be used to reproduce the crash.

TIMELINE

  • 2016-04-12 - Vendor Notification
  • 2016-07-19 - Public Disclosure