Dropbox: avrecode: global-buffer-overflow in get_neighbor()

2017-04-06T06:10:15
ID H1:218966
Type hackerone
Reporter geeknik
Modified 2019-11-03T16:15:23

Description

Source: https://github.com/dropbox/avrecode Version: 2de743d

Built using the Github instructions with afl-gcc and ASAN. Feeding this malformed .mp4 to recode triggers a global buffer overflow.

./recode roundtrip test003.mp4

``` [mov,mp4,m4a,3gp,3g2,mj2 @ 0x61b00001f180] Protocol name not provided, cannot determine if input is local or a network protocol, buffers and access patterns cannot be configured optimally without knowing the protocol [h264 @ 0x61a00001f280] error while decoding MB 2 2, bytestream -11 [h264 @ 0x61a00001f280] concealing 27 DC, 27 AC, 27 MV errors in I frame Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'id:000003,sig:06,src:000000,op:havoc,rep:4': Metadata: major_brand : mp42 minor_version : 19529854 compatible_brands: mp42isom creation_time : 2014-11-14 07:34:24 Duration: 00:00:01.00, start: 0.083333, bitrate: N/A Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 48x144 [SAR 1:1 DAR 1:3], 3 kb/s, 12 fps, 12 tbr, 12 tbn, 24 tbc (default) Metadata: rotate : 0 creation_time : 2014-11-14 07:34:24 handler_name : Video Media Handler encoder : AVC Coding Side data: displaymatrix: rotation of -0.00 degrees FINISHED QUEUING DECODE: 1 FINISHED QUEUING DECODE: 2 FINISHED QUEUING DECODE: 2 FINISHED QUEUING DECODE: 3 FINISHED QUEUING DECODE: 3 FINISHED QUEUING DECODE: 6 FINISHED QUEUING DECODE: 7 FINISHED QUEUING DECODE: 7 FINISHED QUEUING DECODE: 6 FINISHED QUEUING DECODE: 2 ================================================================= ==5924==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000003bceea4 at pc 0x656bd8 bp 0x7ffc81e15650 sp 0x7ffc81e15648 READ of size 1 at 0x000003bceea4 thread T0 #0 0x656bd7 in get_neighbor(bool, int, CoefficientCoord, CoefficientCoord) /root/avrecode/recode.cpp:496 #1 0x6648f2 in h264_model::get_model_key(void const) const /root/avrecode/recode.cpp:717 #2 0x670fb3 in h264_model::probability_for_state(unsigned long, void const) /root/avrecode/recode.cpp:822 #3 0x670fb3 in void h264_symbol::execute<arithmetic_code<unsigned long, unsigned char, 0>::encoder<std::back_insert_iterator<std::vector<unsigned char, std::allocator<unsigned char> > >, unsigned char> >(arithmetic_code<unsigned long, unsigned char, 0>::encoder<std::back_insert_iterator<std::vector<unsigned char, std::allocator<unsigned char> > >, unsigned char>&, h264_model, Recoded_Block, std::vector<unsigned char, std::allocator<unsigned char> >&)::{lambda(unsigned long)#1}::operator()(unsigned long) const /root/avrecode/recode.cpp:1075 #4 0x670fb3 in std::_Function_handler<unsigned long (unsigned long), void h264_symbol::execute<arithmetic_code<unsigned long, unsigned char, 0>::encoder<std::back_insert_iterator<std::vector<unsigned char, std::allocator<unsigned char> > >, unsigned char> >(arithmetic_code<unsigned long, unsigned char, 0>::encoder<std::back_insert_iterator<std::vector<unsigned char, std::allocator<unsigned char> > >, unsigned char>&, h264_model, Recoded_Block, std::vector<unsigned char, std::allocator<unsigned char> >&)::{lambda(unsigned long)#1}>::_M_invoke(std::_Any_data const&, unsigned long) /usr/include/c++/4.9/functional:2025 #5 0x683241 in std::function<unsigned long (unsigned long)>::operator()(unsigned long) const /usr/include/c++/4.9/functional:2439 #6 0x683241 in arithmetic_code<unsigned long, unsigned char, 0>::encoder<std::back_insert_iterator<std::vector<unsigned char, std::allocator<unsigned char> > >, unsigned char>::put(int, std::function<unsigned long (unsigned long)>) /root/avrecode/arithmetic_code.h:107 #7 0x683241 in void h264_symbol::execute<arithmetic_code<unsigned long, unsigned char, 0>::encoder<std::back_insert_iterator<std::vector<unsigned char, std::allocator<unsigned char> > >, unsigned char> >(arithmetic_code<unsigned long, unsigned char, 0>::encoder<std::back_insert_iterator<std::vector<unsigned char, std::allocator<unsigned char> > >, unsigned char>&, h264_model, Recoded_Block, std::vector<unsigned char, std::allocator<unsigned char> >&) /root/avrecode/recode.cpp:1075 #8 0x6916b7 in compressor::cabac_decoder::pop_queueing_symbols(CodingType) /root/avrecode/recode.cpp:1252 #9 0x6916b7 in compressor::cabac_decoder::end_coding_type(CodingType) /root/avrecode/recode.cpp:1226 #10 0x6916b7 in av_decoder<compressor>::model_hooks::end_coding_type(void, CodingType) /root/avrecode/recode.cpp:207 #11 0x36bc9b3 in decode_cabac_residual_internal libavcodec/h264_cabac.c:1721 #12 0x36bc9b3 in decode_cabac_residual_dc_internal libavcodec/h264_cabac.c:1789 #13 0x36de622 in decode_cabac_residual_dc libavcodec/h264_cabac.c:1837 #14 0x36de622 in decode_cabac_luma_residual libavcodec/h264_cabac.c:1887 #15 0x36de622 in ff_h264_decode_mb_cabac libavcodec/h264_cabac.c:2418 #16 0x18b80ac in decode_slice libavcodec/h264_slice.c:2384 #17 0x18de556 in ff_h264_execute_decode_slices libavcodec/h264_slice.c:2560 #18 0x176348b in decode_nal_units libavcodec/h264.c:1646 #19 0x176348b in h264_decode_frame libavcodec/h264.c:1831 #20 0x6f522f in avcodec_decode_video2 libavcodec/utils.c:2115 #21 0x66b870 in av_decoder<compressor>::decode_video() /root/avrecode/recode.cpp:130 #22 0x67de37 in compressor::run() /root/avrecode/recode.cpp:1119 #23 0x6585f5 in roundtrip(std::string const&, std::ostream*) /root/avrecode/recode.cpp:1598 #24 0x638509 in main /root/avrecode/recode.cpp:1650 #25 0x7f73e7c3ab44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b44) #26 0x65451c (/root/avrecode/recode+0x65451c)

0x000003bceea4 is located 60 bytes to the left of global variable 'zigzag4' from 'recode.cpp' (0x3bceee0) of size 4 0x000003bceea4 is located 0 bytes to the right of global variable 'unzigzag4' from 'recode.cpp' (0x3bceea0) of size 4 SUMMARY: AddressSanitizer: global-buffer-overflow /root/avrecode/recode.cpp:496 get_neighbor(bool, int, CoefficientCoord, CoefficientCoord*) ```