bogofilter -- heap corruption through excessively long words

Matthias Andree reports:

Bogofilter’s/bogolexer’s input handling in version 0.96.2 was not
keeping track of its output buffers properly and could overrun a
heap buffer if the input contained words whose length exceeded
16,384 bytes, the size of flex’s input buffer. A “word” here refers
to a contiguous run of input octets that was not ‘_’ and did not
match at least one of ispunct(), iscntrl() or isspace().