GHSA-Q97C-P368-75C4 vLLM: Out-of-vocabulary prompt token id passes input validation and triggers a CUDA device-side assert (engine crash / DoS)

For a model whose vocabulary size is an exact multiple of the embedding padding size 64 — e.g. any Llama-2-family model, vocab 32000 — a single request carrying a pre-tokenized prompt with token id equal to the vocab size 32000 passes vLLM's input-id validation but is one past the last embedding...

vLLM: Out-of-vocabulary prompt token id passes input validation and triggers a CUDA device-side assert (engine crash / DoS)

For a model whose vocabulary size is an exact multiple of the embedding padding size 64 — e.g. any Llama-2-family model, vocab 32000 — a single request carrying a pre-tokenized prompt with token id equal to the vocab size 32000 passes vLLM's input-id validation but is one past the last embedding...