VLLM: Out-of-vocabulary Prompt Token Id Passes Input Validation And Triggers A CUDA Device-side Assert (engine Crash / D

For a model whose vocabulary size is an exact multiple of the embedding padding size 64 — e.g. any Llama-2-family model, vocab 32000 — a single request carrying a pre-tokenized prompt with token id equal to the vocab size 32000 passes vLLM's input-id validation but is one past the last embedding...