vLLM vulnerable to DoS via large Chat Completion or Tokenization requests with specially crafted `chat_template_kwargs`

SummaryThe /v1/chat/completions and /tokenize endpoints allow a chattemplatekwargs request parameter that is used in the code before it is properly validated against the chat template. With the right chattemplatekwargs parameters, it is possible to block processing of the API server for long...

Allocation of Resources Without Limits or Throttling

Overview vllm is an A high-throughput and memory-efficient inference and serving engine for LLMs Affected versions of this package are vulnerable to Allocation of Resources Without Limits or Throttling via the applyhfchattemplate method. An authenticated user can cause the server to become...