vLLM

Fast inference and serving engine for language models

vLLM is an AI product for fast inference and serving engine for language models.