llm --help

Let's help help help devs.

Target: <1000ms. Several important LLM CLI tools take multiple seconds. PRs welcome ❤️

2 measurement groups

tool cold warm version
tensorrt-llm --help29234ms13112ms1.2.1
vllm --help19570ms9219ms0.27.1
sglang --help21557ms8177ms0.5.17
VLMEvalKit --help15846ms5527msv0.2
tokenspeed --help8601ms3072ms0.1.0@9196fa2
transformers --help7779ms2616ms5.15.1
datasets --help2784ms828ms5.0.1
llm --help1451ms646ms0.31.1
openai --help856ms400ms2.34.0
hf --help813ms328ms1.28.0
lm-eval --help1508ms304ms0.4.12
langchain-cli --help652ms216ms0.0.37
ollama --help17ms15ms0.32.14
llama.cpp --help17ms14msb10502