Let's help help help devs.
Target: <1000ms. Several important LLM CLI tools take multiple seconds. PRs welcome ❤️
| tool | cold | warm | version |
|---|---|---|---|
tensorrt-llm --help | 18086ms | 9115ms | 1.2.1 |
vllm --help | 12179ms | 5326ms | 0.23.0 |
sglang --help | 9732ms | 3566ms | 0.5.9 |
VLMEvalKit --help | 9289ms | 3238ms | v0.2 |
transformers --help | 7141ms | 1897ms | 5.12.0 |
datasets --help | 1880ms | 571ms | 5.0.0 |
llm --help | 698ms | 327ms | 0.31 |
openai --help | 557ms | 282ms | 2.34.0 |
hf --help | 540ms | 212ms | 1.19.0 |
lm-eval --help | 968ms | 183ms | 0.4.12 |
langchain-cli --help | 431ms | 151ms | 0.0.37 |
tokenspeed --help | 274ms | 115ms | 0.1.0@9d63ac6 |
ollama --help | 12ms | 10ms | 0.30.8 |
llama.cpp --help | 18ms | 9ms | b9630 |