6083f6dd20
* fix agent submit * update scripts * update process * update scripts * update routing strategy * update scripts * fix strategy * update routing script * update launching script * fix decoding tool call * fix tool calling issues for sglang backend * update * list available llms
5 lines
162 B
Bash
5 lines
162 B
Bash
CUDA_VISIBLE_DEVICES=1 vllm serve Qwen/Qwen2.5-7B-Instruct \
|
|
--enable-auto-tool-choice \
|
|
--tool-call-parser hermes \
|
|
--host 0.0.0.0 \
|
|
--port 30001 |