Tool calling failing using VLLM
#4
by
corgito
- opened
Tool calling from Open Routers version of Mistral Large 3 works perfectly. However when hosted locally tool calling is broken with the model sending an end of chat token without writing any text which seems to be a problem with VLLM's handling of tool processing. I do not get errors on the server side however so uploading the failure point isn't possible at this point.