I saw in the official documentation that they use ChatOpenAI to call a model locally launched via vLLM. However, I’m not sure if this method is applicable to all models launched via vLLM.
Additionally, I’m uncertain whether this approach is the most up-to-date and stable.
I have the same question regarding sglang.
Another important question: If I call a 72B model via an API versus deploying a 72B model via vLLM and then calling it using the method mentioned above, will there be any difference in tool usage and structured output performance?