Hi community,
Speaking as an agentic-application developer myself: I keep meeting people who want to adopt LangGraph with their own self-served local models and custom services, and they can’t see how the layers relate. I often see questions like “how can I run LangGraph on CUDA?”, “does Intel GPU support LangGraph?”, or “how can I use my own models served by vLLM/SGLang for a LangGraph app?”
We all know LangGraph is an agent-graph / state-management framework and doesn’t sit at the serving or hardware layers, but newcomers often don’t get how these connect.
I think it would help to have one notebook example showing the full end-to-end pipeline (serve a local model + custom tool/service + tool-calling agent + LangGraph ochestration), to give newcomers clear context and encourage adoption. If one already exists and I missed it, I’d appreciate a pointer. If not, I’m happy to contribute one.
Happy to hear your thoughts!
Hi, @cheehook!
We don’t currently have a vLLM or SGLang example. But for local models, we do have a complete end-to-end example using Ollama: local-deep-researcher.
The LLM runs fully local — query generation, summarization, reflection, and tool calling all happen on a locally-served model, with LangGraph orchestrating the flow. The serving layer differs (Ollama vs. vLLM/SGLang), but the pattern is exactly what you’re describing, and LangGraph sits above it either way.
Hope that helps!
Thanks @dariel.datoon, that’s a great pointer. I hadn’t come across this interesting repo. Just curious, is there currently an official way to contribute examples like this?
One thought: it might strengthen the story to show a few examples spanning different serving backends on different hardware. That would give clearer proof point that LangGraph runs well across setups, and likely encourage broader adoption.
Any of our open-source repositories you are welcome to raise a feature request or PR.
Or you can create your own repo and share with the community here!
Great, thanks, will definitely consider that!
I’ve gone ahead and raised a PR to local-deep-researcher to add vLLM and SGLang as serving options: Add vLLM and SGLang serving engines by cheehook · Pull Request #135 · langchain-ai/local-deep-researcher · GitHub. Hopefully this serves as a proof point that local-deep-researcher (and LangGraph) works with vLLM and SGLang deployments on different GPU setups.