Integrating OpenVINO GenAI into LangChain

Discussion on Integrating OpenVINO GenAI into LangChain :rocket:

Hello LangChain Community,

I’d like to initiate a discussion regarding the use and community-based integration of OpenVINO GenAI (GitHub - openvinotoolkit/openvino.genai: Run Generative AI models with simple C++/Python API and using OpenVINO Runtime).

Discussion on OpenVINO GenAI Support in LangChain V1.0+

Are there any existing plans or ongoing discussions within the community for supporting or integrating OpenVINO GenAI in LangChain’s major version 1.0 and beyond (e.g., langchain>=1.0.0)?

If anyone has information about potential community contributions to OpenVINO GenAI components (e.g., contribution to the langchain-community package) for future stable LangChain versions, please share it here.

Introduction of a Custom Implementation Enabling NPU Operation and Contribution Proposal

I have developed and published a custom implementation that leverages OpenVINO GenAI, based on the official OpenVINO notebook (openvino_notebooks/notebooks/llm-rag-langchain/llm-rag-langchain-genai.ipynb at latest · openvinotoolkit/openvino_notebooks · GitHub).

My Repository: GitHub - KTC-Security-Circle/langchain-openvino-genai: OpenVINO GenAI を LangChain に組み込むためのライブラリ

(Note: This repository is the result of my personal development and is currently hosted under my school circle organization’s account, KTC-Security-Circle.)

This implementation makes LLM operation possible on NPUs (Neural Processing Units), which was challenging with traditional HuggingFace backend implementations. I believe this can contribute to improved performance and efficiency.

Interest in Community Integration

Is anyone interested in or already considering integrating a component based on this custom implementation into the LangChain community ecosystem (e.g., a Pull Request to langchain-community)? I welcome collaboration and feedback for refinement and contribution.

While my implementation is currently at a personal level, supporting OpenVINO GenAI is a crucial step towards enabling faster and more efficient AI application development for users of Intel hardware (especially NPU-equipped devices).

I look forward to your constructive discussion and information sharing.