Response Format Groq Model Pydantic

i try to build some ai agent with groq model `meta-llama/llama-4-scout-17b-16e-instruct`. i want to structured the output with pydantic. But, it cant.

❯ python main.py
Enter your query: analisis saham BBCA dong
Traceback (most recent call last):
  File "/home/galangzzz/Development/agentic/tes/main.py", line 30, in <module>
    raw_response = agent.invoke({"messages": [{"role": "user", "content": query}]})
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langgraph/pregel/main.py", line 3880, in invoke
    for chunk in self.stream(
                 ~~~~~~~~~~~^
        input,
        ^^^^^^
    ...<11 lines>...
        **kwargs,
        ^^^^^^^^^
    ):
    ^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langgraph/pregel/main.py", line 2936, in stream
    for _ in runner.tick(
             ~~~~~~~~~~~^
        [t for t in loop.tasks.values() if not t.writes],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        schedule_task=loop.accept_push,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ):
    ^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langgraph/pregel/_runner.py", line 207, in tick
    run_with_retry(
    ~~~~~~~~~~~~~~^
        t,
        ^^
    ...<10 lines>...
        },
        ^^
    )
    ^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langgraph/pregel/_retry.py", line 585, in run_with_retry
    return task.proc.invoke(task.input, config)
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langgraph/_internal/_runnable.py", line 684, in invoke
    input = context.run(step.invoke, input, config, **kwargs)
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langgraph/_internal/_runnable.py", line 426, in invoke
    ret = self.func(*args, **kwargs)
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain/agents/factory.py", line 1330, in model_node
    model_response = _execute_model_sync(request)
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain/agents/factory.py", line 1302, in _execute_model_sync
    output = model_.invoke(messages)
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain_core/runnables/base.py", line 5881, in invoke
    return self.bound.invoke(
           ~~~~~~~~~~~~~~~~~^
        input,
        ^^^^^^
        self._merge_configs(config),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        **{**self.kwargs, **kwargs},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain_core/language_models/chat_models.py", line 474, in invoke
    self.generate_prompt(
    ~~~~~~~~~~~~~~~~~~~~^
        [self._convert_input(input)],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
        **kwargs,
        ^^^^^^^^^
    ).generations[0][0],
    ^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain_core/language_models/chat_models.py", line 1823, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain_core/language_models/chat_models.py", line 1630, in generate
    self._generate_with_cache(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        m,
        ^^
    ...<2 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain_core/language_models/chat_models.py", line 1970, in _generate_with_cache
    result = self._generate(
        messages, stop=stop, run_manager=run_manager, **kwargs
    )
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/langchain_groq/chat_models.py", line 621, in _generate
    response = self.client.create(messages=message_dicts, **params)
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/groq/resources/chat/completions.py", line 461, in create
    return self._post(
           ~~~~~~~~~~^
        "/openai/v1/chat/completions",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<45 lines>...
        stream_cls=Stream[ChatCompletionChunk],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/groq/_base_client.py", line 1242, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/galangzzz/Development/agentic/tes/venv/lib64/python3.14/site-packages/groq/_base_client.py", line 1044, in request
    raise self._make_status_error_from_response(err.response) from None
groq.BadRequestError: Error code: 400 - {'error': {'message': "Failed to call a function. Please adjust your prompt. See 'failed_generation' for more details.", 'type': 'invalid_request_error', 'code': 'tool_use_failed', 'failed_generation': 'Executive Summary:\nSaham BBCA menunjukkan performa yang stabil dengan harga terkini sebesar Rp10,200. Rasio PER dan PBV yang sehat yaitu 24.5 dan 4.8, serta pertumbuhan tahunan (YoY) sebesar +10%.\n\nFinansial & Performa Saham:\n- Harga: Rp10,200\n- PER: 24.5\n- PBV: 4.8\n- YoY Growth: +10%\n\nAnalisis Sentimen Berita:\n1. Rekor laba bersih kuartal I.\n2. Ekspansi digital banking masif.\n3. Sentimen pasar positif terhadap perbankan.\n\nKesimpulan & Risiko:\nSaham BBCA cenderung bullish dengan fundamental yang kuat dan sentimen pasar yang positif. Namun, perlu diwaspadai potensi risiko terkait perubahan kebijakan moneter dan persaingan di sektor perbankan.'}}
During task with name 'model' and id '08b15ce1-f8eb-5de4-0d7a-85e078901b1c'
from pydantic import BaseModel, Field
from typing import List, Optional

class ResearchResponse(BaseModel):
    """Model untuk struktur response yang diharapkan dari agent."""
    topic: str = Field(..., description="Topik utama yang dibahas dalam response, misalnya nama emiten saham")
    summary: str = Field(..., description="Ringkasan singkat dari response")
    response: str = Field(..., description="Analisis lengkap dalam format markdown")
    sources: List[str] = Field(..., description="Daftar sumber data yang digunakan, termasuk nama tools dan link jika ada")
    tools_used: Optional[List[str]] = Field(None, description="Daftar tools yang dipanggil selama proses analisis")
class SystemInstruction:
    def __init__(self):
        self.instruction = """
                Anda adalah "FinInsight Alpha", seorang Agen Analis Keuangan Senior dan Spesialis Riset Pasar yang presisi, objektif, dan analitis.

                Misi Utama:
                Tugas Anda adalah memberikan analisis komprehensif mengenai kondisi emiten saham tertentu berdasarkan kombinasi data metrik finansial riil dan sentimen berita terbaru.

                Aturan Penggunaan Tools:
                1. Sebelum memberikan kesimpulan, Anda WAJIB memanggil `fetch_stock_price` untuk mendapatkan metrik harga dan rasio finansial terkini.
                2. Anda WAJIB memanggil `fetch_recent_news` untuk menganalisis sentimen pasar 7 hari terakhir.
                3. Jangan pernah berasumsi atau mengarang data angka jika tools mengembalikan nilai kosong (null). Nyatakan apa adanya.

                Batasan & Guardrails (CONSTRAINTS):
                - Dilarang keras memberikan rekomendasi investasi langsung ("Beli" atau "Jual"). Gunakan bahasa netral ("Cenderung Bullish", "Risiko Tinggi", dll).
                - Jika user menanyakan hal di luar topik finansial dan pasar saham, tolak dengan sopan.
                - Batasi proses berpikir Anda. Jika setelah 3 kali mencoba tools Anda tidak mendapatkan data yang valid, berikan ringkasan kendala teknis kepada user.

                Jangan menghasilkan markdown sebagai output akhir.

                Gunakan struktur informasi berikut untuk response Anda:

                - Executive Summary
                - Finansial & Performa Saham
                - Analisis Sentimen Berita
                - Kesimpulan & Risiko

                Jangan menentukan format output.
                """

from dotenv import load_dotenv
from langchain_groq import ChatGroq
from langchain.agents import create_agent
from tools import tools
from system_insturction import SystemInstruction
from response_structure import ResearchResponse

load_dotenv()


llm = ChatGroq(
    model="meta-llama/llama-4-scout-17b-16e-instruct",
    temperature=0.2,
    
)

system_prompt_content = SystemInstruction().instruction

agent = create_agent(
    model=llm,
    tools=tools,
    system_prompt=system_prompt_content,
    response_format=ResearchResponse,
)


query = input("Enter your query: ")


raw_response = agent.invoke({"messages": [{"role": "user", "content": query}]})


response = raw_response.get("structured_response", {})

print("Raw Response:\n", raw_response)
print("\n")

raw_response["messages"][
    -1
].pretty_print()  # Print the raw response for debugging purposes
print("\n")
# try:
#     print("Structured Response:\n", response)
#     print("\nSummary:\n", response.response)

# except Exception as e:
#     print("Error occurred while printing structured response:", e)




it works if i delete `response_format=ResearchResponse`

hi @Galangzz

maybe try some of these fixes:

  1. switch model - groq:openai/gpt-oss-120b (strict structured + tools work via create_agent), or Llama 3.3-70B for the function-calling path
  2. keep Llama 4, decouple - run agent with no response_format, then format final text with a separate llm.with_structured_output(ResearchResponse, method="json_schema") (no tools bound - dodges both limits)
  3. upgrade langchain/langchain-groq (ships Scout profile structured_output:True - AutoStrategy picks native path). Optionally force response_format=ProviderStrategy(ResearchResponse) - but tools+structured still collide, so best when agent has no tools
  4. schema hygiene - drop Optional[List[str]], use Field(default_factory=list)

response_format=ResearchResponse makes create_agent wrap it in AutoStrategy. When the model exposes no capability profile (old langchain-groq), _supports_provider_strategy falls back to a list that excludes Llama - picks ToolStrategy = force a structured-output tool call (tool_choice="any"). Groq’s Llama 4 models are documented broken on that path (~100% fail): model emits JSON wrapped in <function=…> tags, Groq’s tool parser rejects - 400 tool_use_failed “Failed to call a function.”

Why failed_generation looks valid. Content is good. 400 is not Pydantic validation - it’s Groq’s server-side function-call extractor rejecting the delivery shape, not the data.

Why deleting response_format works. No synthetic tool, no forced call - no tool_use_failed. But you lose typing.

Two Groq limits to design around (Structured Outputs - GroqDocs):

  • “tool use not supported with Structured Outputs” - can’t bind your tools + native json_schema in same call
  • Llama 4 Scout = best-effort only (strict:false) - strict structured outputs only on gpt-oss-20b/120b