Stuck with a no way out

Talking Shop
1

TBH i am up against a wall , i have tried everything over the past few days .

So there is a node with in my graph that just keeps failing the evals over and over , like the best it can do is pass 64% of the test all it has to do is take a look at the current query and related to query has to produce a structured response the structured response is something like

class TurnIntent(BaseModel):

    model_config = ConfigDict(
        extra="forbid",
    )

    qualification_intent: bool
    retrieval_intent: bool
    pursue_response_intent: bool

    repeat_question_intent: bool
    explain_question_intent: bool
    off_topic_intent: bool
    conversation_end_intent:bool

i provide it with the context as well , i have tried debugging the system prompt changing the context i am providing tried chanign models , nothing seems to be working

can you guide me how would you guys debug this specific problem

2

I have found using some few shot examples in the prompt helps the model understand what outcome I’m looking for. I would start with that.