i am using after_model for summary messagses, but dont use summary for next model calls, now I am confused.
I have a class CustomSummaryMiddleware(SummarizationMiddleware)
At first I return messages in after_model with summary like, return {
“messages”: messages,
“summary”: summary_message,
“should_summarize”: True
}
but chunk can only get messages without summary.
I think it may only get messages, so I put summary messages to “messages”, like return {
“messages”: [*preserved_messages, *summary_messages],
}
but it has problem, I found that the model only try once and then exit. why?