Enhancement: Allow ToolCallLimitMiddleware to accept multiple tool names

OSS Product Help LangChain
1

Problem Statement

Currently, applying the same rate limits to multiple tools requires creating separate middleware instances for each tool, leading to repetitive and verbose code.

Current Approach (Repetitive)

from langchain.agents import create_agent
from langchain.agents.middleware import ToolCallLimitMiddleware

# Need to create separate limiter for each tool with identical configs
search_limiter = ToolCallLimitMiddleware(
    tool_name="search",
    thread_limit=20,
    run_limit=10
)

database_limiter = ToolCallLimitMiddleware(
    tool_name="database",
    thread_limit=20,
    run_limit=10
)

api_limiter = ToolCallLimitMiddleware(
    tool_name="api_call",
    thread_limit=20,
    run_limit=10
)

scraper_limiter = ToolCallLimitMiddleware(
    tool_name="web_scraper",
    thread_limit=20,
    run_limit=10
)

agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, database_tool, api_tool, scraper_tool],
    middleware=[search_limiter, database_limiter, api_limiter, scraper_limiter]
)

This becomes especially cumbersome when managing multiple tiers of tools with different limits.

Proposed Solution

Allow tool_name parameter to accept either a single string or a list of strings:

# Proposed approach - clean and maintainable
expensive_limiter = ToolCallLimitMiddleware(
    tool_name=["search", "database", "api_call", "web_scraper"],
    thread_limit=20,
    run_limit=10
)

agent = create_agent(
    model="gpt-4o",
    tools=[search_tool, database_tool, api_tool, scraper_tool],
    middleware=[expensive_limiter]
)

Backward Compatibility

The existing API would continue to work unchanged:

# Still works - single tool as string
single_limiter = ToolCallLimitMiddleware(
    tool_name="search",
    thread_limit=5,
    run_limit=3
)

Use Cases & Value

1. Cost Management by Tier

Many applications have tools with different cost profiles:

agent = create_agent(
    model="gpt-4o",
    tools=[...],
    middleware=[
        # Expensive external APIs
        ToolCallLimitMiddleware(
            tool_name=["openai_api", "anthropic_api", "google_search"],
            thread_limit=10,
            run_limit=3
        ),
        
        # Moderate cost tools
        ToolCallLimitMiddleware(
            tool_name=["database_query", "file_storage", "email_send"],
            thread_limit=50,
            run_limit=20
        ),
        
        # Cheap local tools
        ToolCallLimitMiddleware(
            tool_name=["calculator", "string_parser", "date_formatter"],
            thread_limit=200,
            run_limit=100
        ),
    ]
)

2. User Tier Rate Limiting

Different limits for free vs paid users:

def create_user_agent(user_tier: str):
    if user_tier == "free":
        expensive_tools_limit = ToolCallLimitMiddleware(
            tool_name=["search", "database", "api"],
            thread_limit=5,
            run_limit=2
        )
    else:  # premium
        expensive_tools_limit = ToolCallLimitMiddleware(
            tool_name=["search", "database", "api"],
            thread_limit=100,
            run_limit=30
        )
    
    return create_agent(
        model="gpt-4o",
        tools=[...],
        middleware=[expensive_tools_limit]
    )

3. Compliance & Safety

Group tools that require similar safety controls:

# Tools that access PII data
pii_limiter = ToolCallLimitMiddleware(
    tool_name=["customer_database", "email_reader", "contact_lookup"],
    thread_limit=10,
    run_limit=3
)

# Tools that can modify state
write_limiter = ToolCallLimitMiddleware(
    tool_name=["database_write", "file_delete", "email_send"],
    thread_limit=5,
    run_limit=1
)

Questions for Community

  1. Should limits be independent (each tool gets separate counter) or shared (tools share a pool)?
  2. Should we add a limit_mode parameter to support both behaviors?
  3. Any other edge cases to consider?

Looking forward to feedback from the maintainers and community! Happy to help with implementation if this moves forward.

1 Like
2

hi @Jameskanyiri

yeah! I am all for this concept!

My answers:

  1. imho limits should be optionally independent by passing dicts with tool names as the keys
  2. rough idea: could be useful - when limit mode provided and the limits are not dicts but numbers then the mode enforces shared or not shared limits
  3. might be - let’s see how it plays out
1 Like