LangSmith continues making OpenAI API calls after OpenAI billing threshold is reached

LangSmith Product Help Observability & Evals
,
1

Hi everyone,

I’m seeing an issue where LangSmith appears to continue sending requests to the OpenAI API even after the usage threshold configured for my OpenAI API key has been reached.

Issue:

  • I configured a spending threshold/limit for my OpenAI API key.
  • Once the threshold is reached, I expect API requests to stop or return an error.
  • Instead, LangSmith continues making requests, and my OpenAI account balance goes into the negative.
2

Hi, @Bitcot_Kaushal! I think this is more of an OpenAI issue. Seems like they have this documented.

Does the same thing happen even if you run outside of LangSmith?

3

Hi @dariel.datoon ,

I have enabled the hard reset limit from OpenAI. But it got breached only when the requests are coming from LangSmith. If I test the same with some other platform, it throws a 429 error (request limit reached).

4

Are you doing any sort of batch calls? Any code samples you can provide?

According to OpenAI’s docs, even with hard limits, “Enforcement is not instantaneous. The API Platform can process a small amount of extra usage while the limit state propagates, so recorded spend can slightly exceed the configured amount.”