get_openai_callback not working when using Agent Executor after updating to latest version of Langchain

The problem

Issue langchain-ai/langchain#16798 (closed, 43 comments): ### Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.

Example Code

class MyCustomAsyncHandler(AsyncCallbackHandler):
    async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
        """Run when chain ends running."""
        print("RESPONSE: ", response)
        print("Hi! I just woke up. Your llm is...

## What to do
Token usage callbacks were silently dropped because `AgentExecutor` forced streaming on the runnable. The confirmed workaround is to disable streaming on the executor:

agent = AgentExecutor(..., stream_runnable=False)

or on an existing executor:

agent_executor.agent.stream_runnable = False

Users verified token counts came back after this. On newer setups where you want streaming, pass `stream_options={"include_usage": True}` to the model instead.

In the GitHub issue gh:langchain-ai/langchain#16798 (https://github.com/langchain-ai/langchain/issues/16798), the problem reported was: Issue langchain-ai/langchain#16798 (closed, 43 comments): ### Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.

Example Code

class MyCustomAsyncHandler(AsyncCallbackHandler):
    async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
        """Run when chain ends running."""
        print("RESPONSE: ", response)
        print("Hi! I just woke up. Your llm is... The verified resolution was: Token usage callbacks were silently dropped because `AgentExecutor` forced streaming on the runnable. The confirmed workaround is to disable streaming on the executor:

agent = AgentExecutor(..., stream_runnable=False)

or on an existing executor:

agent_executor.agent.stream_runnable = False

Users verified token counts came back after this. On newer setups where you want streaming, pass `stream_options={"include_usage": True}` to the model instead.