Working with modern large language models requires not only competent prompting, but also an understanding of the economics of the context window. AI Architect and Software Engineer Denis Klymenko shared his experience in systematically optimizing the costs of using artificial intelligence tools.
Working with modern large language models requires not only competent prompting, but also an understanding of the economics of the context window. AI Architect and Software Engineer Denis Klymenko shared his experience in systematically optimizing the costs of using artificial intelligence tools.
According to IT specialists, the main reason for high bills for using the Claude model line lies in the specifics of data processing: the model does not have its own internal memory, so with each new message, the user sends the entire previous dialogue again.
"The most expensive thing about working with Claude is not the answers themselves. It's the "tail" that you drag behind you. The context grows linearly, and the bill grows quadratically: the fiftieth request drags behind the forty-nine previous ones. 98% of all tokens are a rereading of what has already been said, that is, a cache. Two-thirds of the bill goes there," explains Denis Klymenko in his post on LinkedIn.
Based on an analysis of his own work program (30 sessions, 857 million tokens with a total check of $582), the engineer makes the following observations:
The Opus model costs $0.121 per request in a long session and $0.078 in a short session. The difference is 36% for the same quality of work;
sessions with a context volume of over 300,000 tokens accounted for 58% of the total cache read bill;
The most resources are consumed by command outputs in the terminal (36–42%) and read project files (another 28%).
"A session starts with 40,000 tokens (global and local rules, skills, memory), and peaks reached 405,000. Once you dumped a long log into the chat - and you pay for it on every subsequent request until the end of the session. There is no "remove this from here" button, the only way to get rid of the excess is to close the session," notes AI Architect.
A new window for a new topic: for questions not on the main topic, it is worth opening a separate session, because Claude sees all parallel windows.
The specialist concludes that a working session in which the total context volume does not exceed 200,000 tokens is ideal.
