Sensible usage of your tokens

Sensible usage of your tokens

The pricing model for GitHub Copilot has changed from Premium Request Units, which had a fixed fee, to Usage Based Billing which is a pay-as-you-go model where you pay for each token usage. That means that now it is more important than ever to keep your token usage under control.
For tracking your usage you can use the extension we built for VS Code / Codium based editors / Visual Studio / JetBrains: https://github.com/rajbos/ai-engineering-fluency. There is even an NPM package for it, so you can use it in your own scripts and in CLI status lines where supported.

Model costs

First of all it is important to understand the different costs per large language model, the differences between providers, and how the different models work. We now pay for "AI Credits" which is a sum of input + cached + output tokens, where the output tokens are the most expensive as they have to be generated by the models. For the reference tables see the Official GitHub docs. These numbers are used for reference by the 1 million tokens, for example:

Some models have different modality options between them, most of them support images as well as text. Adding images to your context can be very costly: each pixel needs to be sent to the model provider as a binary representation, which will then be translated by the model to describe in text on what is visible in the image. Keeping your images relevant to the conversation to show just the parts that are needed, can be very helpful. So do not grab your entire screen in full HDPI mode and send that to the model, but crop it at least to the relevant part.

Defining your thinking effort

Most models allow you to select their thinking effort. Thinking is part of the generated output and is therefore in cost (compared to reads and caching). Only use medium or high reasoning on tasks that require the agent to figure things out, or where you want an agent to think through a plan. Thinking mode is generally used to let the mode reason on the prompt, define the problem, or compare the prompt and plan to what it thinks it would do to solve the problem. So the flow would be:

Switch to a more lightweight model for the execution of the plan, and use a lower level of thinking here. This prompt does not need a new reasoning cycle, as the plan is already made, so you can save on the thinking costs.

Caching
Some model providers provide caching options with different price points.
See the docs here:
[Copilot billing - models and pricing](Copilot billing - models and pricing).
OpenAI - one price for cached input
Google - one price for cached input
Anthropic - different price for cache write vs cache reads

The hard part is that you do not control when the cache is used, that is up to your editor. Next to that GitHub does not disclose the cache setup, that is up to the model providers. Anthropic indicates a 5 minute cache, which would mean that you could tune your session to be present and continue your coding flow within the cache period. In reality that is hard to do, and not predictable: your editor might decide to inject other tools in the session, thereby breaking your cache. So this is not reliable. 
What we do notice is that GitHub Copilot does use a lot of the cache, as you can see in the numbers in the screenshot of my own usage:

Context sizes

Same for different context sizes: using a longer context size with a longer session can mean more tokens are being sent to the model, and thus more input size costs. Use it where you really need it.

Look at Claude Opus 4.8 for example, where you can choose between a 200k context size and 1 million: 

If you choose the 200k context size then compaction could be needed much earlier in your session, resulting in more cache invalidation. So potentially with the 1 million context size you can continue longer in the same conversation with more cached context, saving input tokens in the long run.

Be sensible in adding too much context

In Ask Mode your editor decides what files to send to the model as context. That can be your open files, files your cursor has just visited (jump to definition for example), and also bigger tasks like @codebase or #problems, especially if you have a large codebase. You can add entire folders to the context, but all that increases your input size and thus input tokens. This is the easiest part of the token cost to keep under control. 

In Agent Mode the agent will figure out what to do by itself. So refrain from giving it vague prompts like: there is a bug in my 5000 line file, find it and fix it. Or adding your 20MB log file to it and let it search for the line with the error. Often the current tools will just read the file in chunks of 200 lines, all consuming your context window. 

Tip

Be smart in what context you add, and be more diligent about it. Add only the parts of the files you need (#selection), or the two lines that show an issue in the logs.

Customization

An important part of your chat session is what the editor sends as input tokens. Even before your user input starts, your session is filled with:

You can tune the tools that you use per session, and that can be worth it. A big tip here is to also customize your instructions at the repository level so that you do not end up with a very big and descriptive set of instructions on the machine level for example covering all of the different technologies you work with, but instead have more focused instructions per repository, or even per folder. That way you can keep the instructions concise and relevant to the work you are doing, and save on input tokens.

Let's dive into some examples below.

Custom instructions

Custom instructions are a great way to tell the model about your codebase and how you work. But they can be very costly as well, especially if you add a lot of instructions. Custom instructions are loaded into each conversation, so having a lot of instructions can add up to a lot of input tokens. Places where the instructions can come from:

For all custom instructions: keep them in a sensible scope. Do you really need to tell every single session about the way you implement tests? Even for readme file changes? You can use different instruction files based on folder paths and file extensions, so make use of that for specific things. Read the docs here: agent-customization.

The general recommendation is to keep your instructions as concise as possible. Often 500 lines of instructions is seen as a lot.

Customizing MCP servers

MCP servers are loaded into your conversation for every session, with their entire tool list and description of when to use a tool. The tools are then available to the model. The model will always try first to see if the prompt is linked to any of those tools. If so, it will send back the response to the prompt with the tool name and discover tool parameters from the prompt. The editor then has to execute the tool, and send the result back to the model again to get a natural language response. That means your tool title, expected parameters, trigger text and description are always send to the model (once per session), racking up input tokens. If you are only working with your own code and you do not need to e.g. call into Figma, or learn about the latest state of your GitHub issue, then you can remove those tools from the conversation and save on input tokens.

Here is an example of the tools that are loaded into the conversation on my machine before I looked at its setup: 

That means that every session I start, will consume around 24k input tokens for just the MCP tools definitions! On top of that the system prompt is added, and the context your editor wants to add to the session (open files, selected code). Only then is your user input added!

So being mindful on the work you are doing here is important: if you are just working on your code, disabling all MCP tools might be very worth it. And when you are done with the work and want to create that PR through an MCP server, enable it (switch to a new session to leave the work process behind and move over to PR modes), then create the PR with the tools, and then disable it again when you are done.

There are a couple of side notes to this:

If the MCP server is not started, then the tools for that MCP server will not be loaded and sent to the model.

Changing the tools you give to the model in the middle of a session will break your cache, so that is not recommended. If you want to change the tools, do it at the start of a session, and then keep it consistent for the rest of the session.

Tools can also directly be loaded by VS Code Extensions, without the need to expose an MCP server.

Several tool providers have been working to limit the amount of tool definitions they load into the conversation. For example, the GitHub MCP server used to have separate tools for GitHub Issues: Get, List, Edit, Delete, Create. Now it has one single tool for GitHub Issues, and the model can figure out from the prompt what action to do with the issue. That is a great improvement, and saves a lot of input tokens because there is less tool definitions to send over the wire.

Skills

Skills are little descriptions for the model to understand when to use a skill. When the skill needs to be triggered, then the extended skill code is loaded into the context and executed. So the benefit of skills is that they are not loaded into the conversation as tool definitions, but only with a short description (this can still be several lines of text). So where the entire list of all MCP tools (hundreds) is always loaded into the conversation, with Skills only the skill description is loaded. The other benefit of Skills is that you can actually have it run code on your environment, and then get deterministic results back.

Skills can also come from different locations, like the extensions installed into your editor (VS Code supports that), or from files in your own codebase. Pruning their descriptions, as well as checking if they have been used at all can save you input tokens as well:

Side notes:

Everyone in the industry is searching for better ways to handle this. Here are some things that we noted from that.

Talk like a caveman

See a reference here: https://github.com/juliusbrussee/caveman. The idea is to tell the model to stop all the niceties and just be brief, or "like a caveman". That indeed shaves off some of the response tokens, and they are the most costly. On the other hand: output tokens tend to be a fraction of your input tokens (for code), so it makes more sense to focus on the input tokens.  

Adding proxy tools between the agent and the current tools

There are a lot of tools that try to become a proxy between the agent and for example your shell output: often the agents are smart enough to use tools in your shell like ls or grep to find things, and by default those might send back more data then needed. Think for example list all the files in a directory when searching for something specific: do you really need all the file attributes and timestamps to find a file? And what about all the formatting info that the model in the end does not care about.

There are several tools created for this specific problem, one of the most known is the Rust Token Killer (RTK). I've heard mixed results for some of those, so test and verify if they actually save you tokens without compromising the quality of the output. An option that GitHub Copilot has setup for this inside of VS Code is to compress the output of the tools differently before sending it back to the model. You can enable this in the settings with chat.tools.compressOutput.enabled (in preview at the time of writing).

Microsoft AI Engineering Coach

Microsoft AI Engineering Coach is also a great resource for learning about how you use AI in your engineering workflow, and they have some great tips on how to optimize your token usage as well. Check it out here: https://github.com/microsoft/AI-Engineering-Coach

AI Engineering Fluency

I built a tool to give you insights into your token usage, which also provides tips around your tool configuration and usage. Find it from the repo here: https://github.com/rajbos/ai-engineering-fluency