Name: Spend Smarter: Optimizing Copilot Tokens Without Slowing Builders
Uploaded: 2026-10-07T17:02:15.227Z
Duration: 58 min 3 s
Description: Spend Smarter: Optimizing Copilot Tokens Without Slowing Builders

Transcript for "Spend Smarter: Optimizing Copilot Tokens Without Slowing Builders": Hi, everyone. Thank you, Austin, for the warm welcome. Yeah. As Austin mentioned, we're gonna be talking a little bit about, Copilot. How do we make sure that we, take care of cost, optimize our token consumption? We'll go through all of that, and then, you know, I see that a lot of people, they're, you know, asking in the chat and in the poll, how do you balance cost with developer velocity? So we'll go through, you know, what are some practices around that and, you know, how do you get more visibility into usage patterns and, yeah, we'll kinda go from there. Awesome. Cool. So thank you everyone for joining. Today, we're gonna be doing our session on how do you spend smarter, optimizing Copilot tokens without slowing builders. So I just wanted to see, and you can enter it in the chat, like, how many users... Like, how many of you are currently using Copilot, and, you know, what are some of the things, like, use cases that you may be using it for? You can put that in the chat while I kinda present and go through some of our topics. But as you know, my name is Rizvan Rafiq, solutions engineer here at GitHub, and, my cohost, Austin Becker, he's a senior solutions engineer, great guy, and super smart. So he'll be able to answer any of your questions in the chat. Cool. So we'll dig right in and dive right in. So when we talk about cost optimization, when we talk about Copilot in general, there are a lot of things to keep in mind because not every prompt, not every use case is the same. What we always want to do and what we wanna try to do is we wanna optimize it for accepted work and not just try to create the shortest prompt. So what does that mean? Like, what's the shortest prompt? How do we optimize it for accepted work? First, what you can do is control the rate. What that means is that if, let's say, that you have a task that is similar to, let's say, opening a file or opening a folder and looking at those contents, you wanna match the model capability and reasoning for that task. So what that means is that, you know, you don't wanna just use the latest and greatest model to, do, like, a smaller task. That's where maybe a more efficient model really makes sense. And using auto where supported is one of the best things that you can do because not only does that, automatically look at your context and figure out, you know, which model may be the most efficient, but it also gives you a discount whenever you use the auto mode with GitHub Copilot. So something to keep in mind for sure. Second, you wanna reduce the wasted work. So what constitutes wasted work? So let's say you wanna you wanna essentially, like, trim any sort of context that may be irrelevant to the task that you put in hand. You wanna make sure that you have a verbose output. What that means is that you have a defined acceptance criteria and what you actually want to get out of your output. And you wanna make sure that you kinda take out any, unnecessary agent loops. And we'll go through all that and kinda show what that looks like. Pretty much for a quick example of this, if let's say that you were working on a task where you're trying to fix a bug that is, you know, plaguing some of the, some of your production workflows. Let's say that the bug is happening only in one of your environments, but for your Copilot prompt, you know, you're saying, hey. Fix this bug and, you know, test it in every single environment. While that's good that you're wanting to test it in all of the different environments, if it was scoped to just one environment and you wanna specifically check, you know, what might be different there. For Copilot and for making sure that the work is, scoped to where you need it to be, that's where, you know, kinda giving it a verbose output of saying that, hey. I want you to check it in, let's say, the staging environment and getting my, output for that staging environment. I want the solution to that specific piece of code. This way, what happens is that it's not trying to learn anything and everything from all of your, environments. It's being scoped just to that environment and just to where the bug is appearing. Cool. Third, you wanna do a first pass quality. So as I mentioned before, you know, providing acceptance criteria, maybe giving it a working example, and getting fast feedback from tests. This really helps Copilot and just agented workflow in general to understand what it is that you are trying to get from an output. So for example, if, let's say, that you have a prompt, to fix a bug, maybe your input could be, hey. My bug started on x date. This happened because of a recent change that happened on pull request number three. If you give it all of those, context and then you say that, hey. For this bug, I needed to stop doing duplicate entries or, you know, I need the bug to not, basically, my expected outcome is that instead of, you know, all of the different environments, not catching or not having a particular, pretty much, like, particular outcome. I want this bug to be tested on the different, environments and basically give me the right outcome each time. I know that was vague, and that doesn't help too much with the specific context, but we'll go through a demo later with something where, we kinda see that in exact practice. Cool. Next. How do we know what actually consumes AI credits? So we can kinda break this down broadly into what surface areas you use and what's actually included without AI credits. So what's nice about GitHub Copilot is that code completions and next digit suggestions, that are on paid Copilot plans, that's already included without AI credits. So let's say that you have Versus Code open and you're making use of code completion that it generates, that's already something that's available to you at no cost right now. Where you do consume AI credits is when, let's say, you're using a GitHub chat session or you're using the GitHub CLI, if you're using the GitHub Copilot app, any sort of third party coding agents, Things such as that will consume AI credits. Now what's an AI credit? This is basically our way of billing. So what that means is that you can think of one credit equal to pretty much 0.01 USD. So let's say that you have a, you know, 100 credit, session, you would be utilizing a dollar worth of tokens. If you're on a Copilot business plan or if you're on a Copilot enterprise plan, each seat comes with its own token allotments. So you get a specific amount of tokens before the usage actually starts. So right now for business seats, it's 1,900. For enterprise, it's 3,900. Next. We talked a little bit about prompting. We talked a little bit about billing. So why is the prompt only being a part of the token bill? This is where things such as input, cashed input, and reasoning really comes into play. So you can think of, as it mentioned down there, the task usage value. It's It's really the sum of, you know, what's your billable token category and its model rate. So as we know, you know, there's all different types of models that we can use. You can use the latest and greatest like GPT six Astra. You could use Opus 5.5. But, you know, those models are gonna be maybe a little bit more expensive for some of the work that you're doing as they're new and, you know, they're kinda getting the bearing of what's going on. That's where if you use, you know, more generalized models, like maybe Opus five point o or maybe you're using GPT six sol. Those models are a little bit more efficient in doing smaller tasks, and that might be something where, you know, your costs will be more, more optimized for the session at hand that you have. Some things else to keep in mind is fresh input. So if you have, a prompt, if you have instructions, and if you have relevant, like, files and tools, all of those matter for when you're going in and, you know, starting a session. Essentially, what that means is that when you start your prompt, the instruction guardrails that you have and all of the files and tools that you input for the token session, for the for the Copilot session, they all can help reduce your cost and also make it so that, you know, the context of what you're trying to get is more focused around what you're trying to do. What's nice about that is that let's say that you have multiple prompts or multiple sessions, well, you'll have cached input. So it'll be reusable context that the model is going to be able to use, and it's gonna be billed at the cache rate. And we'll go through a little bit of what billing looks like. I have an example of it, for... I believe it's GPT 5.4. But then, you know, we'll we'll kinda go through what that looks like and what really is the value of caching. Then what's nice is that cache creation cost is where the model builds it. So the cache will write, and that will give you, you know, reduced rates, and it'll go ahead and generate output. So the answers, what did the model work, what reasoning level did you use, all of that go into play when, you're looking at, you know, what are some of your costs. So the four takeaways is that fresh input, any input where you give it instructions, relevant files, and tools that matter, the type of prompt that you're giving, whether it's vague or whether it's scoped, that matters. Anytime that you can make use of cached input, that reduces your bill. Cash rights are sometimes a good thing, sometimes not, and we'll go through that, what that means. And generated output so that the answer and the model, the amount of work that the model did, with what reasoning level that helps kinda make that prompt and that piece of, your Copilot session build. Next, as I was mentioning, right size the model before trimming the prompt. So what I typically do and what I kinda see, a lot of our solutions engineers do, we just have a default auto slash balance for everyday work. So when I say auto slash balance, I'll open up Copilot, the GitHub Copilot app and show you what that looks like. But think of them as, like, different modes for when you wanna actually run a session and get some output. So if you were trying to work on some docs or maybe some simple tests, that's where the efficiency or the lightweight models really come into play. It is, easy work that it can do with, like, a more efficient model, so you don't have to use too much reasoning or thinking. Where you might wanna escalate is if maybe the correctness is not the best, or maybe if there were some edge case... Edge cases where you need a little bit more deeper reasoning. That's where you could maybe use auto slash balance mode. So what auto slash balance mode is is that it's a little bit more versatile. So if it needs to, you know, make use of a model that maybe a little is a little bit smarter, it'll be... Go ahead and make use of that. Some of the typical work that we see for that is, like, if you have features or any bug fixes, that's where, you know, people usually start out with. So whenever you start out there, you know, if you don't get, like, the right output or something you were expecting a little bit more, then you can escalate it and make use of auto slash intelligence mode. This is the most, you can think of it as, like, the highest tier and the strongest reasoning. So anytime you're working on, like, architecture or anything where you're doing complex debugging, any... If you have, like, high risk design decisions, so something that might affect prod or something that might affect, you know, kind of the way that your... The layer of your land, essentially, This is where you can use a focus plan, before broad broad implementation and then make use of the intelligence mode to essentially get the right outputs that you need. And let's say that even auto mode is not what your, is not giving you the right desire, that's where you can make use of a specific model that you may wanna use. So this is where your GPT six Astros come in. This is where your Opus 5.5 comes in. Kinda gives you a, you know, more, I guess, more control over your output and gives you a little bit more of what what you can do. Awesome. So next, let's talk about giving the agent a map, not the whole library. So as you know, I'll go through a little bit of what agents are, what's... Like, in terms of library, what does that mean? So you can think of an agent as what is actually doing the work. You can think of them... So let's say that you're in a you're in a kitchen. You can think of the agent as a chef. So a chef can do a few things. A chef can cook. They can, give out orders to, you know, other people that may want to, you know, finish your dish. In this case, creating a dish or finishing the dish, that dash would be kind of like your prompt and your output. And you can think of, like, you know, the kitchen staff and how everybody's specialized. You can think of the... Think of those as different agents. So when we say give the agent a map, not the whole library, what we're saying is that, hey. Tell the cook specifically what the cook needs to do. Tell maybe the waitress what the waitress might need to do. Tell the preparer maybe what they need to do. So if you kinda scope it to what's kind of your goal and your expected behavior, This really allows the agent to preserve your context and, you know, tailor make it for what you are actually wanting. You give it the relevant paths, symbols, and interfaces. What that means is, like, let's say that you only want, actions being done on a specific set of files in your repo. Maybe give those paths, where it should look at. Or let's say that you have a big monorepo and you want to, you know, like, only touch the source folder and only certain folders underneath it, but you also wanna give it the context of your actual architecture and you have it in a separate file. If you give those paths and you give those, you know, understandings, it really gives the agent more scope, more relevant scope, and that allows it to give, you a better output. Something else is that if you give it a... Maybe a small reproduction of what you're what you're seeing, if it's a bug or maybe, of what you're expecting, that's something that's really good to do. Or let's say you're debugging maybe a little bit of the error excerpt. Right? Rather than just saying like, hey. I got a four zero four. Like, where did you get the four zero four? Did this happen on a specific time? Can you give me the stack trace of what that looks like? Things such as that really give good context and allows the agent to give you a scoped answer. Something that you may be, wanting to do is remove avoidable noise. So what does that mean when I say avoidable noise? Unrelated tabs, attachments, and history. So let's say that you were working on multiple sessions and, you know, you're working on a session to fix a bug, and then the next thing you do in that session is say, hey. I want you to create this whole new feature. This has nothing to do with the bug. Well, all that history kind of, you know, doesn't help the agent. It has to then now relearn, or start fresh, but it has all of that context within the history. So if, let's say, something is similar to that, it may it may think that, oh, okay. Now I need to make use of what I did for the bug fix. So things such as that, you may wanna know it, route to... Remove the noise. If you have, like, full logs, any repeated search results, those things typically don't really help as, you know, if you're trying to look look for something specific and you know what it is, then giving it the entire log doesn't really help. It just kinda clutters out your context window. So that's something that, you know, you may wanna take a look at and maybe reduce. Generated files and duplicate documentation and conflicting or obsolete instructions, both of these are important to kind of, re... Like, take away from your scope because what it does is that it gives it more, not necessarily, like, specific context, but it gives you more generic ideas of what, you're trying to get from the session. Where it doesn't make sense is when you have, when you have different files and different instructions conflicting with what you're trying to do. So let's say that you have instructions that say, hey. I want you to use node 18 for my tasks, but your current text or your current stack is working entirely in node 20. That's where those instructions really kind of conflict and make it a little bit harder to understand what goes on. So we talked about giving the agent map. We talked about, you know, so far, like, how do you control some of the costs, what are good prompting, and how does that help us, with, you know, getting the right output. Here's a quick example of what a better prompt, can do and how it can prevent expensive retries. So, for example, an efficient prompt may be a little bit longer, but, essentially, what it does is it gives you a focused idea of where you're trying to get to versus something being vague. So if I, you know, told my Copilot app, hey. Fix the checkout. Make it better. It may have my repository open. It may know, you know, what, where I'm working at, but this doesn't really tell it much. Right? It kinda talks about maybe a feature that maybe to you, it's fixing a checkout. But to Copilot, it's just looking at, hey. That's feature number one. Or it's looking at it and saying, oh, okay. That's the... That's part of the, you know, checkout feature. But how do I make it better? I could maybe, you know, make it be red instead of green. But you're not telling it what exactly it is to make it better. Where if you had a focus prompt, you can say that, hey. What is my goal? I'm trying to prevent duplicate charges on checkout retries. Where do I start? So where... Which folder or files am I going to start at? And how do I wanna actually, you know, make use of what is already there and make it better. So for example, in this case, you wanna reuse the existence, idempotency helper. You wanna preserve the API and just add a test. Then what do I wanna do? I wanna show, like, what is it that I'm trying to test the output of. So... Or how am I trying to test the output? Sorry. That's what I meant. If, let's say, that you have a test that you're creating, so, like, a regression test for a repeated request ID, how do you wanna run it? So you were gonna run our NPM test. Then what do I wanna do is if it's blocked, explain why. Otherwise, stop the test when it passes and summarize the change. If you give this prompt a... Like, if you give your prompt a focus goal and output a narrative, this is where it really comes into play, and it gives you the best output that you might require for that task. So one of your prompt recipes... And, this is just kinda going back into, you know, kinda that chef example that I was giving earlier. What you want to do is you want to have a goal. You want to have context. You wanna give it the constraints, in this case, which which is instructions. You wanna give it, an acceptance test, and you wanna give it a stopping condition. This goes back a little bit to our previous slide where we talked about, you know, unnecessary agentic loops. A stopping condition is a great way to prevent it from, you know, going and retrying all of that, retrying all of the things it has done and then going from there. Awesome. So what we can do now is I am going to share my screen. And so we talked about a lot of this, but what does that look like? Right? Like, how do I actually... Like, what is auto mode? How do I go in and take a look at it? What what does that actually mean? Let's go ahead and share. And give me one second. Let me get my Copilot app up. Cool. There we go. So now where we're gonna take a look at this is the GitHub Copilot app. If you guys are unfamiliar, GitHub Copilot app is the desktop app for for GitHub Copilot. It helps you, you know, do any sort of, agentic work, allows you to do automations where you can have things kick off every day, allows you to make use of MCPs, plugins, connector skills. Really, it's just a super nice way to do any sort of agentic work and allows you to, have one place where you do all of your work. Cool. So we talked about prompting. We talked about cost optimization. We talked about how we can make use of, you know, different models, but what does that look like in practice? So here, I have a demo repo that I've created and loaded it up. With the Copilot app, one thing you can do also is that you can actually load either a GitHub repository and added context to it, an ASDOH repository, or you can just open a local folder and start working, and it will have all the context of that folder. So here, I have my demo repo, and I have my files opened up. So Copilot instructions, you can think of this as kind of, something that is a guardrail. So as we talked before, giving specific guardrails as to where your files are, how it works, that really helps Copilot with tailoring your... Tailoring the solution to, what you might need. So in our case, you know, we give it the file here. What are my files? What's the actually... Which methods actually are doing my functions? Where are my tests? What versions do I wanna run it with. Things such as this really help guide Copilot and reduce the amount of, tokens that you actually utilize. So this is something good to have as always. Next, we wanna talk about model selections. Right? So let's say that what type of a mode am I gonna be in? This is something where you can think of this as, you know, whether you want it to be a human in the loop. So whether you wanna actually, say yay or nay to Copilot as it's working through the problem, this is where you kind of have that hand. You can plan first. Typically, where we would use plan first is if you have, you know, a big task or something where you're making architectural changes or something that will affect, you know, your your, pretty much like your environments. Things such as that, you may wanna plan, before you actually do the execution. Then we have autopilot. This is where you can come in and just have the agent do all of the work, prove everything is needed, and go from there. And I wanted to talk about tool permissions as well. As I mentioned before, having the correct tools and, like, giving the right kind of context with the tools really helps, make your prompts and your sessions more efficient. So So one thing you can do is you can either approve all tools without asking, or you can have it always ask or have it be assisted. This is particularly a feature of the GitHub Copilot app. So, what this looks like in maybe a different surface, might be that you have to select the tool. But, you know, this is something that is very beneficial to do as you do your prompting. Next, I talked a lot about auto mode and what that looks like. So here on the Copilot app, what you can do is you can select auto, and this this, works the same way if you were to do it on, you know, your GitHub Copilot CLI or if you were to do this for, in Visual Studio where you're making use of the GitHub chat. Pretty much what this allows you to do is you can optimize it for either efficiency, balance, or intelligence, and you can see the difference, between the three and what they kinda target for. So when you wanna optimize for cost and speed, that's where efficiency kinda comes into play. So think of, you know, smaller models that can, and it's making use of models that are more generalized. Balance, you can think of something where you need a little bit more, help because your task has a little bit more complexity. That's where that kinda comes into play. Intelligence, you wanna optimize it for capability using faster models when the task allows it and so forth and go from there. Then what's nice is within the Copilot app, you can also see what your usages. So you can see what plans you're on, what session. You can use this to go ahead and view the usage and and plan. Then, we talked about vague prompts versus what does a scope prompt look like. So let's say that I have a vague prompt, where, you know, I said checkout is better, go from there, make the checkout better, and fix it. Well, rather than that, if you scope this right, and give it more specific instructions, it really helps eliminate, you know, how much cost you might take and give you a better output. So for example, here, you can see in this demo repository, I'm saying I wanna fix the sequential checkout retry bug. So I'm telling it which folder I wanted to go at, which files to look at. I'm telling it, which tests define the actual missing behavior, and then I'm telling it how do I want it to actually implement the fix. What does that look like? Then what I wanted to do is I wanted to do the test, and then I'm also telling it, hey. What are some things that I don't wanna do? So giving me guardrails as to what not to add so that it's very specific... Is very specific to my, fix that's needed. Things such as this really help with your, with your prompting and also with your costs. So if I were to send this in and I have it in efficiency mode, let's see how much of that actually, uses up. Cool. What we'll do... So as that goes and is working, you... We're gonna switch back a little bit, and I will go back into the presentation. Awesome. So now if I am going on to the next slide. Cool. So we talked a little bit about the different modes that we have and how do we separate the discovery from implementation. So we wanna use deliberate phrases for com, complex work. So we wanna first discover. So identify the relevant files, contracts, what are some of the reproducible failure, and where do we wanna stop. We wanna plan. So what's the smallest change that we can use to go through and plan out what the work is? We wanna implement. So we wanna have, like, a bounded plan, what exactly we're gonna do, how we're gonna implement it, and then we wanna verify. So things such as, you know, running tests, things such as reviewing the diffs, all of those things will be, something that you can do. So if there's, like, a few things that I want you to take away just from this slide, it's you wanna discover, you wanna plan, you wanna implement, and you wanna verify. This kinda holds true for any sessions that you go over. Cool. As we mentioned before, you know, how do you preserve the useful context and cache? You wanna reset deliberately. So as I mentioned, if you have the same bug and prior evidence still matters, you can continue within that session. Preserving the context actually helps with using your cached input, so you're using less tokens. But let's say that you have... You're working on the same task, but there's too much noise before, and you would wanna just kinda start fresh. This is where you can either branch off and compact and then verify, you know, what maybe the handoff summary might be. So what that means is that you wanna verify what actually happens after you compact it and go from there. And the reason why is because, you know, like, you wanna reduce the future context and actually just keep the decisions that you wanna work on and go from there. I see we have a question in the chat. So it says that, hey. I... And this is from Ganesh. I conflict internally about handout versus plan, execute switch within the same session. Which is better token usage wise in two scenarios when the changes are small and when the changes are large? Okay. Great question. And this kinda goes into the slide. The reason where... The reason why one is not better than the other token usage wise is because let's say that your changes are small. If you are working on a... Let's say that you're working on a task where you're implementing a new feature. The feature change is small, and you wanna make use of, let's say, like a Microsoft model, like a MAI model to do that change. In that case, you know, your model choice and the change that you're trying to make, really, that's where the token usage might be a little bit more, wise in that scenario versus, let's say, that you're doing something where it's a large large change, and you're still using that same model for the Microsoft MAI model, but maybe it's not as efficient with larger tasks, that's gonna use up a lot more tokens, and it's gonna use up more context than needed versus if you were to use a different model, such as, like, maybe Opus five point Opus 5.5 to do that task. Model choice really kinda comes into play whenever you have... Whether you're trying to do a handoff versus a plan. For doing a plan, that might be a little bit better with, like, a more versatile model. But if you're doing actual handoff, that's where, you know, using, like, more, intelligent models really come into play. And what if you're in auto? Great question. So in auto, depending on the mode that you're in, whether you're in efficiency mode, whether you're in intelligent mode, or whether you're in cost optimization mode, it will pick those models for you. So auto kinda takes away that reasoning, that you might have to do and, you know, pick one versus the other. That's where auto kinda comes into play and picks it for you and knows the context a little bit better. Cool. And then I think, Rizvan, you had a question. In an agent tick workflow running on a sandbox environment, how do you enforce permissions and keep engineers in the loop? Awesome question. So let's say that you're in the cloud session and you wanna enforce permissions. And I'll actually share my screen, and we can look through what that looks like. Cool. So let's say that I go into my organization, and you're specifically talking about if I'm in a GitHub enterprise session. Awesome. So if you're here within a session, how do you, like, how do you essentially... Yeah. How do you keep everyone in the loop? How do you use auto? This is where, as I mentioned before, you still have auto mode here. This is able to select what you can do, what type of work that you may wanna actually, create. And then for these sessions that you have on the, GitHub Copilot agents, let's say that you want something where, you know, you want an agent to not, go to a next session or not do any more work. What you can do is say that, hey. I want you to implement fix number two for this repo, but prior to doing the work, get an approval for me. If you give it this context, it won't do the actual full work. It'll wait until you go into that session and give, you know, your input as to whether you want the work to continue. You can also make use of configurations here. So if I go to configure the cloud agent, You can actually give it some instructions that you wanted to use, and I think it's actually for... Yeah. Okay. There we go. There's different, effort levels that you can use, and you can also use custom instructions, whenever you're either doing code, like, pull request review. And then those custom instructions, you can also make use of during the cloud agent. I forget where exactly you put the custom instructions, but, yeah, that is something that's available to you within that context. Awesome. Then what we can do is I'll go back into my slides, and we should have a couple different slides that we can go through. But pretty much, as I mentioned, this is just going to be a little bit of a recap. So we wanna make repository instructions small and specific. So we wanna give it the scope of what we're working on, what are some things that we wanna do within each one of our sessions, whether we wanna run the targeted test, whether we don't wanna edit generated files directly. Things such as that is really important for your context. Tool output, a delegation are not free. So kinda similar to what we talked about before. You wanna retrieve precisely, so get the relevant scope of what you're trying to work on. You know, make sure that you're not giving every single log item as needed. Limit the tool set for what you're trying to make use of and delegate selectively. So either using, you know, smaller subtasks that you wanna do with a token that you... With the agent that you handed off to or use a per... Like, use a lighter model when it is sufficient and needed. Then use a deterministic feedback to avoid token spirals. So this is where things such as giving a specific expected behavior, requesting a small diff, running a targeted test really makes sense versus, having to pause repeated failures because, you know, you're not getting the right output that you want. Capturing the, you know, like, capturing the exact failing evidence. Things such as this is where you wanna, implement a little bit more tighter feedback because it helps, it helps really give your output more context and more efficiency. So if there are certain things that you wanna take out of this, it would be that you don't wanna remove tests, human reviews, or security constraints to make something look a little bit cheaper. You wanna really specify it to what you're trying to do and what makes sense for your cost. Cool. And then we didn't talk too much about the GitHub CLI, but these are different commands that you can make use of, that help kinda reduce the amount of, tokens that you may be using. So context inspects the current context footprint. What it also does is it takes out any context that may not be relevant to your current session. Compact, this focuses on any sort of fix. So anything that where you need a fix, it actually summarizes it and makes it relevant. And then you have a couple more commands that pretty much, like, allows you to control how much budgets you're using and pretty much your spend. Then this is just a simple example of where if you have less wasted work and use the same model, this allows you to see kinda what that cost might potentially look like. So as you can see, if you have an unfocused task, that can actually, raise up your usage and make it more expensive versus if you have a focused task that actually allows you to reduce the amount of, you know, tokens you may be using. As I mentioned before, input, cached input, and your input really matters. So that's where, you know, if you were, kinda giving it the right context, it really helps with your cost and your controls. Cool. Next, what we wanna do is talk about governing the pool. So this really comes into play with your budgeting. We have different budgeting that's in place, and additionally comes into play for enterprise admins. If let's say that you wanna, you know, have, like, user level budgets, this comes into play, for each one of your... Each one of your people that have Copilot. So what it governs is each person's total consumptions, whether it's included, you know, AI credits plus additional usage. This always has a hard stop. So what you can think of it as is that you have your individual level user level budget that comes into play first, then you have your cost center budget, then you have your universal. So, basically, the priority would be that if, let's say, you have an individual budget of 500 or more, that takes precedence versus if you have a cost center level budget of 200 versus if there's a universal user level budget of 30. So because your individual is the most, it will always take precedence and you'll have that amount. Then you can make use of cost center included usage. This really helps with scoping down, how much usage you have within the organization per user. Gives you the flexibility to have, you know, like, the same pool of tokens being used within all of your users. Cool. And then, this is just talking about best practices for headroom. So you wanna align user headroom with the team's spend ceiling. So what that means is that if you have an included pool of credits and you want additional ceiling, you can enforce that with budgets. But you wanna have a combined capacity and make sure that, you know, you're giving the users as much, as much budgets as needed for what they're trying to accomplish. So you don't need to have a $30 per user limits for every user. You can make use of individual ceilings and, you know, go above or below as needed. You wanna measure savings without rewarding worse outcomes, so these are some of the things that you can do. And I know we have a little bit of time, so I wanna say, these are the six habits to adopt tomorrow. You wanna have the right size capability. You wanna define what done looks like. You wanna keep the context relevant, preserve your useful cache, verify, you know, any sort of small disk, then stop, and you wanna measure and govern. And if anything, that's what I want you to take out of, this session. Awesome. And that's pretty much it. Thank Thank Yeah. Thank you, Rizvan. Great job. you, There were... And and I know that we're we're right at time, but. there are just a couple quick questions in the chat that I think you could. easily answer really fast. So we'll go through those real quick. I don't know if you covered Vignesh's yet, but it says in an agentic workflow running on a sandbox environment, how to enforce permissions and keep engineers in the loop. So that's the... Kind of the explanation I gave you on the GitHub CLI, the. Okay. cloud system. So that's kinda where that falls into play. Awesome. Awesome. Okay. I just wanna... I didn't know if... I must have missed it because I was... I said, sat down through some other ones. But the other one. is... Hey. Yeah. Vignesh recently asked, I can't find the slash limit option. Is that a plan feature for a future release? Mhmm. So that's a great question. So the slash limit feature, I think you have to have preview enabled. So it's it's something that is currently available for you, but, yeah, it's coming up, and that's something within the pipeline for the CLI. Also, Yep. something to make use of is you really wanna make sure that, check what version of the CLI that you're at. So... Because a lot of the times, it's very version specific. So if you're on, like, a older CLI or something like that, you know, you may wanna take a look at that and update your version to get the latest and greatest. Yeah. And there's... If you're using the CLI, just kinda wanna call out. One, you have to have preview features enabled at the policy level, but then the second thing is there's a lot of times a slash experimental space on command. And I think. somebody asked about HydraFusion earlier. So in order to use HydraFusion, it's the same thing. So a lot of, like, our cutting edge stuff, if you want to try those out... I mean, with experimental features comes a lot of risks. So it could break or hallucinate, but just be aware if you're not seeing specific commands, it's probably because you either don't have the preview features on or you're not in experimental mode. So... I'm Yep. And I see a question here from... Hopefully, I don't butcher your name, but, Celestia Dhanaraj. But pretty much it says, does GitHub in it... Does GitHub integrated with Visual Studio offer the same capabilities as the GitHub desktop app? Great question. Short answer is there's a lot of similar features, but the Copilot desktop app is a lot more integrated with your daily workflows. So your Copilot desktop app has more features that are available to you. But what you can do with GitHub chat session in Visual Studio is that you can actually make use of, you know, auto mode. You can make use of your prompts. You can make use of, you know, your different plugins. So if you wanted to use MCPs, you wanted to use, different skills, all of that's available to you. Where the GitHub desktop app really shines is that you can make use of canvases. You can make use of automations. You can tie in different tools, different plugins. So let's say that you wanted to work with Azure DevOps or you wanted to work with Figma designs. All of those things, you can natively pull in to the GitHub app, and then you can make use of it in, like, a visual sense. So where it kinda differentiates is that you can think of give... GitHub on Visual Studio for developer experience and doing, like, developmental work, where you wanna be hands down and you just wanna use Copilot as needed. And you can make use of the GitHub desktop app for any sort of your non developer workflows or any workflows where you wanna visualize a lot of the work that you're doing. I know for me, Austin, you may chime in as well, but I use the GitHub app a lot for visualizing, like, if I have customers that I'm going to talk to, visually put them up on a canvas and see where those customers are at. Do I need to follow-up on any tasks? It looks at my emails, looks at my, you know, Teams messages, Slacks, all of that, and actually gives me an output of what I need to do. So it really helps with my day to day, and that's where, you know, making use of something like Canvas really comes into play. Yeah. And I think just something that that Rizvan kinda called out is I'm using it more often. I I like the CLI a lot before, and so I went from IDE to CLI and now the app. Again, the the nice thing about about the app is the way that work trees work. So, like, breaking apart different things, being able to work simultaneously, having multiple agents run, whether that's in the background or the main one, like, those are really helpful. And then the last question just because I know we're out of time. Rob asked just around, do we have a way to see budgets within the desktop client? Because he's... He he said that... Which we've all kinda ran into this over the last few months. I burned down my tokens thinking I had, you know, a 100 to 200 left, and I actually hit my million mark, and it shut me off. So Yeah. No. That's a great question. And this is where this is gonna sound a little backwards, but you can actually ask Copilot to give you what your budget is. And it'll... It can look at your individual budget, because it's gonna look at your plan and your usage limit. So it's something where you can ask Copilot, and it'll tell you, you know, hey. Like, of your plan, you have this many credits... Or not this many credits, but you have this much budget left, before you kind of run out of your use. Keep in mind that that may be a individual user level budget or that may be a cost center user level budget. You're not gonna have visibility into, let's say, like, you know, overall, if there's, like, a stop on your cost center level user level budget, you'll just kinda see your individual one currently as it stands. But, the TLDR version, yes, you can see it within the desktop app, and you have it on your profile. The big thing that I often go to is if you just go to, like, your, you know, github.com, and then it should be under settings slash copilot. As long. as you're signed in, you can see them there. And sometimes it's easier rather than, like, oh, in the IDE, I look here, or if I'm in the app, I go and look here. Or if you don't wanna ask Copilot, I just go there, and it'll show you how many tokens you've consumed. And then if you have a budget, where you're at running into that. So awesome. Yeah. Great question so far. Like, very, like, they're very they're very, like, specific to what we're kinda seeing day to day with a lot of our customers too. So... And I think, this was already mentioned, but, yeah, is HydraFusion available? If so, how does it affect token uses compared with using a single model? Great question. If somebody's like, what... Like, what's HydraFusion? Is that part of the Avengers? It's not, basically what HydraFusion is. It's a new way of thinking that GitHub is enabling. You can think of it like this. HydraFusion, if you make use of the HydraFusion, quote, unquote, model, what it does is that it looks at your task. So let's say that you give it a specific task that you wanna do. I want you to implement 10 features within my app that I'm currently building, and I want you to do it the best way possible. Typically, what happens is if you use auto mode or if you, if you use a, specific, like... If you use, like, a specific, model, you're just using one model to achieve all those different tasks. Maybe the model, like, you know, splits those tasks out into its own sub agents or into its own sessions to do those tasks, but it's still making use of one model. And you're just kind of at the mercy of that model and how efficient that model can do all 10 of those tasks. Let's say that for two or three of those tasks, maybe a different model makes more sense to, complete the task and actually be more efficient in doing so. That level of thinking, that picking of different models for the different tasks that you give it, that's what HydraFusion is. So what HydraFusion does is that it looks at the task that you give, compare it across different models without you having to, you know, incur, like, incur any sort of token cost, and it picks the best model for the best use case for that task. So let's say that the first part of the task, you know, Opus makes sense for that. It'll make use of Opus. Then for the second part of the task, if, let's say, like, you know, Astra makes sense for that... For for that particular piece of the task, it makes use of that. And then maybe third, it makes us a little different model. But it, picks the different models as needed and pretty much makes it so that, it's very efficient with getting you an output. And what we've seen... And I I don't know, Austin, if this may be the right benchmark, but as far as I can remember, I think it's... It was 60% more efficient and cheaper to use HydraFusion versus if you were to just use any sort of model to delegate your task to. So, really, you can think of this as a way to, optimize your costs and as a new way of, you know, delegating what type of work you're gonna do with what different type of models where it really takes away, like, you having to figure out which models to use. Yeah. And I think, like... So 100%, Rizvan, like, I I I love the way that you explained it. The big thing... And this is where I think we're seeing a lot of the industry shift as a whole, like, not only Github, but Anthropic and OpenAI, is they're starting to shift of, like, how much routing can we do on the back end so that the... There's less context switching or management from a developer side. Right? Like, at the end of the day, when the developers are using the tools, they should just use the tools. And if there's a more efficient way to do it, we should be able to do that programmatically on the back end. And so. there's a lot of talks and kinda shout out Universe because it's gonna be happening later this month, but, there's gonna be a lot of things that are coming out with Universe on how we are rerouting those, how we are optimizing, and then the benefits that we should start seeing. We are also doing things from, like, a Copilot perspective by using things, we're calling memory or Copilot dreaming. There's also been talks of things like software factories. The idea is we want to be able to make these as efficient as we can so that, again, there's less mental load on the developer themselves, and we're doing things to try and optimize. Like, I've even noticed since we've enabled, things inside on, like, our own GitHub because we do a lot of, like, the experimental testing. I've noticed that it just works. Like, it just does things better rather than me having to massage the model or maybe. switch different models or, hey. I'm gonna do a plan with this model, and I'm actually doing implementation with a different model. Auto mode has been better. I've been able to delegate tasks a little bit better and reduce those overall consumption limits too. So the point that I want to allude to is, like, it's important to be cognizant because when you're seven days into the month and you blow... You know, you've blown your entire limit, you're gonna be pretty upset. However, being able to do these things is gonna become more natural and built into the products of the future. So I think there's just a lot that we rely on all of the LLMs to do, where in the future, there's gonna be a lot of orchestration systems around that. So got it. That's a great way to put it, Austin. I think, Rob had a question that he says, can we use balance but also beat up models that are subpar? I'm thinking of you, Google. Okay. Gemini is not too too bad, but I see your point. So short answer, Rob, yes. We can use balanced mode and also weed out models that may be subpar. Where that kinda comes into play, however, is on the enterprise side of things. So within your enterprise, you can actually configure what models you may wanna use within, you know, your Copilot sessions and what models are gonna be available to you. So you can pretty much, like, do it there at a GitHub Enterprise level to say, like, hey. I only wanna use x y z models. These are the models I care about, and I don't wanna use Gemini. Gemini is, like, that's where you can kind of put that into play. I don't believe that you can do that right now as it stands with the auto mode, but that may be something potentially in the future might be possible. This is really kinda where HydraFusion comes into play because HydraFusion will know all of that, and then go from there. Pending, I guess, but better. Yeah. Yeah. Pretty much like, yeah, you have your set of models that you... That you're allowed, and that kinda helps reduce which models you use, or which models you can use. In that way, it's scoped to, you know, like, hey. We wanna use just these models within our, within our environment. Awesome. I know we have two more minutes left. If there's any other questions or anything else, feel free to type in the chat, and we'll try to answer. Forty five minute presentation. Oh, yeah. That's funny. I think we'll do one last one, and then we'll we'll wrap up. So we can do the. question around, does HydraFusion use bring bring your own key models as well? So... And then I can I can answer Lisa's while you answer the other one? Yeah. So HydraFusion, as of right now, does not use bring your own key models. This is something that I think Github as a whole, we're trying to look at, you know, how best to kind of implement this. So as of right now, HydraFusion doesn't do that, but in the future, that probably is something down the down the road map. I just don't know, you know, too much of that details, but that would be something that'd be really useful. And then I think Lisa's questions, and we'll kind of end there. Awesome. Okay. You already did that. Great. Awesome. I know we have a little bit of time left. So, again, thank you everybody for joining. Hopefully, you got some value out of the presentation, and we'll kinda send over the recording and some of the materials. And, yeah, thank you again for joining all your questions. Have a great day.