Name: How to Trust the Code You Ship
Uploaded: 2026-09-29T17:16:25.679Z
Duration: 59 min
Description: How to Trust the Code You Ship

Transcript for "How to Trust the Code You Ship": Hello, everyone. I believe. we're... Good after. hello. Hello? Hey, Eric. How are you? I am great. Sunny that was, Francisco. that was our webinar. Thanks also for joining us. Yeah. We've got nothing to be worried about with application security. It's everything's fine. It's fine. Oh, Alright. Let's go ahead and get started, and, let's, let's introduce ourselves. Eric, I'll let you go first. well, even before we do that, here's a quick overview of of, What we'll be talking about today. We've got, we'll set the stage for why we're all here and why application security is something worth caring about. We've got a couple of slides, but the lion's share of our focus today will be on actual demos. We want to show you how this stuff works, how it works in GitHub and the workflow of your developers teams because you're not here to to listen to the marketing guy. You're here to see a solution that you can actually deploy in your own environment. So my name is Eric Tully. I'm the product marketing lead for security quality here at GitHub. I've been here for about four years coming up to my fourth GitHub universe at the end of October. We, are very excited and have some cool new stuff to show you and a little sneak peek to show you today, but I'm not gonna say more about that until later. Leo, say hello. Hi, everybody. So my name is Leonardo Quimbaia. I am what's known as a field security specialist here at GitHub. I am what's known as a product expert in GitHub Advanced Security as well as GitHub code quality and also all the cool Copilot features that we're using in Advanced Security and Copilot. So we're gonna be talking a lot about that today. Awesome. Okay. We are gathered here today because, this is not news to everyone, but it does kind of put a picture at the the worry and the weight of the responsibility that all of us in this field are feeling right now. That there's more code, more vulnerabilities, and just less time to deal with it than we've had in the past. We've seen for output that the commits on GitHub now top 2,900,000,000. That's probably more than 3,000,000,000 since we built this slide a month. And that's three x the volume that we saw just this time last year. Agents of course are writing a growing share of that code. So the question is can we find the risk in this code? What about the risk in our backlog or in the legacy code that no one really wants to touch but we have to still maintain? The second scary graph is AI expands discovery and helping us find more vulnerabilities than ever. For example, in open source dependent bot alerts now top 400,000,000 a quarter and that's five x what we saw this time last year. Finding is is not necessarily easy or free, but it is not the hard part. The fixing is because the backlog is growing faster than any of us can work it down. But this is the scary bit, exploitation. Just three years ago, it was typical for, a disclosed vulnerability to take about six months before it became a working exploit. But today, it it's a day. It's fallen to a day. And that means that it doesn't make sense to have a plan to catch up on these things. We have to stay ahead of it. And the way that a lot of us have structured our development and security organizations and workflows has been handoffs. And we don't have time for handoffs like we used to. We have to fix we have to fix code and fix vulnerabilities as we find it. And that means more responsibility being delegated to the developers. That's the whole shift left, but more important now than ever, so that we're not creating a new backlog. So to to give you a quick preview of what we're gonna show you today. First, the finding part. We'll show you how to catch vulnerabilities in any code, whoever or whatever wrote it. And we will give you a sneak peek at how we can use AI to scan full repositories, including legacy languages. And second, fixing. You might be familiar with the original Copilot Autofix. We launched that, a little over two years ago and it was very much in the products of 2024 when we were all thinking about prompt engineering and one shot prompts and how good they could be. Copilot Autofix would describe a vulnerability give you a fixed suggestion it was a good place to start but we have a massive upgrade in capability that we're going to show you today called agentic autofix And we'll show you how you can take a vulnerability and turn it into a, merge ready pull request. And then finally, govern. These rules that we put into place determine how or whether code merges and the best practice of course is to build that into the system rather than hope people will opt into doing it because without governance security is is a hope and not a plan. And actually I think this is a good place to start our demos today. But before we do, I'm gonna stop sharing, and let's let's kick this off with a quick poll because we're curious as to what's top of mind for everybody today. Alright which of these costs you the most right now? I'm going to read through the the questions here we'll share the results just taking ten seconds or so. Review capacity that hasn't grown with code and volume, a security backlog nobody has time to work down, or exposure in code that no scanner covers. It's a horse race. We see a trend. The backlog is an issue. Alright. Let's close the poll, and, Leo, show us show us some solutions to what we're talking about here. Okay. So what we're gonna do first... So before I start sharing my screen, what we're gonna do first is I'm gonna first speak about things from a governance and security perspective. So the big question is, how do we turn on these different features in order to make sure that we can trust the software that we're shipping. So let me go ahead and begin the share, and I believe it is this guy right here. Okay. Excellent. This is exact same... This is this is exactly what I wanna share. Awesome. Things are working as they should. Don't you just love that? Alright. So I'm gonna be working in two different places within GitHub, to kinda showcase these particular features. And so the first place that I wanna start is gonna be where we want to be able to turn on the advanced security features at scale. And so I do this by either going into my enterprise settings, or I can go into my organization settings. And for those of you who are not very familiar with how GitHub is structured, or at least GitHub Enterprise Cloud or Enterprise Server, at the top, you have your Enterprise, and your Enterprise is gonna be made up of everything underneath it. And then, a level below is gonna be your organization. So, there are many organizations per enterprise. And then, within an organization, there's many repositories. So, many repositories to an organization, many organizations to an enterprise. And so, I can go ahead and create something that's called a security configuration. I can do that at the enterprise level, and I can do that at the organization level. I'm gonna go ahead and show you at the organization level because it's pretty much the same between, but there are some nuances with the organization level that allows me to specify what repositories I want to turn things on for. And so, what you'll see here on this page is I have a bunch of different configurations. Either they're, formed at the enterprise, and I can just apply them at my org level, or I have some organization, configurations that are local to this organization and not available to other organizations. So, the one that I like to talk about the most is one that's all gas and no breaks. And so, this is essentially an organ, a configuration where I'm turning on all of the advanced security features for my organization and then just turning and then having these apply to all my repositories. And you can see here, like, I have all of my secret scanning, things turned on. So a part of, you know, trusting the software that you ship is also making sure that you don't have secrets all over the place in plain text that can be easily extracted by anybody who just so happens to get access to your repositories. So, anybody who has write access, read access to any sort of repository would be able to see a secret in plain text. It doesn't even need to be exfiltrated from your environment. It can just be sitting there, and somebody can use a secret that's available to them in a way that wasn't intended. So, that's something that we want to limit as much as possible, and that's what GitHub secret scanning is for. And there's really cool features like, push protection that would prevent developers from adding in new secrets to the repositories. So, these are things that I wanna make sure are turned on. Then, if I come down here, I have my code scanning, which is going to turn on CodeQL, for example. And CodeQL is the static analysis tool that GitHub uses, to perform, well, static analysis, to understand what sort of vulnerabilities exist in the codebase in your first party code, and then identify them, let you know what to do with them. And then we use Copilot Autofix, as well as Agentic Autofix, to, create fixes and to address those fixes and remediate those vulnerabilities as soon as possible. Then, of course, we have dependency scanning, using tools like Dependabot, using, our dependency submissions tools, as well as dependency graph, everything that you would need in order to create an SBOM, for example, or to find, identify malware alerts with your dependencies, things of that nature. That's where we would turn on these particular features here. And then, we can also set a specific policy, such as: Do I want this to be the default for all of my repositories? Do I want this particular, configuration to be where everything, do I want to be, the default for all my repositories, just my private and internal repositories, or all public repositories? I can have different, defaults for these different classifications of my repositories or different visitabilities of my repositories. And so, from here, I have my configuration. I haven't applied it anywhere yet, but let's go ahead and go back into the organization view, And I have this little section here called repositories. And this is where I can go ahead and say, you know what? I wanna select all my repositories, select all of them, so I have 40 out of 40, and I'm going to apply all gas no breaks to all of my repositories. And once I do that, you'll see here that we have enforced all gas no breaks. So enforced means that, this is something that repository owners cannot change. The only people who can change this configuration are gonna be security managers as well as organization admins and owners. They can make those changes, but repository owners cannot make changes, Whereas this, other security configuration that I've applied, you can see here it's the secret scanning default, which just means this is a security configuration that I created that only turns on secret scanning. Well, repository owners and admins can make changes because it's not enforced. So depending on the policy that we want to set as an organization within our enterprise, we can enforce them or not enforce them to prevent people from making changes as necessary. So, this is how we're able to do things from a security perspective. But if I want to apply additional policies, I will need to go ahead and do thing... Something called a repository rule set. So I'm gonna go ahead and jump into that. So I'll... I'm coming over here to... I'm still in my organization view, and I'm still in my organization settings, and I wanna go under Repository to Rule Sets. And here is where I can create my new rule sets. I can create branch rule sets, tag rule sets, push rule sets. Like, a push rule set would be something along the lines of, you know, I don't want people putting, Excel spreadsheets in my repository, for example. A tag rule set would be, I wanna set certain policy around specific Git tags. And then branches is I want to create a particular rule set that is going to protect, specific branches in my repository, like my default branch or a staging branch or a development branch or a feature branch. I want specific policies for different types of branches in my environment. This one, I'm going to call protectmain, and I'm going to go ahead and set this one as active. And then I can set up a bypass list saying, like, you know, I want certain people or agents or teams to be able to bypass the rules that I have in here. For now, I'm not gonna add anything. But you'll see here that I'm gonna be targeting, I can say, target repositories. I want this to apply to all repositories, but if I want some more nuance, I can create that nuance. And then here, I can say to target specific branches. And in this case, since I'm protecting my main branch, I'm just going to I'm just going to include the default branch. Now, you can change the name of your default branch however you want to. The default naming scheme for GitHub is main. It used to be master. We changed it to main. And if you have other default, naming schemes, you can go ahead and just use this default branch because whatever is a default branch, whatever it's named doesn't matter. This will be protected with this rule set, for example. Or if I have a specific pattern that I'm using to name my branches, then I can go ahead and use IncludeByPattern. Now, what you'll see here are a couple of rules. The rules I want to focus on for our presentation here today are gonna be a few things. So, first and foremost, I want to make sure that we require code scanning results with every pull request that is coming into my default branch, for example. And here, we're gonna have CodeQL be that tool. And so I wanna say, anytime that there are high or critical vulnerabilities that are being introduced with a pull request and CodeQL identifies them, I want this, that particular pull request not to be auto merged and not to be merged. I want these things to be taken care of first before they're merged into my main branch. In this example, I'm using my main branch, my default branch, as my production branch. Now, I can do the same thing with code quality. So code quality is a new tool that GitHub came out with around the July time frame of this year, and it is a tool that's built inside of GitHub that allows you to identify any sort of code quality issues as well as identify, test coverage. And there's also a, restrict... I'm sorry, not test coverage, code coverage, I should say, that in a PR, we can see, of the new code that's being introduced, how much of that new code is being covered with tests. And so we can also restrict code coverage from that example too. Next, I can say, Hey, I want to make sure that, secret scanning alerts are resolved in the PR. This is something that's currently in preview, and I can go ahead and click that and say, Hey, I want to include just the provider patterns, custom patterns, or even, like, other generic patterns. And if you've seen our previous Secret Scanning webinar that we did, you know, I would suggest, you know, checking that one out because we have more details about what all these things, mean. And, of course, I can automatically request Copilot code review on any pull request that goes into my main branch, and then I can also require license compliance, results before merging, meaning that if I'm introducing new dependencies that have licenses that are not permissible within my company, organization, what have you, I can set that policy at the, organization or at the enterprise level, and then I can create this particular rule set to say I wanna make sure that we are abiding by that license policy. So, with all this in place, I can go ahead and hit create, and I'm going to have an interaction with 1Password. And then after that interaction with 1Password, I will have created a, repository rule set that will protect my main branch. And you'll see here everything that we've set: restrict deletions, block force pushes, require code scanning results, code quality results, making sure secret scanning alerts are resolved, automatically request Copilot code review, and require license compliance results before merging. So, we created a security configuration to turn on our security tools. We've created a repository rule set to make sure that when these tools are running, that they meet, that certain things are met or certain requirements are met in order for code to be promoted into that new branch. Now, one last thing before we transition to the next part is where do you even turn on code quality? Because I talked about code security, how to turn that on from configurations, repository rule sets. Well, code quality is also in the setting page under Security and Quality. If I just come over here to code quality, I can click on this button that says Enable code quality, and I've enabled code quality. That's really all that it takes. So, let's go ahead and transition to the next part, but I wanna make sure that we queue up our next poll. So, Eric, can we get that poll up and running, please? Yeah. Let's do it. You might need to stop sharing for a moment. Sure. I have ceased my sharing. Okay, well, thanks for that overview of of, how all this stuff gets turned on and enforced, but what's the most important thing to enforce? Where would broader coverage when it comes to finding these vulnerabilities matter more? Is it AI generated code? Is it, legacy systems that people are reluctant to have to hassle with? Or are there just the overall backlog of quality maintainable defects that that contribute to problems or or or reductions in uptime for your systems. And there's a clear trend here. Yeah. I'm seeing AI generated code shipping faster than review is pretty dominant here. Yeah. You know, the backlog just it just moves. You make something faster. It just creates a new backlog somewhere else. Well, fortunately, we've got some solutions that help out with this. So let's close the poll and get into finding. Alright. So So let me go ahead and begin to share the screen once again. Alright. So now I'm in my repository view, and I've created this repository. This is my benefits administration demonstration, and you'll see this is for a fictional city called New Panda City. Fun fact, I had Copilot create a logo for New Panda City, and I said New Panda City is very famous for its silver mines. And so it created a little panda miner in front of a cave or a mine. So I think that was... And it did all with SVG. Like, it was just all straight code, all straight scalable vector graphics. Just really... To me, like, that was kind of a a funny little thing. Like, it didn't just draw it. It drew it with code, which I thought was kinda neat. Now, let's go ahead and we've applied a security configuration, so we've turned on our features. We have our repository rule sets that we've enabled here. There's a few other things I wanna show you that's gonna be unique here, and these are things that are in public preview as well as in private preview. And these are things we're gonna be talking a lot more about at Universe come the end of next month. So if I go back into my settings and I go into Advanced Security, there's two little features I want to showcase. And that's gonna be here. So, we have a couple of options here. So, we have something called AI Scan for repositories, AI Scan for pull requests. So, what do these things mean? So, AI Scan for repositories is going to be a feature where we are scanning languages that are not, traditionally covered by CodeQL. So CodeQL has coverage of, like, C and C plus plus Rust, Ruby, Java, JavaScript, TypeScript, Python. I'm probably missing a few languages here and there. But for languages like COBOL, languages like Bash, Bash, languages like Elixir or, SAP... Not SAP, SAP or something like that. Anyway, and there's a whole bunch of languages that are out there, and CodeQL doesn't cover them all. So what we do is we're using AI to help scan code that's being introduced via the pull request to get a better sense of how much additional risk are we getting outside of languages that are being... Or that are covered with CodeQL. So this repository is written in COBOL. It's written with Java. It's written with Java, JavaScript, TypeScript, and, well, specifically more, Vue. Js, just a framework for JavaScript and TypeScript, and as well as shell scripts. And the whole premise behind it is it's a repository that has that Cobalt back end that we're sort of modernizing with Java, and so it's kind of in a transitionary state where we have, like, a Java middleware sort of business logic with a Cobalt back end and, of course, a Vue. Js front end. All that just kind of details, but... And sort of like the spirit of what we're trying to say here. And also, raise your hand if you are a COBOL developer. Let me just look at the audience here and... Oh, it's a lot more than I expect. I expected saw four hands up in the air. So, you know, to you COBOL developers out there, thanks for raising your hand and being brave. The next piece is gonna be this, AI scan for repository. So, I talked about pull requests. I did it a little bit backwards. But AI scans for repositories, this particular feature is currently in private preview. It should be coming in public preview in the in the near future. And the idea behind this is, well, if we can use AI to scan for languages that are being introduced with the pull request, well, let's do backlog scans too on, in this case, on our default branch to get an idea of, like, what sort of, security backlog or what sort of issues we've had previously. And we can identify them with AI. So we can identify, things with CodeQL in the pull request and in our repositories. And we're also using AI to help extend that static analysis scan for other languages outside of CodeQL. So, we're able to get both sides of that particular story. And then, of course, Agentic Autofix. So, we're gonna kind of play with these things and kinda show you what these all look like. So I already have a pull request in mind here, and it's gonna be this one, dependent verification against legacy civil registry. How nice is that? And, of course, I have some code that I've introduced. I've changed about 10 files. And if you're not familiar with our new sort of pull request, view, you know, hey. Try it out if you haven't already. It's currently in preview. Give us feedback if if you have some thoughts on it. But I have a bunch of different files here, and you can see what those files are: my depverify. Cobol, dependence. Js, my dependence view dot vue, etcetera, etcetera. So I have some Java files. I have some JavaScript files. I have some COBOL files. Cool. I'm gonna go ahead and come back to my conversation view, And what I'll see are a couple of things. So first and foremost, remember, we are running CodeQL in this pull request, we are running code quality in this pull request, and we are running an AI scan in the pull request as well. So three different scans that we're running here. So all of the responses that we're seeing in here are all gonna be coming are gonna be coming from two sources, the Advanced Security bot and the code quality bot. So, just to make sure we're all clear on that. And what we're gonna see here is, hey, Advanced Security found a vulnerability, a cross site, scripting vulnerability. I can go ahead and show paths. This is something that Code, CodeQL identified because view is JavaScript in TypeScript, and CodeQL supports that. So, we're able to see the results here. And so, we can see the source to sync, you know, somebody using window. Location dot search, and then once we go all the way down to sync, we can see, hey, it's returning back that information to the user. So, a potential client side cross site scripting vulnerability. And then in here, we have something, that Copilot provided us, which is a fix to this vulnerability, to say, like, you know, hey. If we change the code in this manner, we can help remove that vulnerability. And so what I can do is I can say, well, I wanna ask Copilot about this. I can commit the suggestion and then rerun my checks, or I can add a bunch of my suggestions to a batch pull request and then have all of that go as in, like, one big commit, for example. So, I have some choices. I can also dismiss this and be like, You know what? I like cross site scripting. I want to keep it here. We gotta keep things at least a little bit interesting for, you know, all of the hackers out there. We wanna give them some some ways in. No, we don't. We don't want that at all. So I can see there's other types of vulnerabilities here. And then we have a code quality bot problem. So, again, we had code security, provided some information. Now we have code quality coming in and saying like, hey. You can improve the the the quality of the code by doing this type of thing. So in this case, it's saying, hey, this method overrides this particular method here. It is advisable to add an override annotation. Hey, that's gonna be helpful for future me or any other developers who are gonna be working on this code because, later on, if I forget what I was doing here or somebody else needs to take on my backlog, hey, at least they're able to know I did an override or this is what I was trying to do. And so you can see here, boom, we are overriding this particular function, this particular method, and I can go ahead and commit this to, you know, help improve the quality of my code. Further down, I see there's another GitHub Advanced Security Bot. So these are gonna be the ones that AI helped identify. So the the first two with the code Advanced Security Bot and the code quality bot, both of those found things in Java and the other one in JavaScript. Those were found with CodeQL because Code Quality uses CodeQL in the background as well. Now we're looking at the AI findings with the Advanced Security Bot. And as you can see here, these are findings from, finding from our shell scripts here. And I can go ahead and scroll down, and I can see, like, oh, okay. So this is the issue that AI identified. It gave me a long, explanation on it. And then it went ahead and gave me an autofix. So I can go ahead and commit the suggestion and move forward. And it did the same exact thing for COBOL as well. Can you believe it? We are doing static analysis on COBOL in GitHub. I've been working here for almost four years, and I never thought I'd live to see the day. But now I'm alive, and this is happening, and you're seeing it live here on this webinar. Imagine that. Things are happening, ladies and gentlemen. They're happening a lot faster than we think they are. So we are looking at a co... COBOL, whoo, a COBOL vulnerability that is being fixed by AI, and we can go ahead and look at how AI wants to fix this. I don't know COBOL. I don't mess with COBOL. COBOL and I, you know, we're not good friends or anything like that. But, hey, this looks good to me. I'm gonna go ahead and commit the suggestion, and we can go ahead and move forward. So, this is sort of the experience that you can expect moving forward, is you're gonna see that we can do static analysis with CodeQL in our PRs for CodeQL, languages, but we can also use AI to help extend that to other languages and to help extend that sort of, detection. So, that way, we're able to find all sorts of areas that can introduce risk to your applications. And once we commit these suggestions and then we run our tests again because every time you make a new commit on a PR, we rerun all the tests: the CodeQL tests, the CodeQuality tests, as well as some of the AI tests. And that way, we're able to ensure that you can trust the code that you're shipping by making sure that we're taking care of these things as they are relevant inside that pull request. So, I feel pretty good about this. Eric, do you feel pretty good about this? Because I think right now, I'm feeling so good. Let's kinda get another pull up here, and we can go ahead and move on to the last section of my presentation here. So let me go ahead and stop my sharing, and we can begin polling. Yeah. So you've shown us how we're gonna find more vulnerabilities than ever, which sounds like a great way to increase our backlog, which everyone, really cares a lot about. So here's the question. Are we ready to trust are agents up to the task of being able to take a vulnerability and running with it? And how do they do it? So are you already handing your security backlog to an agent? Are you ready but haven't started yet? Or are you still skeptical and wanna see more? This like that 15%. the most even. Yeah. It's more even than the than the rest. And, yeah, I I kinda agree because I think the answer for most of us, well, it kinda depends. Show me how it works and give me some confidence as to, you know, whether or not this is, worth trusting. Is it better than nothing? Does it introduce new problems as it's resolving old ones? So, that's that's a great segue back to the next section of the story. Absolutely. Okay. So let's ignore the AI scan if reporting errors because, you know what, sometimes demos just don't work the way you want them to. But right now, I am looking at my security backlog here. So these are all the code scanning alerts that I have from that repository we were just looking at. And so if I come over here to tool, I can see that there's four CodeQL identification... Or four CodeQL identified vulnerabilities and 12 of them that were found with our AI scan for repositories. So let me just look at the CodeQL ones first to kind of reintroduce people to CodeQL and how that works. And while we're looking at this, I'm gonna go ahead and run another AI scan. So, this is something that's in private preview currently and will go into public preview in the near future. So, and it's also something we're gonna talk a lot about at Universe. But the key thing here is to understand that this is gonna be something that we are going to be, the way this will work is when I hit that Run button, it's gonna go ahead and, the AI... Actually, I think that we can see what that agent is doing. So I'm just kinda veer off into uncharted territory here. Maybe I can't. You know what? That's fine. I deserved that. I shouldn't have veered off into unseen territory. But what you'll notice here is that we are running this currently. Oh, hey. There... I just had to click the run-in progress button. Okay. Hey. It's live demos. You know what? You you you roll with the punches. So right now, we're gonna be running this particular demo. So we're gonna give it some time, and I'll click on that button again a little bit later. And I'll give you some details about what's sort of happening in the background there. But while that's working, let's look at the CodeQL identifications here, the CodeQL identified vulnerabilities. So, for here, we have some deserialization of user controlled data. And so what we can see here, again, very similar to what we saw in the, pull request is, like, hey. If I click on show pass, I can see the data flow from source to sync. When tainted data comes in, what we do with that tainted data, do we validate it, do we sanitize it, are there any sort of barrier between us taking in that data and then moving that data somewhere outside the code base, whether it comes back to the user, whether it goes into a database, whether it goes to some other source or some other place. Where does that, data leave the codebase? That will be our sync here. So, we have that data flow. We have our explanation, of course. And then I can say, you know what? I want Agentic Autofix to fix this for me. So I'm gonna go ahead and click on assign to Copilot. And what that's gonna do, it's gonna go ahead and get Copilot to look at the alert, to look at the the serif output from CodeQL. It's going to look at, you know, how this has been addressed in the past. It's also gonna look at some of the, Copilot instructions. So if you tell it to talk like a pirate, it will go ahead and do that as well. And then it'll go ahead and suggest a fix. But what we'll also do is whatever code that Copilot Autofix is going to suggest, it's also going to check that against CodeQL, and it's also gonna do its own sort of AI scan on that code. So whatever code that Copilot Autofix wants to suggest to us, it's also gonna check itself using some deterministic tools as well as nondeterministic tools in order to to help us trust the code that we ship better. You know, it's almost like we named this webinar after that. Like, roll credits. Like, we did it, ladies and gentlemen. Mission accomplished. But that's what's happening in the background. That's what we're doing with, Agentic Autofix because we wanna make sure that, hey, one, we are fixing the vulnerability. Two, we're not introducing more vulnerable code to the code base. And then, three, we're just making sure that, hey, this... Does this actually work? In that case, we can go ahead and create that pull request, run those tests again, and then you can go ahead and merge it and be more secure in the code that you're writing. So that's what's happening right now. So just take a moment and kind of think about this for a second. How long does it usually take somebody to go from we've identified a vulnerability to we have completely fixed that vulnerability? And I believe, the the meantime to remediation was, I think, about, like, six months for some people out there in the world where we have identified this vulnerability that we want to be able to address, but it takes us a long time to get to it. Now in the world of AI and agents, we've seen that six months get reduced down to about 42. And I don't mean, like, forty two minutes as in this is the agent running for forty two minutes. I mean, we've performed the scan. We've identified the vulnerability. We have an agent fix the vulnerability. We have the the agent validate that it's not introducing any new vulnerabilities, and then we merge the code. We recheck the code to make sure that there aren't any vulnerabilities there again to verify that's fixed the vulnerability, and then we've merged it. So from six months, that whole process, down to forty two minutes. That is an enormous savings of time. Like, imagine you wanted to get into woodworking, and instead of practicing woodworking and cutting off all your fingers over the course of six months, you could be a master woodworker in forty two minutes. Now, this ain't the Matrix, but sometimes I kinda wish I could learn kung fu right now. But you gotta admit, like, that's a huge time savings in terms of just, like, mean time to remediation. So I just really wanted to push that back there. And that whole rant did last me three minutes, so I feel good about that. That was really good timing. Pat on the back, Leo. You did great. So let's look at this pull request real quick. So, what you'll notice here is that, hey, we're running all of our tests again, which is good. We want to be able to do that. The agent itself did some test in the midst of its changing. So, if it does, like, introduce a vulnerability, it'll redo the code to make sure that we're not introducing a new vulnerability with that fix. And so, we should see that, Hey, this is going to go ahead and address this particular issue, that we shouldn't see that vulnerability anymore. And we also see, like, what sort of plans that it took here. So, inspect the import flow, replace the unsafe deserialization, add focused regression, and then review the diff, scan changed files for secrets, and run parallel validation. These were all the steps that the agent took in order to fix this particular vulnerability. I can go ahead and approve the workflows to run. It'll go ahead and do that. I'm gonna say, hey. We're ready for review. It's gonna go ahead and start that. I can go ahead and, Hey, I also want to assign Copilot to this to review this particular PR to fix this. Hey, let's do that too. Hey, let's do that a little bit later. Cool. Let's just scroll up so we don't see that big red banner there. Anyway, you kinda get the idea here, is that we've done due diligence here. We've fixed things here. We are making sure the agents are able to fix things, check things, and move forward. Cool. Amazing. That's with an original, if you will, CodeQL alert. But what does it look like when we're talking about an AI scan for the, the pull request... Or, no, for the the repository? So I'm gonna go ahead and jump into AI scan here. And you can see that, hey. Last week, we identified all these different issues in my... Let's look at one of these, COBOL ones, actually, because it did also find something in Java too, which is great because that means that... It doesn't necessarily mean that, like, hey. You know, co... CodeQL missed something. But it just might mean, like, you know, hey. We don't have that particular query because that might be a noisy query for CodeQL, and so we don't use that one. But our AI scan was able to increase that breadth of detection and give us more detection from the repository level. So, it was able to find something else here in our Java. Then, if we go ahead and take a look at our COBOL one here, I just wanna show you, like, what that experience looks like from an AI scan, and it's gonna look very similar to what the the CodeQL alert looked like. And I can do the same exact thing, assign this to Copilot, and it'll go ahead and fix it. So, in the same way that I can use Copilot to fix a CodeQL issue, I can use Copilot to fit... To fix an AI scan issue. But it goes even further than that. I can come over to Dependabot, and I can look at any of my Dependabot vulnerabilities here. And I can go ahead and say, hey. I want to, do something with this. And in the... I think in the near future, I don't think we've released this just yet, but I need to verify this because I think all these have pull requests associated. So we already present a fix here. But for some of them, we can say, like, assign this dependable alert to Copilot and have Copilot fix this. Assign my my Trivy or my Gripe vulnerability to Copilot and have Copilot fix it. We can use agentic autofix not just for, GitHub tools inside of GitHub, but we can use it with other tools that are running and reporting data inside of GitHub as well. So, agentic autofix gives us a lot more breadth, a lot more depth in terms of tools, in terms of, things that we can do with it. So, I know we are about, we have about thirteen and a half minutes left. And so, I wanna go ahead and stop sharing, and we can go ahead and take any sort of questions that we might be getting. We have a bunch of great subject matter experts in the chat already answering a bunch of, questions. Eric, do you know if there's any sort of questions that people might have had that might have been interesting that popped up that we might be able to respond to? Well, it's very typical when you see something cool like this and you know that AI is working it to wonder, well, we saw how fast it was, but what's what's it gonna cost me? Because, you know, there was three minutes of clock time when we you assigned that vulnerability to Copilot. And so we know it's pretty efficient. And we're in preview, and so this is based on somewhere just under 10,000 different runs of of assigning to Copilot for a a fix like this. But the the p 50 typical cost is about 50¢ to generate that that fix. Now your mileage will vary depending on the size of the repo and the complexity and and other variables that, we would love to be able to predict it perfectly. But just to set an expectation that this is something that is, it's it's not nothing, but it's it's it's an affordable way to deal with this and certainly cheaper than than throwing, brute force and hours of developer time. That's a problem. But, from our moderators, do we have some I know that they've been busy answering questions in the the q and a. Is there anything that you want to uplevel that we can address live? I'm looking I'm looking through. Oh, Sean Mosher. Will this be made available on GitHub Enterprise Server? When you have, do you go wanna take, that one, Leo. Eric, or you want me to Well, it's it's it's a yes and no answer. So you start and and I'll fill in if? I need to. So for a lot of the AI features that we are currently talking about, the majority of them are not going to be available in enterprise server. Now that doesn't mean that it won't ever be. But in the short term, it doesn't seem like we're gonna be doing a lot of these AI features in enterprise server. But, Eric, if you wanna add more nuance to that, please feel free. Yeah. But, it's a it's a great question, and, we're clearly been working on solutions to this. If you haven't yet registered for Universe, you might be interested in registering for Universe. You can attend it. Physically, you can attend it virtually because, we do we have made some progress, and we'll have something to to show you a universe. Is Dependabot going away? No. Dependable is just gonna get better and stronger. Dependabot is getting better and stronger. Again, we can't we can't spill the beans, but, there's cool stuff that you'll learn, October 28 at Universe about dependibot specifically. Oh, Joe Artola asks, are these tools available with an enterprise license or that you require a different type of subscription? Now what we've seen today, the code scanning and and, capabilities are part of our GitHub code scanning SKU. And you might if you've been around the block with GitHub for a little while, you will remember that we've had this package called GitHub Advanced Security that includes code scanning and secret scanning. We've split that package up so you can just get code scanning or just get secret scanning, or you can still get both of them together. And so what we've looked at today, these have been code scanning capabilities. Now the assigned to copilot, that's something that you'll be able to do without having a separate copilot plan. It's just something that, will be included and then you'll you'll pay for the usage just like you would any type of of AI consumption. Here's an interesting question. Why is he wearing blue wristbands? That's a great question. I was wondering the same thing. Yeah. So the blue wristbands are essentially wrist rests, and I can put my wrist on my keyboard. I can put it next to my mouse, and I don't get, like, that carpal tunnel thing that everyone's afraid of in the tech industry. Highly recommend it. Okay. Mike, I'm not gonna read my Crocker. I'm not gonna read your whole question, but it's a great question. The, the contrast between an AI scan and a deterministic scan, these are very, very typical concerns because those of us in security, we're very accustomed to having, like, repeatable results. You run it twice, you get the very same results. And that's a plus of deterministic scanning, but what it hides are the blind spots because you can keep running and get the same results, but you'll still miss things. And and, Leo, you actually showed us an example of that where where CodeQL, which works very hard at not being noisy, but, the the AI found a vulnerability. You know what? Maybe you do need to pay attention to this after all. And I think that this is a great example of perfect being the enemy of better, where when you have AI vulnerability scanning, which we just call AI scanning at GitHub, that there there are some variations, and a lot of times those variations are improvements. You know, we are constantly experimenting with more effective and more efficient models to find and fix vulnerabilities with AI. It's not it is always a work in progress. We move about as fast under the covers as you see some of the, frontier model makers move in public. But we know that we have enterprise customers who are relying on the safety of this stuff, and so that is a a critical concern for GitHub and Microsoft. But it is always top of mind. How can we make this faster? How can we make it cheaper? How can we make it more reliable? And so every time that you're using AI scan, in GitHub, it's literally the best that we can deliver that day, and it's probably gonna be better, a week or a month later. Here's another question. Is this available? Can we enable it with Azure DevOps repositories? This is a completely different product. This is GitHub, GitHub enterprise cloud. I can't really speak to Azure DevOps, but I would definitely recommend that you go talk to our friends on the Microsoft side who can probably answer, more road map related questions on Copilot and how that works with, Azure DevOps. Here's a great question from Ben Sarris. Could we ask Copilot what security tasks or recommendations are left to run-in our setup and repositories? This is something that you can do in the Copilot CLI today. It's a little bit hard to discover because it's a command you have to type. But if you type slash security dash review, it will give you an overview of the the status of the security issues and what you're working on and the tasks that you have. And this is a capability that we know is so important to development teams and individual developers. And we have, new expansions of what it can do and where to do it, that we'll be showing you at Universe, four weeks from today. Yeah. Let's take just, one more question. Oh, go, ahead. there's actually, an example I want to show in the demo. I just didn't have time for it, which was identifying an issue in the security review using the GitHub... The Copilot app and then having an agent identify that fix and then suggest that fix and push a pull request with that fix, all done in the Copilot app. And, also, something that I do with the Copilot app is I have, automation that gives me a list of things I need to do for, you know, in this case, a lot of the open source repositories that, my team are responsible for. It lets me know, like, hey, here's all the dependable alerts for public repositories in this particular organization that are really easy to fix, might need some review, and are are blocked. And so, I know which ones I can go ahead and tackle first, and then which ones I need to have a little bit more time and and... To to work on. Another thing is, like, you know, there's a lot of GitHub notifications that end up coming through. One of my team members uses the Copilot app to pull all those different notifications and then give him a digest of, like, here's some things that are assigned to you that are high priority, based on your criteria, and that gives it to him on a daily or weekly basis. So there's a lot of things that you can do with the Copilot app in, conjunction with using GitHub and using advanced security to get all that information and to get an idea of, like, well, how can I start triaging some of these alerts? How can I start fixing these things? Can I get AI to fix some things and tell me which things that need my attention? You can do that as well. Thanks, Liam. Chacon, I'm gonna use your question as the last question that we answer in the live Q and A. Of course, you can always talk to your GitHub rep and forward your questions to them and ask for for a demo. You can click the request a demo button there at the top of your screen, little blue button at the top. And we, of course, would be delighted to to share more, show more, and answer the questions that you have. But, Jay, you asked a question that we're we all ask. When will this private preview feature be available? Is there a schedule? And, the honest answer is that when a feature achieves the quality and reliability that deserves to be made public preview, that's when we move its its status and availability. So what we showed you today, none of this is long term road map. This is all either currently available in GA in public preview or that AI scan for full repositories isn't currently in private preview. All of these are intended to be publicly available. We're not gonna we're not gonna prioritize a date over, safety and effectiveness, but they're all expected, in the GitHub universe time frame. And with that, I will, first, a huge thank you to our moderators in the back end who have been busy ask answering questions to Leo for putting together this demo. It was really cool to see, Cobalt in the mix for the first time. Most of us don't have to deal with it, but when you do, it's nice to know that GetUp's got your back. And, of course, thanks to our whole events team and the Goldcast team for for running a great session. Yes. There will be a recording available. We will send you follow ups that will give you links to that. We encourage you to share this with your colleagues because, I know you know people who would love to learn more about this, and and we'd love to share it with them too. Cool. And with that, thanks, everybody, and we'll see you next time. Thank you, everybody.