How to Automate Security Testing for Your GraphQL Service

Welcome, everybody. Thank you for your time. We really appreciate it.

In this workshop, we're going to be going through automating security testing for a GraphQL service. So what we're going to do is take a vulnerable example GraphQL service, and we're going to put some security testing in the CICD pipeline for that repo. And that really gives us some nice coverage around the code and the application which helps us find bugs earlier, because really, that's the point. You want to find those bugs as soon as possible.

So we're going to be, we'll go through the agenda, but just at a high level, we're going to be forking an example repo, modifying that repo, adding in some GitHub actions to do some testing. We're going to be hooking up CodeQL from GitHub and Stackhawk scanning. So we'll be able to really get some good testing into, around this application and surface some of those vulnerabilities.

A little bit about me before we get into things. My name is Topher. I'm a Lead Software Engineer here at Stackhawk. I've been here since, almost since the very beginning of the company, I was the second engineer hired after a principal architect. When I started, there was no source code at all. So I've been involved kind of building everything for Stackhawk from the ground up, process, code, architecture, all that kind of stuff. I'm in Colorado. I've been in the startup world for quite a long time, since the mid 90's, was my first one. I started out in the Bay Area. Probably, the one that you might have heard is I worked, I was at Netscape early on, way back in the day. I was there when they were inventing JavaScript and SSL, or TLS as it's called now, and all sorts of stuff. That was super fun. I've got some kids, love to ride my bike. And my main hobby is play music. So I try and get out and play as much as possible. And Nicole is my co-host, by the way, she's the one who's going to be helping out with the Discord and everything.

So before we get into the workflow steps, I wanted to kind of give a little bit of overview of the security stuff that we were talking about and why it's important. So in this workshop, we're going to be doing, implementing three different types of automated security testing for this GraphQL service.

And so the three major types, the way we think about it, the way I think about it is we have this first one here called software composition analysis. If you use GitHub, you've probably seen Dependabot give you some notices, or at least they've asked you to enable Dependabot. And kind of what this does is it just looks at your dependencies and says, hey, are there vulnerabilities in these dependencies and then tells you about it. It's useful in that, you know, if you're using something, the big recent one was the log for shell one. If you're using that library or if it's a downstream dependency or something like that, this will let you know. The other, kind of the downsides are it's not looking at things in your code, in your specific logic for your code or in your application. It's really just looking at known vulnerabilities and dependencies. But it is very fast and it's really cheap and easy to implement. So it's a good one, software composition analysis. It's good to know. It's good to have. We're gonna be enabling that as part of the first step that we're gonna go through.

And then the second one is static application security testing. I also call it SAST. So if you hear me say SAST, that's what I'm talking about. So some big vendors here are Snyk, CodeQL, that's what that top icon is. And what this does is it looks at your code. So it analyzes your code and it's not quite a compiler, but it looks for things in your code. It's got a parser and it tries to identify known patterns in your source code and alerts it to them. It's really nice because it has a specific line of code that it will point out and it's really helpful. The downside is that it's not always right. Like it's just a guess, it's a well-educated guess, but it's still at the end of the day. It's just a guess. But we're gonna be implementing that as well. So we're gonna turn on CodeQL as part of this demo and then maybe if we have time, sneak as well, because we just launched that.

So we've got the third one then is the Dynamic Application Security Testing or DAST. So what this does, this is different in that it actually stands up an instance of your application and then runs a scanner against the live running application. And so it's very useful because when it finds a problem, it's actually a problem in the running application. One of the things that's difficult, though, is that it doesn't point out the line of code like the SAS does. But you know that this vulnerability is here and you can go then go trace it down and try and figure out what's going on. So DAST tools, we've got StackHawks, ZAP and BurpSuite. Today we're gonna be going to be implementing StackHawk, which I work for and it's a great product. And yeah, so those are the three different types of security products that we're gonna be implementing on this sample application today. We're gonna be putting them in the build pipeline. So on every commit that goes against any branch, just the way we're gonna set it up, it will run these tests and then we will know if that commit caused a regression. And so as kind of conventional wisdom at this point, but it's worth pointing out that the earlier you find a bug in the dev cycle, security or otherwise, the faster and cheaper and quicker and better is to fix it because you know that whatever I just did, whatever I just checked in changed something and caused something, this alert to trigger, the security issue to be found. So those are the three different types. And that's kind of why we're doing it. Stackhawk, we think we do this really well. So we are built around ZAP, which is a open source scanner. We build it and package it specifically for CI CD. So we make it real easy to use in GitHub Actions, Jenkins, code builds, all kinds of different build systems. We make it easy to run in CI CD and easy to understand what's going on. So we take the results just at a high level. What happens is our scanner runs and pushes the results to our platform where they get aggregated and there's historical information and you can kind of go back in time and see if things changed. Kind of, you can do reports, all kinds of functionality and features around the DAST test that get run. So basically we want it to be fast and easy to use and understand because there's some security tools out there like, hey there's a problem and then you really don't know. It takes a bit of detective work to figure out what's going on.

So we try and make that easier. Okay. So these are the kind of high level steps for the workflow. Time to do some stuff. Make some things happen. All right.

So step one, we're going to use GitHub actions to automatically build a GraphQL application. So the step one, I talked a little bit about it. We're going to take a vulnerable GraphQL application. The way we're going to do this is we're going to fork that into, I'm going to fork it into my own personal GitHub account so I can modify it and then add it to a GitHub actions for a CI CD pipeline.

And then step two, we're going to add the SCA, the Dependabot scanner, so that we can get alerted when, if there are any problems with the dependencies. And we'll go through that, kind of look at how to set that up. There's a ton of different options around that. We're not going to go through them all, but some useful ones, some helpful ones, the ones I use, I will go through those.

And then the third, step three, we'll add GitHub CodeQL, which is a SAST product. It'll scan the applications code base. And if when it finds anything it will, on the commits, it will let us know.

And then step four, add Stackhawk to scan an instance of the running app. So this is during the CICD build process. We'll add Stackhawk so it can run the scanner, find any vulnerabilities, it'll report them back to the Stackhawk platform. The other two, actually, the results show up in GitHub. So we can look at things there, but with step four, it goes, the results get reported back to Stackhawk. So we'll go through Stackhawk and look at what the scanner found and kind of what that means and how we can do something with that.

So if you haven't worked with GitHub Actions before, they are GitHub's CICD offering. They have a huge marketplace of different things you can do. It's basically controlled by checking in a YAML file or two into your repo in a specific place. And then once GitHub sees that, it will then perform actions.

So the first thing we're going to do is we are going to fork this Vaughn GraphQL API which is, so CACAH is the StackHawk public repo. We have this vulnerable app here. It's a Ruby app. It's actually a third-party one. If you go, we go look over here. So it's this carve system. This is a CACAH Vaughn GraphQL API, which is based on this carve system one. Oh actually, just to give you guys an example, here it is running locally. You don't have to do this as part of the workshop. I just wanted to show everybody that this is what this vulnerable app looks like. It's your standard GraphQL introspection interface. I just wanted to show you guys what it looked like. So let's go ahead and fork this. Okay, so the Vaughn GraphQL API. So yeah, so go ahead and create the fork.

And so now I've got a fork here. And what we're gonna do is we're gonna basically add the CI to this repo as in GitHub actions. So like I was saying before, what you do is you put a file in this repo that GitHub can recognize. And in this case, it is this build and test YAML. So what you wanna do is in your fork, you wanna do this add a file, create new file, and then the name of the file is GitHub workflows build and test.yaml. And then in our workflow guide or in the steps is this example, not example, but here's the contents of the file. So this is a YAML file. It just basically says, we're creating a new GitHub Action here, we're gonna run it on the Ubuntu.

It only has two steps to start here. We're gonna clone the repo, and then we're gonna build the app. And the app is very simple to build. It just uses Docker Compose. So go ahead and commit that.

So we're gonna commit directly to main, which is fine. So now with that file checked in, you can go over to the GitHub Actions tab. And you will see, based on that commit, if you go look at that commit, that's one I just did. We now have a action called Build and Test here. And if you haven't worked with these before, you can go in and look at the output of what's going on. So if you remember, we had two steps. We had the clone repo and build the app. And those are right here. Clone the repo does pretty much exactly what you expect, just does a good clone on it. And then build the app. It's running that Docker Compose here.

Start to talk about adding Dependabot. So, dependency scanning with Dependabot in the Workflow Guide, that's step two. So, we won't, we'll wait for this action to finish, but actually we don't have to, but. So if you go in GitHub, in your cloned Vulnable Graph API, if you go to the Settings tab, Code Security and Analysis. So they keep changing this, but basically what you wanna do is, in here you want to enable your dependency graph. And what that does is it allows you to then turn on some Dependabot features. And so, for this workshop, what we wanna do is the alerts. So you can read what it does here, but basically when it detects some of your dependency has a vulnerability, it will alert us. And then we want security updates. This one's pretty slick. It allows Dependabot to open pull requests automatically to resolve the alerts. So we will see that. We'll turn this on. And then when it does find an alert, if it thinks it can resolve it, it will open a PR for us, which will be pretty nice. So, you know, this is just a vulnerable example app, but imagine in your real app that you're working with, if when there's an issue if Dependabot finds something, if it automatically just opened the PR for you, that would be a lot, a lot easier for you and your team to deal with than having to go research it and figure it out.

Okay. So with those enabled, now we go over here. So now we go to this, from the settings to the security tab, we go to Dependabot. And hey, Dependabot found stuff. So we have, looks like eight open ones that it found, Axios, inefficient problem here. So you can go into each one of these and get some more detail. So in this one, it's saying, hey, go from the version, you know, 0.18.1 to 0.21.2. And it will fix some vulnerabilities for you. They do some nice things around telling you kind of, you know, hey, is this severe kind of some information about the vulnerability that it found. One of the things that the kind of the security world has, if you're not super familiar with it, is it's got these IDs, the CWE and CVE and the GHSA. And these are just information around the vulnerability so that you can learn more about it. So this one tripped these kind of alerts, you can go and this is a common, basically database, if you haven't gotten into the CWE stuff before. So that gives you information about it. So in this case, we've got some regex security leak here, which upgrading the library will fix. And like we were talking about before, Dependabot actually opened a PR to upgrade that library for us. So if we go in here, we go, oh, cool. So this is nicely formatted PR. Now, this is an example application, I'm not gonna merge this PR, but if you think about it as if this was your application or, you know, my application, your team's application, basically, you know, you'd probably have a bunch of tests around this. So when it did do this PR, at least the way we have things set up for our builds, it would run a series of integration tests, series of unit tests, and all that kind of stuff.

So we would be very comfortable with moving between this version to the new version, would be okay. In this case, I'm just gonna leave this here and go back to security here. Now, it didn't go back to security, Dependabot. It didn't open PRs for all these because it wasn't a clean upgrade. So you can get from information here. So it said the latest can be installed, this is one, five, 10. So there's these other dependencies that it was not, basically, it's kind of a fail-fast thing where it's like, oh, if I can't do a clean upgrade on a single library, then I'm just gonna let you know about the vulnerability, let you know some details about what you could do here. And again, here's more detail on the bugs. So this is what was identified in the library. These CWE IDs are pretty common in the security world. So back to security, a list of dependable things here. So you can go through each of these and kind of get an idea. Now, I will tell you the first time you run this, if you haven't run it against your repo or your project, it's gonna find a bunch of these kinds of things. And so, yeah, dedicate some time to kind of going through and learning a little bit about what they are, how much impact they could have. Like we said before, the thing with the SCA type tools like this dependency scanner is it may or may not be an issue, it just lets you know that, hey, the dependency has a known security vulnerability. So it doesn't actually tell you if it surfaces in your code. So like, if this was something to do with regexes and you didn't use regexes at all, it might not be an issue for you, but this takes a little bit of digging to sort of figure that out. So and we go look at that file.

So instead, as part of the step, we added this GitHub workflows directory using this add file. Yeah, so Nicole just posted the guidebook. If you go in there and go down to step one, the second step of step one, there's the build and test YAML. And so in dot GitHub slash workflows slash build and test dot YAML, we have this. And we saw those steps over in the action. So we've got clone repo and build the app. So if we go look at the actions, we go in here, we got clone repo build app. And then once you get that YAML file in there, then just go head on over to settings, code security and analysis, and you can enable dependency graph and then turn on the first two options or enable the first two options for dependabot. And then you can actually, those run as GitHub actions as well. And then you can see those runs here because we had, it did a PR. So it caused those two actions to run. So that was in the workshop guidebook. That was step two, dependency scanning with dependabot. And we can go into security dependabot and here are what dependabot. Here's what dependabot found. So I think we're going to move on to step three here. We're going to enable static code analysis with code QL. And this is also done in GitHub. So we're going to go to security code scanning. So in the repo, you go to the security tab, go to code scanning, go to configure scanning tool. And hey look, the very first thing is code QL analysis. So we're going to go ahead and configure this. And this is yet another GitHub action. So what this, basically, when we go to configure the code QL, what it's going to do is it's going to drop another GitHub actions yaml file into that dot github slash workflows directory off the root of the repo. And it's going to have a bunch of configuration stuff in here. We're not actually going to change too much. We're just going to commit this as is, see how the default works cause it's actually pretty good. So, just to go through that again, so you go into security code scanning, configure scanning tool. Oh, for me it comes up right away if it doesn't come up, you can just search for code QL. And then it will be able to find that workflow. You want the code QL analysis by github and go ahead and just click that configure button. And all it does is it just adds this file into your repo. So here's the file. You can look it over if you want. I'm just going to go ahead and commit it here. So on the right here you do start commit, commit directly to main, commit new file. So now in our dot github slash workflows directory, we have two files. We have the one we added for dependabot, which actually was just, not for dependabot, sorry. The one that we added just to get the github actions to build the application. And then this code QL one that got added when we added the code QL workflow. So if we go to actions now, we have those two show up and you can, so this is, by default, you see all the workflows. We have those two defined using those two YAML files in the github, github dot github workflow directory, and then the commit that I just did to add that code QL YAML triggered two of these. So we have the code QL one, which is the new one. And if you go in here, you can what's analyzing JavaScript. Cool. It's doing all kinds of stuff. And then we go, and then this was the build and test one that we added. It's just by now, by now this is kind of old hat to us cause we've already seen this, but it's basically building that. Okay. So code QL completed.

So now if you go in the repo, if you go to the security tab before when it clicked on code scanning, we got that prompt to go search for workflows, but this time because we have one installed, it actually went through. Now what this has done is it's gone through and it scanned the source code of the repo and it's looking for known security, vulnerability patterns in the code itself. And it found four of them here. And like we were talking about before, the cool thing with the SAS products is that it highlights the line of code for you in the file. So the way this app is written, again, this is a example of a vulnerable app. Someone put a secret into the code. And so that's, I would agree, that's a critical thing. Whoops, sorry. So here it is on line 42 and app.ts. The secret's in here. It gives you more information about that, including like, hey, don't do that. Use environment variables or some sort of runtime resolution. But it's nice because it tells you the source. You can go look at the source directly, if you don't believe it. Maybe it's secret. Oh, no. Not the line of code. It shows you line of code, sorry. And then it gives you some code examples here. Again, here are these CWE IDs. They're going to link to the MITRE.org so you can learn more about them there. So that's the first code scan, code QL alert that I've found.

If we go down here, we can look at some more of these. So this one uses user ID, which is just a very finite random number here. So we probably want to beef that up, make that a little better. And again, it's the same kind of thing. We can go in here and say, Hey, use some examples of writing some better code to get that random ID. And then the third one here, yeah. When you log out to the console user information, maybe there's sensitive user stuff in there like user IDs or credit cards or who knows. So definitely want to take a look at that. So you want to either mask out your logs or don't log that, or there's a number of different ways to solve it. But what this is doing is telling you the line of code. And also the, you know, like, does it think it's critical or high? And then this one here, cookies transmitted in clear text without SSL. So that's code QL for this app. It's pretty nice. So at this point what we've got, we've got our dependency security scanner going that lets us know when it finds problems in our dependencies, or known vulnerabilities in our dependencies that may or may not affect our application. Code scanning is a little bit more direct. Oops, here we go. So this is saying, hey, in this line of code, this looks like a problem. So that's very useful. It's certainly worth looking at, but it's, you know, again, if that's in a test or if it's in, you know, some piece of code that doesn't actually get invoked, maybe it's not an actual issue in your running application. So, but it's very good to have. But, so the third one we're gonna do is StackHawk. So we're gonna run our scanner, which is going to build a standup, an instance of the application in GitHub actions, and then run the scan against it directly. And then if it finds problems, when it finds problems, you know that's a problem in your running application. So let's go ahead and get that going here. So this is step four we're doing here. Dynamic app scanning with StackHawk. So in order to get this working, we're gonna sign up for a new StackHawk account. So I'm signing up for an account here. When I do this, it's gonna send me a confirmation email. If you don't have a StackHawk account already, you can use your GitHub login. You can use your Google login, or you can create an email one. I created an email based one here just because I have so many. I went through that, created it, and it sent me a confirmation email. I clicked on that, and I landed here. So creating a new account in StackHawk, so I'm gonna say, yes, that's fine for the org. I'm using that email based one here. So we have an option, you can either load in some sample data or just start scanning. We're just gonna start scanning here. Okay. So what we have to do here, we're gonna be copying and pasting a little bit of information out of StackHawk into GitHub here. When you get to this point, you have the scanner set up. This is the first step of the onboarding modal here. What we wanna do is we wanna do, we're not gonna do the CLI. So you can run StackHawk either as a developer tool locally, we have a, it's just another command line utility called Hawk. There's different ways to install it on OSX. You can use Brew, but there's other ways to do it too. We're not gonna do that as part of this because we're running this in GitHub actions. So we wanna run it in Docker. So we're gonna, in the scanner setup for StackHawk, the scanner type, we want StackHawk Docker image. And then this is the part we're gonna cut and paste from StackHawk into GitHub.

So what we're gonna do here is we're gonna take our API key out of StackHawk and you can either select it like that or just click that guy. So we wanna copy the API key, the StackHawk API key. We're gonna go back into GitHub here and we're gonna add this as a repository secret. So we go into settings, this is in your Volny clone, Volny App Vulnerable Graph QL API. So we go into settings, secrets, actions. So in settings, there's a secrets tab and actions underneath there. And in actions, we wanna add a new repository secret. So what we're gonna do is we're gonna take the API key out of StackHawk. We're gonna put it into GitHub as a actions secret so that when the action runs, it can just look that up. In this case, we're gonna call it Hawk API key, all caps. And then we just cut, we paste the Hawk API key out of StackHawk into here. So just to recap, in the first step of the StackHawk setup, we click the StackHawk docker image. So you have two options, CLI or docker image. We want docker image and we want this API key. Copy that. And we go over to back to GitHub for our repo settings, secrets, actions, new repository secret, name, Hawk API key. Secret is the key out of StackHawk. Go ahead and click add secret. And so now we've got it. So now this environment variable will be available to all the GitHub actions, which will have the API key.

Alrighty. So that's cool. So now we've got that saved in GitHub. We will configure the rest of that in a sec, but we gotta go back to StackHawk here.

So scanner setup, we're gonna click next. App details. The way that StackHawk organizes the scans, keep in mind that what's happening is you have your GraphQL application that's gonna run. StackHawk has a scanner that's going to go essentially attack that application. And then it's gonna report the results, whatever it finds back to StackHawk. And the way it reports those things is it groups them into applications and environments. So here's where we're gonna set up our first application. Depending on your team and organization, there's no real one way to do this. A lot of folks will do it by service, so maybe I'll call it UL, so we'll call it by the service. Beyond application, we group things by environment, so we'll call this development. And then we want to put our host. For host, no. So what's gonna happen is GitHub Actions is going to run the Valny application in the CICD pipeline. It's gonna run it in Docker. In Docker, it's gonna run on port 3,000, effectively what's the local host for that CICD pipeline. So, as far as the StackHawk scanner knows, it's running local host port 3,000 with no SSL. No SSL. So go ahead and put http://localhost3000.

The next step here is telling the StackHawk scanner what kind of application it's gonna be testing. So, there's lots of options here, we support lots of different things. The one we want is API. So GraphQL API. And what this is gonna do is, I talked a little bit earlier about how, what StackHawk does is it takes the ZAP scanner and it configures it specifically for a given use case. This is part of that. So what this is gonna do is it's gonna tell the scanner to load up the GraphQL, vulnerability tests. It's gonna configure kind of the introspection. It's gonna configure a bunch of stuff for you. So you wanna pick in the app type application type API and then API type GraphQL. And then when you pick GraphQL, it wants to know the introspection point. And it just tax that onto that URL, that host URL that we gave it previously. So this looks right. So we're gonna have HTTP colon slash slash localhost colon 3000 slash GraphQL. So that's gonna be where the scanner is gonna go figure out what this particular application has in terms of data types and mutations and all that kind of stuff.

Okay, so this is the last step here. What we wanna do is we wanna take this again, we're gonna copy this StackHawk YAML. So this is very similar to how the GitHub actions work where you check in a YAML file in a specific place in your repo. And then the actions or StackHawk know where to find that. If it sees it there, then it will do stuff. So what this is, is our StackHawk configuration file. Basically it's called stackhawk.yaml. If you look through it, we've got the ID of the application we just created, the M, there's the host, localhost 3000. So this is the customized part for GraphQL. So we want GraphQL enabled. There's the introspection point. We're gonna go for all operations. We're doing posts here, auto-input vectors, all that kind of stuff.

So we're gonna go ahead. What we wanna do is we want to copy this, go back to GitHub, go back to our Volney app here, go to the code tab. And like we did in that first Dependabot step, we're gonna add a file. So we wanna create a new file. I'm gonna call it just Stackhawk, stackhawk.yml. And then the contents are just pasted right out of the wizard. The stackhawk modal, or setup guide, they used to be called wizards, I don't think anyone calls them wizards anymore. But anyway, so we wanna cut and paste the stackhawk.yml out of the stackhawk setup flow into here as a new file, and then we just go ahead and commit this.

If we go to actions now, so now we're set up here, so now we have to add, now we have to tie the stackhawk scanner to the build step. So I just created that stackhawk.yml so that triggered our two existing steps here, or two actions, so we have build and test, which just builds the application, and then the code QL one. So the last thing we have to do here is, I have those two optional steps in the guide, if you didn't use the setup flow, it just kind of reiterates that stuff. So what you want to do is add a stackhawk scan to your build and test workflow. That's the step I'm on right now. So add a stackhawk scan to your build and test workflow. So what we're going to do here, is we're going to go back to the code. We're going to go into .github slash workflows, and we're going to modify that first build and test.yaml that we created for the Dependabot step. So it had those two steps, we had clone, repo and build the app. And so what you want to do here is just, we're going to add some more steps. So we want to get the file here and just click on edit this file. And then we're going to add two more steps. And they're in the workflow guide. The whole file is in the workflow guide. You can just take the last two steps here. We're going to have run the app, which just does a docker-compose up. And then we have a new step called hawkscan, and it's going to use basically a pre-built action called hawkscan action. And it's going to use that API key secret that we put in earlier. So the way you reference those is you say secrets dot and then the name of the key. We named ours hawk API key. So we're going to go ahead and specify that here. So what that's going to do is it's going to run the scanner. It's going to pull in that API key from the secrets repository. That API key, it will use that to then upload the results. It actually streams them back to the Stackhawk platform. And yeah, and then we can actually watch that. So we'll put that in here. Get this going here so we can close this guy. So we're adding in these two steps. So when we commit this now, so we're going to commit that. We see we have run the app in Hawk Scan with Docker Compose up, and then the Stackhawk Action with API key. So with this new commit, we have our new steps here in the build and test Action. And you can see that they, I don't want to see that, they showed up here. So now we have, after build the app, we have run the app, which is that Docker Compose of Dash D. And then after that, it will run the scanner. So if we go back here, if we go back to Stackhawk now, we can basically, we're finished. So now we're just waiting for the scan results to show up here. So as part of the build process, that's what will happen.

The question is, how does CodeQL work? Does it use keywords? For example, console.log user, does it look for the keyword user, or some other technique? Do you have any insight on that? Yeah. It has a parser, and it looks for patterns. So it's not quite a compiler. Like it doesn't actually compile the code and then look at execution flows, although some of them can do that.

I don't think CodeQL does that. I think it's essentially a very, very sophisticated regex machine. So it looks for this one here, the insecure randomness. So it has a bunch of patterns that it looks for for different types of source code. So for JavaScript, and I don't know the exact implementation, but I would guess that it looks for random int, and it says, oh, it's putting some kind of a narrow range for a random int. And then user ID. Yeah, I don't know. It must look at that and say, oh, something ID. That's important in the security context. So that's kind of the downfall of SAST is it doesn't, it may or may not be a thing. It's just looking at the patterns. It can only go to a certain point as to whether it's an actual vulnerability or not.

Yeah, so here's their thing. So random source get, that's probably like the trigger, like, oh, look at this. They're using get random values. Now we're in the running. So the scanner is running right now in GitHub. So if we go back to Stackhawk here, we can go click on the left here. We can look at our scans. So here's the scan that we started. So a volume GraphQL. It started, like I said before, it's actually streaming results back. So the scanner is running right now against that instance of the GraphQL application. You can watch it go here and see kind of what step it's on, what it's doing, which or how long it took. So it's going to go through all these different things. And these are all the ones that are tuned for, remember we picked, it's a API and specifically it's a GraphQL type. So it's going to, it's tuned to that type of application. So it finished, scan completed, and it found some stuff. So this is very similar to the CodeQL that security list, vulnerabilities list in GitHub for the CodeQL one. But this is things that are actually found in the running application. So these are things that you definitely wanna look at. So these are the findings, just before we dive into the findings, this is the paths it looked at. Now for GraphQL, because the URL is always GraphQL and it just texts on parameters, this isn't super, it's not gonna be a lot of different stuff for a traditional site. It'll have a bunch of routes here. But it does have operations, found 11 of them. I'm not sure what's going on there. Okay, so we'll go, there we go. So these are the operations that it found in our GraphQL application. So it found a bunch of mutations, including one, called super-secret-private-mutation. What? That doesn't sound good. And then the queries. So we got search, post, user, all that kind of stuff. So like I was saying, paths aren't super interesting for GraphQL, but operations are. So we detail them out here for you. So these are the things it found, operation-wise. And then these are the security findings that it detected. So we can go into each one of these. So because it's a DAST scanner, it's testing the application, basically, through that host and port, we don't know about the source code. So the codeql, the SAS products will tell you which line of code they found that pattern on, where that user ID was. But, so we don't, STATCOT can't do that. But what it does do is it traps the request and the response, and then points out the evidence that triggered this alert, this finding. So here we have this remote OS command injection. Again, we have CWE ID, which is, you know, the same, it's the same database, same thing at statmitre.org. We've got details here. We've got the, it was a mutation, super secret private mutation. Hey, the evidence is right here. So here's the request and response. So this is where it's, this is the good stuff. So there, this is a request it made. The command variable was cat x etsy password. That's not great.

That's not great. And what's even worse is that it actually did cat etsy password. And so right here you can see the evidence is highlighted as part of the response here. And so that's, you know, that's not a great vulnerability to have in your GraphQL application. And here's some more information on it. ScanRule was able to retrieve the contents of a file on the operating system. So instead of source code, we have this request and response objects that we can look at. If you want, you know, you can look in here. You can see the headers. It was a post GraphQL. Here are the cookies that got used, but here's the variables that got posted.

So there's a question. Is the GitLab implementation for StackHawk similar to the one for GitHub? It is the same one. So it's the same, StackHawk stays the same. The scanner is the same. It just knows, all it does is it gets run in whatever the GitLab actions equivalent is. So in Jenkins, it would run in Docker. If you're running it at the command line, I do that all the time on my box. It runs and then it just, it runs against the application. It just runs against the host and port. And then it reports the results back to StackHawk. The limitations, the only limitations I can think of would be around maybe some environment stuff between the two, but really everything's configured via that StackHawk YAML file. So we'll go back here. We can look at the StackHawk YAML. If you go look at the docs, this file can get really big and complicated, but there's a lot there you can do. So it really is the same. It's the same scanner between the two.

Okay, so another thing that we do is we provide this curl command, which lets you reproduce it. So this is how the Hawk scan, this is how our scanner found it. And so we basically give you this. So you can imagine if you're doing this locally on your machine, you're trying to develop some things, you run the scanner, it finds this problem. You can reproduce it using this recreate request. You just copy this, paste that down to your terminal and run it. You can see the query here, mutation super secret. And then the variable, yeah, variable cat Etsy password. And then another thing you can do is, let's say you're okay with this or no, let's say you're not okay with it, because it's not great. You wanna assign that. So you can mark it as assigned. You can mark it as false positive, if you don't think it's an issue, or risk accepted. It's just basically a way to kind of keep track of these. And then the next time the scanner runs, it will sort of, it will remember that and say, oh, it's not new, it's false positive. You've already dealt with it. So that was the first one, the second one. You basically go through the same process. So you kind of look at this, you go, oh, it's CWE 89. Here's some more information on it. It's a query, query, query is a single tick. And what did we get back from that? So yeah, it actually just passed that single tick down to the database. So yeah, that's not good. So that could be a lot more malicious. So you probably want to address that. Do some SQL sanitization, inputs sanitization on your input. So those are the two high ones that the scanner found. Then we start getting into the mediums here, but it's the same kind of thing. You look at the CSP policy here has a wild card and you know, the evidence is here. And if you click over on response, it highlights it for you. So it can, so you know exactly where it was in the headers, super helpful, similar kind of thing. So you go in here. And so this is where cross-domain, this is across all of these different operations. So you want to, and your response headers coming out of your GraphQL app, you want to, you want to be a little bit more better about cross-domain configuration here. All right, so if you were doing things at the command line, you can tell StackHawk, or even through the Docker command here, you can tell it to rescan and it will only run things.

It will only run the tests just for the findings that it found in this one. So if you have a really long one that takes a long... Basically, the use case is if you have something that takes a while and you don't wanna run the whole thing every time, you can just have it only run against the one, the findings that it found from the previous one. So basically, you say, hey, run the scan, it's a rescan, and here's the scan you start from. So that's a new feature, thought I'd point that out. I spent a lot of time on it. As you run scans, you'll get them in here, and it will let you know the differences between them. I don't think the... I don't think this does this. This just has open ones. But we kind of keep track of things over time, so you can look at new ones that came in, or if you addressed one, then this number will go down, that kind of stuff.

Oh, look at this. You just ran your first scan. Um, yeah. I completely forgot about this over here. Let's see, dependent bot, we went through that. CodeQL. Scans code, adds patterns. Basically, yeah, they just... Oh, it's open source, so you can go look at those patterns if you want. Stackhawk. I think we covered a lot of this stuff. You know, one of the things I didn't talk about was our integrations. So we integrate with a whole bunch of different stuff, so you can have, including like things like Jira, so you can create issues directly in Stackhawk for finding. So if you imagine that SQL vulnerability, if you have the Jira integration hooked up, you can go in there and say, oh, you know what? Send to Jira, and then it will create the Jira ticket for you and tie it back. That's super helpful. We also have these sort of more notification based stuff where you can do send me a Slack every time a scan fails, that kind of thing. Same with Teams. We have a generic web hook, so we can send out events as things happen in our scanner, if you wanna get the payload, if you wanna load them in, if you wanna automatically drop things into some sort of, you know, database so you can keep track of these things on your own, you can do that. I think that was kinda it for the, oh, there's sneak preview custom variables. So this is a new feature that we are releasing like now, I think, or next week, maybe, not sure. No, it's available, but we're gonna do an official launch next week with lots of resources and just materials to help people get up and running with custom variables for GraphQL API's. So what this does is it allows you to put in custom variables in a specific format. So if you have an email field here, you can say, you know what, I wanna inject some fake data in the form of an email, and it will create like a random email-looking string and use that as part of that, like you saw, where did the cat etsy password, it will start putting those kinds of things in those places. So it's really, really great for basically running through your data checks, cause I mean I've written lots of code where you sort of assume it's gonna be an email and then you handle one pattern, but then there's some other thing that comes in you hadn't thought of. So what this does is it lets you inject those in, so you can do it, there's a whole range of them. So there's like emails, phones, UUID, all that kind of stuff. And it will look like pretty, it's realistic looking data that will come in. It really stretches the application. It's great to have. Topher, correct me if I'm wrong, you can supply your own variables too if you wanna test something like a password reset? And yeah, so that's another really useful option. Yep, so yeah, so right above that is the, if you don't wanna use the Faker library, if you don't want, if you know that there's a string that you wanna look for, like let's say you had a password or there's some string that if you had a bug where you know it caused a specific problem and you wanna force that string, you can do that here. So you can specify these custom variables and then it will use those as part of the tests. So a couple of different options here, you can either specify hard-coded strings, like if you know that there's a problem string or like I said, if it's a password, if you really need that, or if you just wanna generate a smart fake-looking data, you can do that as well. And we talked a little bit about the GraphQL configuration, there's a lot of options here. So our docs are trying to be pretty thorough. This workshop is really just about setting up those three types of security scanners. And with those three, I think, yeah, so for this application, now we have three different types of security scanning enabled. We've got the software composition analysis using Dependabot, tells us about dependencies. We've got the SAST, which will go look at the source code for patterns that looks suspicious, and it will alert you to those. And then we have DAST using StackHawk, which ran against the actual running live application and found some issues in there. So we know that those are actual issues and we got to go look at them and triage them, either fix them or say, you know what, we're okay with that for now. But with those three things, we've just added some pretty good tooling around our vulnerable application here, our example application. Going forward, hopefully you can use this to implement this in your own GraphQL service. And then the idea is, you know, once these are in place, when you get a new one, the first time you write it, you're gonna get a bunch of stuff, you go through and you sort of triage and sort of fix stuff and say, no, I don't care about that library or whatever, or I'm going to fix that. And then going forward, whenever you get a new one on any commit, then you know that, oh, hey, that commit changed something and I have to go figure out what the change was and why did this trigger this alert? But, you know, that's a lot better than it going through the whole process and then getting deployed and then, you know, you get an email that says, hey, we found this problem out in your production server. We definitely don't want to do that. So we want to do it as close to the commit as possible. And so I think, let's see next steps. Yeah, so now you basically translate kind of what we did into your environment. So if you're not using GitHub actions, whatever kind of CI CD provider you are using, we StackHawk supports a lot of them.

Dependabot is, I think it's pretty easy. We've enabled it for all our stuff in our GitHub repos. CodeQL is one SaaS provider. Snyk is another big one that's out there. We have integrations with both. So yeah, you can use those.

And then, you know, stay in touch. StackHawk, we got a blog. We're doing stuff all the time. If we've got time, I could show some Snyk integration. So Snyk code is a lot like GitHub's CodeQL and that SaaS tool. It will scan for patterns in your source code and then let you know the line file number or the file and line number of where it found the problem.

And so what we do, our integration with actually both of them, I just don't have the GitHub one ready to go, is we will try and correlate a SaaS finding between Snyk and a Dask finding in our scanner. So basically we'll use that CWE ID to correlate the two so that you can see that not only was it found in the SaaS scanning where it's like, you know what, that user ID doesn't look right or that's a hard-coded secret password. If our scanner finds it, then we will link the two and you can say, oh wow, this actually got found in the running application by the StackHawk scanner.

So let me get this wired up here. So I'm in StackHawk here, we're gonna set up a Snyk integration, Snyk org ID, so we have an account in Snyk for our organization and then just generated an API token here. So now we're gonna link the two together and the Snyk, because it works on your source code, you have to give it access to your repositories because it has to read the source code. So we're gonna, we have our, this is the Volne graph API, so it's the one that we forked basically. So source code wise, it's gonna be the same because we haven't changed anything. And I only have one application in StackHawk, which is the Volne graph API. So that's pretty much it. So you give it your org ID, your API key, your Snyk org ID, your Snyk API key, and then you hook up, you link your source code repo, that you've already set up in Snyk with your StackHawk application. And you say finish. And so now they're here. So if we go back to our repo here, we go to our action. We go to the build and test action here, workflow. We just wanna rerun this guy. It will rerun, and as part of its rerun, it has the Hawk scan step. So Hawk scan here. The difference is this time when it runs the Hawk scan step, it'll run the scanner. The scanner will run against the application, send the results back to the StackHawk platform. This time though, we have the integration with Snyk there. So as the findings come in, StackHawk will, our platform will talk to Snyk and say, hey, do you have any findings for this? And if it does, then it will correlate the two and then we can see that in StackHawk. And so that's really, really nice because StackHawk found the problem in the running application. And then you can go through the integration to the actual line of code that Snyk found. So you kind of tie those two things together. It's a problem where the SAST tools will say, hey, this looks like a problem. This user ID looks bad or this secret looks bad, but it might not actually be an issue in the running application if that piece of code isn't ever used or if it's just a test utility or it could be any number of things. That logic path doesn't get exercised in the code, whatever it is. And then the other side, StackHawk, hey, we found a problem, but StackHawk isn't code aware. It's running outside of your application. It's running against a host and port. It knows it's a GraphQL application, but that's about it. It can't get into the source code. So we're trying to bridge the gap here between the two by this integration with Snyk and the CodeQL one as well. We just released the CodeQL integration. It's basically gonna look very similar to what I'm gonna show you with the Snyk one. I just hadn't updated this workshop because it's so new. So it's building the app here.

Now it ran the app. Now it's running hawk scan. As you can see, this partially answers what's the difference between this and GitLab. The core of it, the scanner is the same. It's just running this hawk command in Docker. It can be run in Docker. It can be run at your desktop. Then all the CICD providers have ways to run things in Docker, or you can try setting up the hawk command directly in there. We're doing some of that stuff with Microsoft Azure right now. They have a way to say, hey, you know what? I'm gonna run a Java command. So a hawk is a Java application. So you can set up an action that says, hey, don't run the scanner in Docker. You don't wanna run it directly, but you have to have Java set up, and then you have to download the hawk. We provide it as a zip file, so you can download the zip file, unzip it, and just run it directly. So that sort of depends on how the CICD system works, how well it supports Docker, how well it supports just running arbitrary third party stuff. But anyway, so this is the same output as if I was running it on my laptop, or if you were running it on your machine. So it runs this hawk command. There's the stackhawk.yml file specified. It's in the route of this project now. So it starts up, you can see it's running version 2.9. Here's the, you know, localhost 3000. That's that host that we configured when we were setting up. All done. I'm gonna go through this though. So here's, you can get a little kind of a taste of what it's doing here. So it's, it found 11 GraphQL routes. It's gonna try them all. Goes through here, does all that, runs, you know, a bunch of status stuff. And then at the end of it, for the command line, we actually output this little helpful summary here. So you go in here. So there's that remote OS command injection. Here's the SQL injection. It's basically the stuff we found. I don't think it prints them all. I think it only prints out some of them. But, so it ran, that's the output. And then if we go back to here, so now we've run another instance. So at 10 o'clock, Denver time, we ran the first one and then we just ran the second one here, but with the Snyk integration. So now we have indicators here telling us that some of the findings got correlated against assessed findings in Snyk. So if we go into SQL injection, we go, okay, cool. We looked at this one previously. We go, yeah, it's that query. It's a single tick. That would cause a problem. SQLite error, remember this one? Yeah, sure, we're just passing those down. We're not sanitizing the inputs. The difference now is now we have this Snyk code tab here. We can click on this and we go, oh, CWE89. Wow. Snyk found that in this post.ts file.

And if we go here, we go, oh yeah. Yeah, that's, so if you look at this thing, it's taken this query right off the arguments, right off the URL. It's not doing anything with it. And it's just, oh, arguments, sorry, the arguments. Yes, so it's passing this argument straight off the query into this. Yeah, so a single tick would cause this like, basically to be parsed and finished. You'd have some extra stuff. So yeah, we need to address that. But you can see how this is a problem detected in the running application. And then this is the actual line of code that looks pretty suspect and needs to get fixed. You can also link back to Snyk and look at it.

Similar kind of thing where they have, here's the line of code, a little bit more information on it. And then there was another one. So this server leaks X powered by... So here all the operations that were detected to have this problem. So we go into the Snyk code now, and it's maybe in a couple of different places. So we go here, doesn't like that. And it doesn't like. Oh, yeah. So, yeah. So this one is... This one too. Yeah. So by default Express sends this X powered by header. And so it's just leaking out some information about your application. So if a malicious third party, you could say, oh, I know this is an Express app. So now I know I can go probe for certain Express vulnerabilities. Yeah, so you can use helmet middleware to sort of not disclose that information. But yeah, there's two places in the code. You should go look at that and figure it out. Yeah. So that's our sneak integration. It's pretty nice, you can see between the scan runs here, we've... Nothing really changed because we didn't change any code and do anything. The only difference is we added a sneak integration here. And then like I said before, the other SAST provider that we integrate with is GitHub CodeQL. Yeah, CodeQL. And it looks very similar. In fact, you can have both running at the same time, I think, but it would have, it would just basically have CodeQL markers here and links would be back to CodeQL. So that is trying to bridge those two. We had the Dependabot software composition analysis. We have SAST, the static analysis and the StackHawk dynamic analysis. We're really trying to bridge those two together because being able to say, hey, this line of code is causing this problem in your running application is really powerful. And it's a great thing if you're an engineer, like being able to know, being able to see that, like when you do a commit, you commit a bunch of changes and then this thing triggers and it's like, oh, you know what, this line that you just added has this problem in it. Doing that at the commit time is super powerful. It just, it makes things real easy and fast. That is all we've got. Yeah, next steps, scanner CI updates. It's all good stuff. Hopefully this translates pretty easily into your GraphQL apps, the CI piece. Like I said, if you're not using GitHub actions, we support a ton of them. The basic idea is the same that you set up, if it's GitLab or Jenkins or whatever, you basically set up the scanner, the StackHawk scanner to run in your CI, CI, CICD pipeline against your application. And then it reports things back. The Dependabot and CodeQL are, because those are operating against, you know, source code, you don't have to, you don't have to have a running application. Those are a little bit easier to get going. Sneak is very similar. You basically, when you go to run, go to use sneak, it will want access to your GitHub repos and you say, yeah, that's cool. And it checks them out and then runs, runs its pattern matching against your repos directly. But it's the same thing as a CodeQL that we did. Hopefully that translates, see there's Sneak or CodeQL, it's very, very similar. And then we've got our updates, stay tuned.

There's always new stuff coming out. I think, yeah, I think that wraps it up. Like Topher said, I mean, we kind of went through like a really robust security system here with various types of security testing going on at once. So thank you for that Topher, that was awesome.

And like he mentioned, we went through a couple specific examples using particular tools to make it easy, but there's so many tools you could use. CodeQL can be swapped with Snyk, and I believe Snyk even has like a software composition analysis tool as well. So all of these work, you know, hand-in-hand together, you know, to give you really complete coverage of your APIs and applications.

So please feel free to use any of those resources that we dropped in the Discord to go further, you know, with this experimentation now. I mean, if you were full in the workshop, you've got a StackHawk account, you're on a free trial, I believe it's an enterprise level free trial when you go through that. Yep, so you've kinda got access to everything during that time. So play around with it, connect some things, and our team will also be in touch with some more resources on how you can just play around and explore with StackHawk and connecting some of these tools. So, yeah, thanks everybody for joining. Thank you Topher for going through such a good workshop. Hope you enjoy the rest of your day.