Introduction to the AWS CDK: Infrastructure as Node

Hi, everybody. Thank you for virtually attending my talk. I'm going to be talking about the AWS CDK today. And the title of my talk is Introduction to the AWS CDK Infrastructure as Node.

So, I guess the very first question to answer is, what is the CDK? So, it's an abbreviation for Cloud Development Kit. It is an AWS-specific infrastructure as code tool. It's similar to tools like Terraform that you may have used in the past. It supports a variety of languages so you can write your applications in JavaScript, TypeScript, Python, Java and C sharp. I believe there are plans for other languages in the future as well. And the way that it works under the hood is that high-level JavaScript or TypeScript classes map to these things called CloudFormation templates, which I'll talk about again in a couple minutes.

If you've used the AWS CD or SDK in the past, the CDK is not the same thing. The AWS SDK is used for making simple API calls, whereas the CDK is actually able to maintain state about your application, allowing you to update it and redeploy it without having to go through the same checks that you would if you were just making direct API calls. The CDK itself is fairly new. It reached general availability in the middle of 2019. So there are still some AWS features that are not supported yet, since AWS is quite massive. And then it's also free to use, but I've included an asterisk here because while the CDK itself is free to use, if you start using it to deploy things like EC2 instances or other AWS resources, you will be billed for those things.

So I guess next, you might be asking why would I use this? So as I said before, there are lots of tools out there like this, things like Terraform and whatnot. So I'm going to actually a report from the Cloud Native Computing Foundation that surveyed approximately 17,000 developers. If you disagree with these quotes, please don't shoot me, I'm just the messenger. But, JavaScript is the most popular server-side programming language among cloud-native developers, and 62% of cloud-native developers are using AWS as their cloud hosting provider. So the CDK really meets developers where they're at. There's already a lot of developers out there that are writing applications in the cloud with JavaScript and TypeScript and deploying them to AWS. So, by using a tool like this, you can write your infrastructure as well as your applications using the same tools and languages. And that's kind of always been one of the upsides to Node.js was that it allowed front-end and back-end developers the same common language and this just kind of extends that even further into your deployment and ops.

The CDK also offers you testable infrastructure, so there is an AWS CDK slash assert module. It integrates really well with Jest and snapshot testing. And then, the last thing I wanted to mention is you really shouldn't worry about vendor lock-in because you're probably already locked in unless you have the most extremely trivial app. Being able to go from AWS to something like Azure is probably going to require some effort in porting. So, the CDK is not going to lock you in any more than you're kind of already locked in.

And then, before I dive in a little too much further, I wanted to something that I found out while I was researching the CDK that I thought was kind of interesting, was something called the JSII, which stands for the JavaScript Interoperability Interface. And this is kind of the tool that sits underneath the CDK that allows it to be written in TypeScript, but still supported by all the other languages that I named earlier. And when I originally read about this, I thought that it probably worked by just generating code that was kind of language specific. But actually, under the hood, it relies on a Node.js runtime, and what it does is it will execute your TypeScript code and send it back and forth, the input and output back and forth as JSON, between the Node child process and whatever host process has been running your application. And so there are some generated bindings that are part of the CDK output, so you can write your your code once and you'll get the same API across all of these different languages. And that does unfortunately come with a performance hit, because you're spawning a Node process, you're interacting with a child process, instead of generating the code directly. But I just thought that that was something that was kind of cool and we wanted to share as part of this talk.

So back to the overall point of this talk, so I had mentioned CloudFormation earlier. CloudFormation is a well-known AWS service that basically allows you to define your architecture, your resources that you'll be using in the cloud as a json or a yaml template file. And then it can be deployed to you know via the CloudFormation service. It will take care of you know creating all the resources for you that you need, such as instances or Lambda functions or anything like that. And it can actually manage large groups of resources as a single unit which is called a stack. And so stacks can be created, they can be updated and modified, they can also be deleted and everything happens to that entire group as one single entity. So it makes the the maintenance overhead a lot simpler for for developers. And as I kind of alluded to earlier, the fact that it maintains state between your deployments allows it to be much more powerful than individual AWS SDK calls, where you would kind of have to manage all that that state yourself. And then within CloudFormation, there are basically inputs and outputs to your to your CloudFormation stack. The inputs are called parameters and these are good for passing in things like sensitive information, or if you are deploying a website maybe what the domain name is for that stack and things of that nature. And then it can also generate output. So if you tell it to provision an S3 bucket you might not know what the name of that bucket is ahead of time. So that can be used as an output which can then be imported by another CloudFormation stack. So you can see that it's kind of composable and very powerful, but the biggest drawback is that CloudFormation is very verbose. So to maintain these JSON and YAML files by hand can actually be quite a bit of work and it's very error-prone. So the CDK kind of saw the power of CloudFormation and introduced this API on top of it using TypeScript and what it does is it gives you these building blocks called constructs. Constructs are kind of the way that you build and compose together CDK applications and there are a variety of different types of constructs at different levels. So, for example, the level 1 construct.

These are basically CloudFormation resources. CFN here stands for CloudFormation in shorthand. For an example, there is a CDK construct called CloudFormation bucket and this is just as bare-bones as a wrapper around CloudFormation as you can get. But then level 2 constructs take that up a level to introduce an intent based API. Building on the same example, instead of a CloudFormation bucket, you might use an S3 bucket and this has a much more user-friendly API. And then finally there are what are called patterns which are even higher level and can compose a lot of different components together. These can really accelerate your productivity. So for example, you can, instead of having to create Lambda functions, wire them up to an API gateway and things like that, you can just use the Lambda REST API pattern. Like I said, it can really accelerate productivity.

And so the way that these applications work are you take your various constructs and you compose them into what's called a construct tree. And then it's also worth pointing out that AWS provides a large number of constructs from their library, but it's also possible for user LAN to kind of create their own constructs, to kind of be more tailored to your need. So there are, I guess, two special kinds of constructs that are worth discussing a bit. There are apps and stacks. So the root of the construct tree is always going to be an app component. The reason it's called an app is because CDK applications are referred to as apps. Like I said, this is always going to be the root, but then stacks are constructs that map to CloudFormation stacks. So I mentioned that in CloudFormation, groups of resources are kind of grouped together as a stack, and this is kind of a one-to-one mapping there. So apps contain stacks, so you can have more than one in a single app, and then the stacks are usually what contain the other constructs. One thing worth pointing out is that each stack is configured by an environment, and I'll talk about that right now. So an environment is an AWS account ID, plus a region where the stack is going to be deployed. So this is basically saying this user is deployed to this region in the cloud, and so an environment has to be provided for every region that you're trying to deploy to, and for every separate account that might want to deploy your stack. If you don't specify an environment, then the stack is what's considered to be environment-agnostic. But these are of pretty limited use. And then to provide the environment, if you've ever worked with any of the AWS tools before, there's a variety of ways to do this. In the code itself, you can pass in values to the stack constructor, but you can also pass in a profile flag from the command line, or specify different environment variables, and it will automatically pick those up in the code and know where to deploy your stuff. And you can also set up your AWS profile so that you have a default environment, but that's more of a local development type of use case.

So with all of that, I think we can actually generate our first CDK application. So I've added a simple command line up here using mpx, so you don't actually have to install anything yet. mpx AWS CDK, that is the name of the module. Init app, and then we're passing the language, we wanna use JavaScript here, but you can pass in any of the other supported languages.

And so this will create a full application skeleton for you. One note is that you have to run this from inside of an empty directory so that it doesn't accidentally overwrite anything that might exist. So this creates a bin folder, and so you can actually run this file directly. You can see the little shebang at the top of the code here that most node binaries include.

So what this does is it imports a test stack that was also generated for you, instantiates your application with the new cdk.app constructor, and then it just creates a new test stack. And within the test stack, you can see it passes the app as the first argument, so that is how the Construct Tree keeps track of which constructs are children of which other constructs. And then we're gonna be calling this one test stack, so that's what the second option is there.

So looking at the stack itself, you can see this is the code that was generated. I left in some of the comments and other things that were automatically generated, but in future examples, I'll be removing those to save space. So the first thing that we do is import the cdk itself, and then we create a TestStack class, which extends the cdkStackBase class. The constructor for a class takes a scope, an ID, and props. The scope is the parent construct, as I mentioned before. The ID is what the logical ID of the construct is gonna be whenever we synthesize and deploy these applications. And then props are an optional map that can be passed in that are used to define other properties of your construct.

So if we wanted to add more constructs, then we could do something like this, npm install dash dash save awscdk slash awsec2, this will basically install the cdk constructs for the EC2 service. In this simple example, I'm adding a VPC to our stack. The reason I'm doing this is because VPCs can be deployed at no additional charge, so if you wanted to experiment with this at all and deploy it, you should be able to run this without actually incurring any charges.

So next, I wanted to talk about how these applications are kind of synthesized and deployed. So you can see we have the overall workflow here, you'll see cdk CLI at the top. Cdk deploy is one of the CLI commands, I'll talk about what some of the other ones are later. And what it does is it reads in your application source code, it goes through what's called the construct phase, where user code is run and all of the constructor chain is executed. So it's basically building the construct tree. The prepare and validate phases are hooks that are available to constructs, but they're generally not needed. And sometimes they're actively discouraged. And then next comes the synthesis phase, which generates something called a cloud assembly. The cloud assembly is going to be your cloud formation template, as well as any other deployment artifacts that are needed to deploy your app to the cloud. And I'll talk about what those other deployment artifacts are in a little while. And then the deploy will actually take your cloud assembly and you know, upload anything that needs to be uploaded to the cloud and start a cloud formation deployment. And once that succeeds then you have a CDK app that is actually deployed to the cloud. You can actually do the synthesizing step as as a separate step without needing to deploy it, which is nice for like local testing and unit tests. So you would do that via the MPX AWS CDK, synth, and then the name of your stack. So in this case, test stack, this will generate your little translate your CDK app into a cloud formation template. It'll generate any of the deployment artifacts, such as Docker images, if you're deploying any lambdas, it will bundle those up for you. And then it'll write everything to both standard output, since this is just a synthesis and not a deployment, but will also create your CDK dot out directory, which is where everything is kind of stored before it's deployed. And then you can see to the right here, I actually have captured what the the cloud formation template looks like. And you can see that, you know, it's taking up the entire height of the slide and is very unreadable. And that's, you know, based on a CDK app, that was roughly a dozen lines of JavaScript. So you can see where the, you know, the increased verbosity comes from with cloud formation, as well as you know, the expressive power of the CDK.

Next, I wanted to talk about assets and bootstrapping. So any local files or directories that need to be uploaded to S3, or any local Docker images that need to be uploaded to the elastic container registry, are known as assets. If your application uses any of these assets, then you'll have to do something called bootstrapping, which is something that has to be done per environment, so per per user and region combination, it will also create an additional cloud formation stack for you called CDK toolkit. And it consists of an S3 bucket and an IAM set of resources for deploying the assets to the cloud. One thing to note is that you will likely be charged for this because you're you'll be storing assets on in AWS. So I wanted to look at kind of a more realistic example.

Now we'll deploy a static S3 site. I included an asterisk here because again, you will be charged by AWS if you run this. So I've created a new stack here called the website stack, which extends the CDK stack class. And then inside, I'm creating a new s3 bucket. So I'm going to call my bucket website bucket, I'm going to make index dot HTML, the the index page for my site that I'm deploying. And I'm going to grant public read access to the world so that anybody can, can come and view my site. And then to deploy it, I'll use the S3 deployment bucket deployment construct. So I'm attaching it to the stack, I'm calling this deployment, my website, the sources are going to come from a local directory that I'm calling www, that will actually create assets, as I mentioned in the previous slide. That's the part that you're going to be charged for. And then it will upload everything to the bucket that we just created called website bucket.

One thing that's worth pointing out here, this is kind of a small example, because it's it needs to fit on a slide. But you could take this further and add things like DNS or CDN to really make a nice static site for yourself. So then to actually deploy the site, we would say mpx AWS CDK deploy the website stack, and then I'm actually going to pass in my AWS profile from the command line so that it knows what environment to deploy to. But the first time that I run this, I'm going to see the following error. It says this stack uses assets, so the toolkit stack must be deployed to the environment. So what that means is I have to bootstrap this environment first, and you can see below I've added the mpx, AWS CDK, bootstrap, and then pass my profile that will create the bootstrapping infrastructure that I spoke about before, then I can rerun the same deploy and everything should work according to plan this time.

So if you want to see what those deployed resources look like, in the top left here I have the CloudFormation console that shows the app and the bootstrap stacks. The top right hand corner is the S3 console that's showing the app and bootstraps buckets. Bottom left is the app bucket containing the static assets. And the bottom right is the site actually deployed and running in production. Like I said, if you wanted to make this a little fancier, you could do something like add DNS and the CDNs. You could have a more user friendly URL there. And then to clean up, you'll use the CDK Destroy website stack. I also recommend that you go into the web console for AWS and look around S3 and CloudFormation to see that all the resources are cleaned up. There I've linked to CDK issue below where bootstrapping does not always clean up after itself. So I wanted to kind of provide you with commands here that you can use to clean up or just click through the web console and you should be able to do the same cleanup by hand. And then I wanted to wrap up by talking about the CLI. So these are all basically the commands that we use to do the deployment. You can install the AWS CDK locally. If you do that, then it becomes known as CDK and not AWS CDK.

So that's one thing to note. You can also install it globally with the dash G flag, but I don't really recommend it. Global dependencies are usually not the best thing. So I recommend adding it to your package JSON and then calling it from from one of your NPM scripts.

There are also a handful of other commands here that I didn't touch on. CDK LS will list the stacks that are in the application. Dif will compare the stack with the deployed version or local CloudFormation template. Metadata can display metadata that has been attached to a stack. Similarly, you can pass in key value pairs called a runtime context to your application using CDK context. There's a doctor command, which can look at your project and kind of link it for potential problems. And then CDK doc can just use to open up the official documentation in your browser. It's kind of a nice shortcut.

And then finally, I wanted to include some additional resources here. So the first link is the official documentation. The second link is the CDK source code. And then the remaining links are a combination of AWS and community created resources for learning the CDK and providing reference examples. So it can really get you started in up and running pretty quickly. And that's all I had. Thank you again, for for coming to my talk and have a nice day everyone.

I'm glad to see that a lot of people haven't used the CDK before. Hopefully, you know, the talk will be some inspiration to give it a try. Yeah. Yeah. Were you surprised by those answers? Not really. So it's kind of a newer tool. And also it's used for deployments and a lot of like software shops, developers don't really handle the deployment. So I wasn't really surprised. Gotcha, gotcha. Yeah, that makes sense. Let's take a look in the Node Talk QA channel.

Some questions for you. David, Liam asks, is any AWS service not covered in the CDK? Yeah, so the the CDK does not have coverage for all the different things that AWS can do yet. I know that they're actively adding new things. But also AWS is always adding new things as well. So it's it's very hard to catch up with you know, something that's already so massive. Yeah, that makes sense.

Max asked, Isn't Terraform more efficient because it does not use CloudFormation template? So I don't actually know how Terraform works under the hood. But Terraform, I mean, Terraform is a great tool. It's it's been around for a lot longer and is battle tested and works across other clouds as well. But you know, like I said in the talk, one of the advantages of the CDK is first, it's first class support from AWS. But second off, you know, it lets JavaScript developers kind of write their deployments using JavaScript instead of having to learn Terraform and ES5 versus ESL that Terraform uses. So I think, you know, it's not a replacement for everything, but it's just another tool for your toolbox. Right, right.

I, I am a JavaScript developer. And I love that so many things are moving to JavaScript. JavaScript is everywhere. We have a comment from Juan Gonzales. He said the talk really piqued my interest about the CDK. One more to add to my learning list. So just what you were talking about not being very surprised that folks haven't, haven't tried it out yet. When was the first time that you tried it out? How was how? How did you get into, into into this? So I'm actually fairly recently I heard I heard about it and read about it via Twitter. And so I started looking into it and thought it looked interesting. And I kind of wanted a reason to learn it better and experiment with it. And then I actually got asked to speak at this conference. So I did the whole you know, conference talk driven development, or whatever they call it. I basically learned it for this talk. And I had a really good experience using it. So I hope others do too. Yeah, that's awesome. Well, thanks for learning it for the talk.

I have done this as well conference driven development. It's always really interesting to see. I think it's a it's a really nice way to teach people because you're like learning as you go. And like the questions that you had are probably the questions that folks that are learning and has, um, did you find any any challenges while you were while you were getting started with this? I know I thought it was actually really, really well documented and smooth. I didn't try to use any of the features that have AWS that the CVK doesn't support yet. So I didn't kind of run into any of those rough edges. But you know, I found out about that JSII technology that I mentioned in there and kind of went down a rabbit hole with dad and was kind of surprised to see how that worked under the hood. So yeah, it was it was a good time. You're nine and you looked into the hood as well. That's very interesting.

Let's do another question. We've got, um, someone asked, I'm currently using terraform. Why would I migrate to the CVK? So I, you know, if you have an existing workflow and existing setup, I wouldn't recommend just tearing it out and rewriting it just for the sake of rewriting it. But if you're, if you're starting something new on AWS, if you don't know terraform yet, I think the learning curve would be, would be a lot less using something like the CVK. So that's kind of where I would recommend it. I know there's a lot of diehard terraform fans out there who aren't going to make the switch and that's fine because you don't really need to. Yeah, switching things is especially infrastructure is super complicated. Have you ever, have you ever had to do a kind of migration like that? Not from one, not from one infrastructure tool to another Now. Lucky you. I haven't had to either, but I've just like heard the horror stories and oh, I, I get shivers just thinking about it.

Um, somebody else asked, the CVK is an open source project. Can I contribute or at least provide input? Yeah, so if you if you want to contribute, it is open source. I haven't personally contributed so I can't comment on what that experience is like if it's if it's pleasant or not. But the even if you don't want to contribute code, there is a roadmap and other things that people can request features on. You there's like a GitHub project board where you can kind of see what the team is working on and what they're planning for the future. So I think one thing that that jumped out was like going support was something they were planning to add. So yeah, you can you can definitely give feedback and kind of see the development is happening in the open. So you can kind of see what's going on.

Nice. I love that. I even if it's just like seeing what's what's up ahead, it helps us make decisions about if we want to implement it or if things are coming up. And I think it's awesome. If you have that input, I know that companies building software for other developers are often keen to hear how developers will adopt that kind of stuff. Yeah, at the end of the day, I mean, they're building it probably for their own internal purposes, but also for, you know, all the AWS customers. So, why not try to see what people actually want and need? Yeah, that makes sense.

Um, Alexi in the chat is asking, which level of AWS experience is required to start using AWS CDK productively? I would say you just have to know what on AWS you're trying to build. So, you know, if you know JavaScript, and you know what you want to build on AWS, I think it's extremely easy to get started. If you've never used AWS before, and you don't even know, you know, what a lambda function is, or what a what a container is, or something like that, you're probably going to need to know what you're trying to build first. But other than that, the ramp up, at least, in my opinion, was was very simple. Nice. And hey, Colin, we've been asking, we've been asking our speakers today if they can share a little bit about since we're asking about experience, we'd love to know, how did you get into programming? What's your story? I was a computer nerd as a kid. So I I remember getting AOL and things like that. And then wanted to learn how to make web pages. And then web pages have JavaScript. And I just I really liked what I could do with JavaScript there. Learned, you know, I guess more formal programming in college, and still ended up coming back to JavaScript anyway, because it's it's just such a fun language. It is a really fun language, isn't it? Do you have any like highlights of like, what things you've built with JavaScript that you're really excited about? Things, so I guess one of my bigger contributions and JavaScript was when I was doing some work on gRPC. They didn't have a a TypeScript implementation of the server side of that. So I got to build that out. And I think it's currently being shipped with the official gRPC module.

And I got a couple of conference talks out of it. So I think that's amazing. That's really cool. And we also we were talking backstage you came and gave a talk at JSConf Columbia in 2019. And I remember one of the things that stood out was your contributions to the technical steering committee. Can you do you want to talk to a little bit about your your role on the TSC for node? Uh, yeah. So I've been contributing to node for for a while now, I think since 2014. And the TSC is is kind of the ultimate decision making body although we don't like to be we like for the other collaborators and kind of general consensus to drive things. But whenever there is an issue, then sometimes there has to be a vote or something like that. But I also want to just take a second and say that if you get a chance to go to Columbia for one of their conferences, I highly recommend it. That is so sweet. Yeah, we were talking about how we, we try and give our speakers a really, really great speaker experience. So definitely check us out if we do conferences, once this is all over. What did you like the most? What did you like the most about your experience in Columbia, Colin? The I thought the stage setup was really nice. I like that, you know, the kind of, I don't want to say pampering, but the way that the speakers were taken care of, as far as like transportation from the airport and all that it was just a very smooth experience for my first time in South America. So thank you for that. I love that. Thanks so much for joining us. Colin, everybody, please give a big round of applause to Colin for joining us for that q&a and that amazing talk.