Video Summary and Transcription
Jamal Sinclair-O'Garro introduces himself as a senior software engineer at Netflix with experience in algorithmic and electronic trading. Spinnaker is a multi-platform continuous delivery platform used by companies like Grubhub, Airbnb, Google, Netflix, Chime, Box, and Target. It provides infrastructure management and deployment through pipelines, allowing gradual rollouts and canary analysis to ensure safe changes. Spinnaker improves velocity and is recommended for streamlining JavaScript applications across different providers.
1. Introduction
Hello, my name is Jamal Sinclair-O'Garro, a senior software engineer at Netflix. I have experience in algorithmic and electronic trading. I'm based in New York and a fan of hip hop and sports, especially the Knicks.
Hello. My name is Jamal Sinclair-O'Garro and welcome to my talk on Spinnaker as a continuous delivery solution for JavaScript applications. So, who am I? As mentioned, my name is Jamal. I'm a senior software engineer at Netflix within our platform engineering organization. Prior to working at Netflix, I've been here about two years, I worked my entire career in finance where I focused mostly on algorithmic trading and electronic trading. I've built over 15 desktop algorithmic electronic trading applications that have gone firm-wide at my previous firms. I'm born and raised in New York where I'm currently based. I'm a hip hop head, sneaker head, I'm a Yankees, Giants, and Knicks fan, and the Knicks are killing it right now, so let's go Knicks.
2. Spinnaker Overview
Today we'll learn about Spinnaker, a multi-platform continuous delivery platform that supports multiple cloud providers. We'll discuss its functionality, deployment orchestration, resource management, and its usage by companies like Grubhub, Airbnb, Google, Netflix, Chime, Box, and Target. We'll also cover high-level concepts such as providers, applications, clusters, server groups, load balancers, and firewalls in the Spinnaker architecture.
So, today's roadmap. What are we going to focus on today? Today we're going to learn about Spinnaker, what it is, we'll understand at a high level some of the overview of its functionality, and then we'll try to figure out why you might want to use it for your JavaScript applications.
So, with that being said, what is Spinnaker? Spinnaker is a multi-platform continuous delivery platform that supports multiple cloud providers or environments. So, think AWS, Google Cloud, and Azure. Spinnaker gives us the ability to configure cloud resources. So, think of your instance type security groups if you're from AWS land. We can orchestrate our deployment or define the orchestration for deployments through something called pipelines that we'll talk about shortly. And we can also manage or deploy resources. So, think of like auto-scaling and logic that allows us to perform things like rollbacks in case things don't go well. Spinnaker was developed at Netflix internally and has since been open source and is now community maintained.
So, we know about Spinnaker, but the question you might have is, who's using it? Who's using Spinnaker in production? So, right now there's a few companies. There's Grubhub, there's Airbnb, Google, Netflix, of course, because Spinnaker came from Netflix. There's Chime, Box, and Target. So, as we can see, this is a small subset, but there's a very large amount of companies that are using Spinnaker in production for their delivery solution.
So, now let's talk about some high-level Spinnaker concepts. So, first, let's talk about a provider. So, a provider is a cloud service provider. So, like AWS or Azure, Google Cloud that provides computing resources, you can build your infrastructure on top of and deploy and serve your applications. Now, in Spinnaker world, an application is a collection of something called clusters that we'll talk about shortly. So, think of it as a way to keep track of our application, the application configuration, and also its infrastructure settings. On top of that, there's also a server group, a server group is a collection of instances. So, think of your instances of your Node application, or it could even be like Kubernetes instances as well. A cluster is a grouping of server groups in Spinnaker. And then we also have load balancers that basically balance traffic between instances inside of a particular server group. We have a firewall, which allows us to set our networking security traffic rules. So, let's look at a high level of the Spinnaker architecture. So, here we have an example of two applications. As mentioned, an application is a way for you to group your apps resources and infrastructure. And you can see here, we have a collection of clusters. And each cluster is basically a set of security groups, right? Sorry, server groups.
3. Spinnaker Architecture
The Spinnaker architecture gives an overview of clusters, server groups, instances, load balancers, and firewalls. The actual screen shows different server groups and clusters, where actions like rollback, resizing, disabling, deploying, and cloning can be controlled. Spinnaker provides a centralized panel to define infrastructure settings that are abstracted across servers and providers.
As we see here, cluster A has three server groups, cluster B has two server groups, and cluster C has one server group. If we look at application two, we see it's slightly different. And if we zoom into cluster D within application two, we'll see that we have two server groups. These server groups contain instances. So, these boxes here are instances. And here we see we have our load balancer, that's going to take traffic, push it through a firewall, and then basically decide where that traffic is hit in terms of our instances are concerned. So, that's a high level overview of the Spinnaker architecture in a nutshell.
Here's an example of what clusters look like in the actual screen. So, you see, we have our different server groups, our different clusters. We have our instances. Here, we have a way to actually control some of the actions on our server groups. So, rolling back, resizing, disabling, deploying, even cloning.
Spinnaker allows us to define our infrastructure. So, to specify settings on our environment without having to navigate through many different pages to do so. In Spinnaker, we'll see in a second in our next screenshot, you can do this from all one centralized panel. And it's also abstracted across the different servers and providers. So, you don't have to worry about if you're moving from AWS to Google Cloud to have to like learn that infrastructure or learn where everything is at. Spinnaker will give you one unified way to manage those settings.
4. Infrastructure Management and Deployment
Spinnaker provides a unified way to manage infrastructure settings across different providers. Deployment management is achieved through pipelines that automate key flows, such as image pulling, testing, canary releases, and deployment strategies. Canary analysis allows gradual rollouts and comparison against a baseline to identify statistically significant differences.
So, you don't have to worry about if you're moving from AWS to Google Cloud to have to like learn that infrastructure or learn where everything is at. Spinnaker will give you one unified way to manage those settings. So, here we have it. Here's an example infrastructure management page. So, a configuration deployment cluster where you can see here, you can specify your accounts, your region, your VPC subnets. Even here we can see that we can configure load balancers, security groups or instance types, and even down to availability zones. So, as we can see here, one benefit is that Spinnaker gives you this high level abstraction that lets you manage your infrastructure from a simple panel that can be extracted away and generalized across many different providers or cloud service providers.
Next, let's talk briefly about deployment management. So, what does this mean? So, Spinnaker has a concept of something called pipeline. And a pipeline at a high level lets you define how your code is deployed. It allows automation of key flows. So, think of like pulling an image down from somewhere, running tests, running canaries, or even allowing a manual judgment to gate a application from being promoted to production automatically. You can also manage building your images, canary releases, and enforcing other deployment strategies.
Let's briefly, before we close, let's talk about canary analysis. One really powerful feature of Spinnaker is that it allows us to run something called canaries. What is a canary? A canary is basically a deployment process where a change is gradually rolled out and compared against a baseline. So, let's say we made a change to a configuration or change to some code. What we want to do is run a baseline of what our current production looks like. And then we're going to run a replica of that that contains our code, right? Our code change, configuration change, and then what we're going to do is pick up some metrics. And for the metrics, we're going to try to see if there's any material differences that are statistically significant.
5. Pipeline and Canary Analysis
Spinnaker allows running canaries to gradually roll out changes and compare them against a baseline. A canary score is computed based on metrics to determine if the change is safe for production. Spinnaker is a powerful tool that improves velocity and makes working with cloud providers easier. The community is active and passionate. If you want to streamline your JavaScript application across different providers, Spinnaker is the tool for you.
And if you look at the actual pipeline as in production, we can see that for every stage that we go through, if something is green, that means that that stage is actually completed and it has been successful. If we had any issues, it would actually turn red, or there's other different colors that would actually specify different statuses, but those are the two ones that we mostly care about. And if you look here and drill into the actual pipeline stage itself, you can see more information and more metadata about it.
So, let's briefly, before we close, let's talk about canary analysis. One really powerful feature of Spinnaker is that it allows us to run something called canaries. What is a canary? A canary is basically a deployment process where a change is gradually rolled out and compared against a baseline. So, let's say we made a change to a configuration or change to some code. What we want to do is run a baseline of what our current production looks like. And then we're going to run a replica of that that contains our code, right? Our code change, configuration change, and then what we're going to do is pick up some metrics. And for the metrics, we're going to try to see if there's any material differences that are statistically significant. So, think of increases in latency, increases in error logs, increases of anything that looks suspicious that is different from our baseline. And what we do is we take that information and compute a score, what we call a canary score. And that score determines if that change is safe to put out to production. And this is done for both code and configuration changes, as mentioned.
So, as we wrap up this brief talk, I'm just going to give a few conclusions and takeaways, is that Spinnaker is a very powerful tool and is fully featured. It helps improve velocity and makes working with cloud providers easier once you become familiar with it. And the community is very passionate and very active at conferences and social channels. So, if you are a JavaScript developer, engineer, team lead, and you're looking for a way to streamline your JavaScript application, your JavaScript application, whether it's a node application or is it from an application, and you want to make it generic across all of your different potential providers, AWS or Google Cloud, Spinnaker is probably the tool for you. Here are a few resources. So, the Spinnaker docs is a really good place to start. The Spinnaker Slack channel is pretty good. That URL is pretty long, but this deck will be shared. It's a clickable link. Feel free to check it out. And here's my contact information, my GitHub, my Twitter, and my LinkedIn. And with that, I will open it up to any questions.
Comments