How to Get CI/CD Right in 2021: A Guide to CI and CD

Rate this content
Bookmark

Software delivery is a top priority for organizations that own software, yet it remains one of the most challenging problems enterprises face today. Continuous integration (CI) and continuous delivery (CD) are software practices that allow organizations and teams to deliver code to customers quickly, safely, and repeatedly. Whether it's to improve development, operations, or security, CI/CD pipelines give engineers and teams more time to work on things that matter and less time struggling with the risk, standards, and velocity of deployments. Join this session to learn about the components of CI/CD and how to build and scale pipelines for the future.

This talk has been presented at DevOps.js Conf 2021, check out the latest edition of this Tech Conference.

FAQ

CI/CD stands for Continuous Integration and Continuous Delivery. It is a method used to frequently deliver apps to customers by introducing automation into the stages of app development. The main concepts attributed to CI/CD are continuous integration, continuous delivery, and continuous deployment. CI/CD is important because it allows for the rapid, reliable, and repeatable deployment of software, enhancing the software delivery process and enabling a more agile project management approach.

The main components of a CI/CD pipeline include source code management, continuous integration, automated testing, and continuous delivery. This setup helps automate the steps in software delivery, from the initial coding through to deployment, ensuring that the software can be reliably released at any time.

In continuous integration, developers frequently merge their code changes into a central repository, after which automated builds and tests are run. The key goals are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.

Continuous delivery extends continuous integration by automatically deploying all code changes to a testing and/or production environment after the build stage. This ensures that you can release new changes to your customers quickly and sustainably in a systematic manner. It focuses on minimizing the end-to-end cycle time of delivering iterations.

To ensure safety and security in CI/CD processes, organizations can implement automated security testing, use secure coding practices, manage dependencies carefully, and utilize secrets management tools to handle sensitive data. Regularly reviewing and updating security practices to address new vulnerabilities is also crucial.

Organizations can improve the speed of their CI/CD processes by implementing value stream management to identify and eliminate bottlenecks, using automated tools for faster builds and deployments, and continuously measuring performance metrics to identify areas for improvement.

Key focus areas to succeed with CI/CD include ensuring the pipeline speed, safety, and repeatability of deployments, along with a strong emphasis on user feedback. Prioritizing these areas helps organizations streamline their development processes, reduce risks, and deliver high-quality software efficiently.

Tiffany Jachja
Tiffany Jachja
9 min
01 Jul, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This talk provides an introduction to CI/CD, discussing its key components and how to succeed with it. It emphasizes the importance of speed, safety, and scaling in CI/CD, highlighting the need for unit tests, value stream management, metrics, and addressing deployment challenges. The talk also emphasizes the continuous nature of DevOps and the importance of gathering feedback and releasing changes to a subset of users.

1. Introduction to CI/CD

Short description:

Hi, everyone. Thanks so much for joining this session on CI/CD. I'm Tiffany Jackja, a technical evangelist at Harness. We'll discuss the key components of CI/CD and how to succeed with it. CI/CD is the heart of continuous integration and continuous delivery, enabling quick and safe deployment of changes. It involves a pipeline that automates processes in the software development lifecycle. The pipeline includes continuous integration, focusing on the development workflow, and continuous delivery, provisioning the infrastructure for production. The goal is to get changes into the hands of users in a safe, quick, and repeatable way. By applying governance, secrets management, and security benefits, we can build a strong foundation for scaling services and working with different kinds of services.

Hi, everyone. Thanks so much for joining this session, a how to guide on CI CD. I'm Tiffany Jackja. I'm a technical evangelist at Harness. We help simplify and scale software delivery. I'm actually really excited to be sharing this presentation because I know for a lot of people looking into DevOps and you know making that journey better. CI CD comes up very often and so we're actually going to share some of the key components of CI CD and then we'll actually get into some of the ways that you can better succeed and position yourself for 2021.

So let's just get started. I know we don't have a lot of time in a lightning session but I really just want to share a very basic CI CD pipeline. And so when you think about CI CD pipelines and just the idea of a pipeline, you, it's really just this conduit for being able to automate some of these processes that we have in a software development lifecycle. So if you have an idea or a change to be made, we actually want to deliver that or put that into a pipeline that gets you to production.

And so that process can involve making some code changes, right? Building that code change, testing it and then deploying it into this temporary environment, it's non-production environment, where you can test it and play with it, right? And verify and make sure that it's ready before actually releasing it to a customer and having it be available to the outside world, right? And so this idea of getting from idea to production very quickly is the heart and sort of the soul of continuous integration and continuous delivery.

And so there are two main points to a CI CD pipeline, and you really do want to think about it in this way. So this first part is about continuous integration. And when we talk about continuous integration, we're actually trying to focus on that development workflow, getting from code to packaged artifact. And really what happens in a continuous integration process is you have some source code management tool, right, or repository of code. And whenever someone pushes a change or makes some type of change, you can actually trigger a CI pipeline where you get to build that application, ensure that it compiles, ensure that it has all the dependencies that you need, that you can run some very basic unit tests, some very cheap and fast tests to ensure that these changes can go through, right. And then once that happens, once it's packaged, right, you can actually version it and have it be available to run on a server. And that's when we actually talk about continuous delivery. That's when we talk about provisioning a server, right. Provisioning some infrastructure, ensuring that infrastructure has everything that it needs to run an application, right, whether it's like a database, or maybe it needs some setup, some administrator to manage it, right. And so in this continuous delivery aspect, we're trying to think about ways of releasing it into a production environment, delivering it there, not just having it run there, right. And so thinking about really strategies, thinking about verification and just in case things go wrong, strategy for rolling back issues. And so to even get to continuous delivery or this idea of whatever, continuous flow, even DevOps, really, we need to think about this ability to get all kinds of changes into the hands of users and not only just being able to do it once or twice, but actually being able to do it in a safe, quick and repeatable way. And this definition comes from Jez Humble, who wrote a book on continuous delivery. And so that's why today I think we're starting to see a larger ecosystem around CI and CD, right, this idea that we can apply governance to our CI-CD pipelines, we can apply secrets management, security benefits, right, and actually build out a whole ecosystem around this, and have really good foundation for whatever we may want to do. Because the reality of it is that we have a lot to do, right, whenever we want to scale a new service or even just ensure that we can work with different kinds of services, sometimes this process can take weeks or even months, and the reality of it is we shouldn't be taking that long and eventually we'll be able to solve for a lot of these things with CI-CD.

And so, this gets to the latter part of my presentation about how to succeed with CI-CD and what are some key areas that you want to focus on. And so, I really kind of think about it in these three areas about speed, safety, repeatability, and users. And there's a lot of different ways that you can get better at this, right? It's not a one and done kind of deal, and you may, you know, invest in certain areas over others first.

2. Speed, Safety, and Scaling in CI/CD

Short description:

Maybe speed doesn't matter because you don't need to deploy every day. Consider having enough unit tests and building capabilities before investing in a perfect CI/CD pipeline. Value stream management helps eliminate idle time. Metrics measure speed and tempo. Safety ensures changes are problem-free and secure. Sustaining and scaling delivery requires visibility, standardized templates, and addressing deployment challenges. Gathering feedback and releasing changes to a subset of users are crucial. Remember, DevOps is a continuous process. Thank you for joining!

Maybe speed doesn't matter because you don't need to deploy every day. Then maybe you want to consider, do I have enough unit tests? Am I building enough capabilities in a particular area before I go and try to invest in all of these areas or try to have this perfect CICD pipeline when no one needs it yet, for example.

And so, some of the ways that you can start thinking about speed is through value stream management. So, this is eliminating any areas of just, you know, waiting idle time, basically. You know, if someone commits some code and it actually doesn't get packaged into an artifact a year, three years, that could impact how often you actually go to deploy.

And so, the second half right here, I have a couple of metrics that you can use to kind of measure speed and tempo. And I'm going to be sharing more metrics around this area. I do want to go a little quicker and also talk about safety. So, safety is about being able to ensure that all of our changes are, don't cause any problems and that they're secure. Things like the SolarWinds hack, that was a really big issue. Things like being able to understand your micro-services, your architectures, those are all really important as well.

And then there's this area of being able to sustain and scale out continuous delivery and software delivery. And so, it's thinking about, well, do I have enough visibility into all of my processes? Do I have pipeline templates, standardized templates that have been proven out that I can just repeat around? And, you know, how do people feel about this? Are they burnt out every single deployment? Does it take a village to deploy? All these things can be a really big challenge.

And then this final area that I want to get to, for this talk, is around your users, right? Being able to gather feedback, being able to also release changes to a subset of users before you go on. These are all really important aspects. And so, these are some areas that you can think about. I know that we're running out of time. I do want to say that if you are going to be thinking about DevOps and about CICD, just remember that, you know, this is a continuous process. And so, as we build out new processes, we can get better and improve on our processes and really, really do enable that DevOps lifecycle. So, I just wanted to say thank you so much for joining this session. If you have any questions, feel free to ask me. I'll be around.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Levelling up Monorepos with npm Workspaces
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Levelling up Monorepos with npm Workspaces
Top Content
NPM workspaces help manage multiple nested packages within a single top-level package, improving since the release of NPM CLI 7.0. You can easily add dependencies to workspaces and handle duplications. Running scripts and orchestration in a monorepo is made easier with NPM workspaces. The npm pkg command is useful for setting and retrieving keys and values from package.json files. NPM workspaces offer benefits compared to Lerna and future plans include better workspace linking and adding missing features.
Automating All the Code & Testing Things with GitHub Actions
React Advanced Conference 2021React Advanced Conference 2021
19 min
Automating All the Code & Testing Things with GitHub Actions
Top Content
We will learn how to automate code and testing with GitHub Actions, including linting, formatting, testing, and deployments. Automating deployments with scripts and Git hooks can help avoid mistakes. Popular CI-CD frameworks like Jenkins offer powerful orchestration but can be challenging to work with. GitHub Actions are flexible and approachable, allowing for environment setup, testing, deployment, and custom actions. A custom AppleTools Eyes GitHub action simplifies visual testing. Other examples include automating content reminders for sharing old content and tutorials.
Fine-tuning DevOps for People over Perfection
DevOps.js Conf 2022DevOps.js Conf 2022
33 min
Fine-tuning DevOps for People over Perfection
Top Content
DevOps is a journey that varies for each company, and remote work makes transformation challenging. Pull requests can be frustrating and slow, but success stories like Mateo Colia's company show the benefits of deploying every day. Challenges with tools and vulnerabilities require careful consideration and prioritization. Investing in documentation and people is important for efficient workflows and team growth. Trust is more important than excessive control when deploying to production.
Why is CI so Damn Slow?
DevOps.js Conf 2022DevOps.js Conf 2022
27 min
Why is CI so Damn Slow?
Slow CI has a negative impact on productivity and finances. Debugging CI workflows and tool slowness is even worse. Dependencies impact CI and waiting for NPM or YARN is frustrating. The ideal CI job involves native programs for static jobs and lightweight environments for dynamic jobs. Improving formatter performance and linting is a priority. Performance optimization and fast tools are essential for CI and developers using slower hardware.
End the Pain: Rethinking CI for Large Monorepos
DevOps.js Conf 2024DevOps.js Conf 2024
25 min
End the Pain: Rethinking CI for Large Monorepos
Today's Talk discusses rethinking CI in monorepos, with a focus on leveraging the implicit graph of project dependencies to optimize build times and manage complexity. The use of NX Replay and NX Agents is highlighted as a way to enhance CI efficiency by caching previous computations and distributing tasks across multiple machines. Fine-grained distribution and flakiness detection are discussed as methods to improve distribution efficiency and ensure a clean setup. Enabling distribution with NX Agents simplifies the setup process, and NX Cloud offers dynamic scaling and cost reduction. Overall, the Talk explores strategies to improve the scalability and efficiency of CI pipelines in monorepos.
The Zen of Yarn
DevOps.js Conf 2022DevOps.js Conf 2022
31 min
The Zen of Yarn
Let's talk about React and TypeScript, Yarn's philosophy and long-term relevance, stability and error handling in Yarn, Yarn's behavior and open source sustainability, investing in maintenance and future contributors, contributing to the JavaScript ecosystem, open-source contribution experience, maintaining naming consistency in large projects, version consistency and strictness in Yarn, and Yarn 4 experiments for performance improvement.

Workshops on related topic

Deploying React Native Apps in the Cloud
React Summit 2023React Summit 2023
88 min
Deploying React Native Apps in the Cloud
WorkshopFree
Cecelia Martinez
Cecelia Martinez
Deploying React Native apps manually on a local machine can be complex. The differences between Android and iOS require developers to use specific tools and processes for each platform, including hardware requirements for iOS. Manual deployments also make it difficult to manage signing credentials, environment configurations, track releases, and to collaborate as a team.
Appflow is the cloud mobile DevOps platform built by Ionic. Using a service like Appflow to build React Native apps not only provides access to powerful computing resources, it can simplify the deployment process by providing a centralized environment for managing and distributing your app to multiple platforms. This can save time and resources, enable collaboration, as well as improve the overall reliability and scalability of an app.
In this workshop, you’ll deploy a React Native application for delivery to Android and iOS test devices using Appflow. You’ll also learn the steps for publishing to Google Play and Apple App Stores. No previous experience with deploying native applications is required, and you’ll come away with a deeper understanding of the mobile deployment process and best practices for how to use a cloud mobile DevOps platform to ship quickly at scale.
MERN Stack Application Deployment in Kubernetes
DevOps.js Conf 2022DevOps.js Conf 2022
152 min
MERN Stack Application Deployment in Kubernetes
Workshop
Joel Lord
Joel Lord
Deploying and managing JavaScript applications in Kubernetes can get tricky. Especially when a database also has to be part of the deployment. MongoDB Atlas has made developers' lives much easier, however, how do you take a SaaS product and integrate it with your existing Kubernetes cluster? This is where the MongoDB Atlas Operator comes into play. In this workshop, the attendees will learn about how to create a MERN (MongoDB, Express, React, Node.js) application locally, and how to deploy everything into a Kubernetes cluster with the Atlas Operator.
Azure Static Web Apps (SWA) with Azure DevOps
DevOps.js Conf 2022DevOps.js Conf 2022
13 min
Azure Static Web Apps (SWA) with Azure DevOps
WorkshopFree
Juarez Barbosa Junior
Juarez Barbosa Junior
Azure Static Web Apps were launched earlier in 2021, and out of the box, they could integrate your existing repository and deploy your Static Web App from Azure DevOps. This workshop demonstrates how to publish an Azure Static Web App with Azure DevOps.
How to develop, build, and deploy Node.js microservices with Pulumi and Azure DevOps
DevOps.js Conf 2022DevOps.js Conf 2022
163 min
How to develop, build, and deploy Node.js microservices with Pulumi and Azure DevOps
Workshop
Alex Korzhikov
Andrew Reddikh
2 authors
The workshop gives a practical perspective of key principles needed to develop, build, and maintain a set of microservices in the Node.js stack. It covers specifics of creating isolated TypeScript services using the monorepo approach with lerna and yarn workspaces. The workshop includes an overview and a live exercise to create cloud environment with Pulumi framework and Azure services. The sessions fits the best developers who want to learn and practice build and deploy techniques using Azure stack and Pulumi for Node.js.