Software development process sometimes is seen as the process that is modernizing the world, but in essence, software devs look at Henry Ford’s assembly line to figure out the CI/CD pipelines, agile and Kansan all come from Toyota Corp mgmt studies…Do not undermine the efforts made by the software world to improve the efficiency, but be more humble when we look at the world and think that we are the ones who are modernizing the world.
Reliability is the most important feature of a software.
Simple testing can prevent most critical failures.
Continuous Integration is a process of ensuring that every change that gets integrated into a repository goes through various checks, that we are confident about its reliability.
Desired features of a CI tool:
- Ability to create jobs to run build, tests, packaging, etc. by integrating various tools
- Connect those jobs to create a sequence of execution pipelines
- Run these pipelines in an automated way
- Ability to trigger these pipelines based on external events, e.g. pushing code to git repo
There is a difference between the Ds, CDelivery is when there is a “manager” stopping the automated deployment to a production environment to give the thumbs up to push to production. CDeployment is when there is no human intervention between the CI and the deployment to the production environment, meaning no human in between jobs and production environment deployments.
- Agile: Continuous Integration and Continuous Delivery requires an agile approach with smaller batches of features to create features faster.
- Build: should be automated and should be quick to compile app
- Tests: should be automated and should cover most unit testing
- There is an approach called the “test pyramid” which states that the farther along you are in the following test stages, the longer it takes to test:
- Unit tests
- Component tests
- Service tests
- API tests
- UI tests
- Exploratory tests
- There is an approach called the “test pyramid” which states that the farther along you are in the following test stages, the longer it takes to test:
- Deployment: should be automated
- Revision control: keep all configuration, build files, deployment logic, etc. in a repository to allow revision control while changes are being integrated
- Development workflows: using tools like Git and Github or Gitlab
- Code review: always have an extra pair of eyes before integrating changes
- Setup revision control system
- Define development workflow and code reviews
- Setup CI platform
- Create CI pipelines - Build, Test, etc.
- Build and manage artifacts - packaging jobs
- Setup deployment tools
- Create deployment pipelines
Leverage docker to improve your CI/CD.
When running Docker command docker container run, it first attempts to pull the image locally, if not found, these are the events:
docker image pull: finds the image in Docker registrydocker container create: downloads the image into Docker host, and image is launcheddocker container start: starts container
REMOVE: next video is about how to make containers persist
A default Docker image is pulled from hub.docker.com/docker/centos:latest, which each are:
- hub.docker.com registry (default)
- docker namespace (default; user, org or project)
- centos image
- :latest tag (default)
Unit Testing is the testing of individual functions and their output based on an input value. Devs tend to write simple, single purpose functions, bc the more complex a function, the more difficult it is to test. Unit tests are run on each function of our code without any dependencies.
Integration Testing is testing that all aspects of an app work together. Successful unit testing only ensures that functions are running correctly outside the context of the app.
Integration testing is a compelling use case for containerization of apps even if we are not ultimately deploying our app in containers. It is so much easier to automate the process of integration testing with a containerized app, esp. across a distributed team.
End-to-end Testing (E2E) is automated UI testing. Devs need to define what behavior the test user should cover and expect. E2E tests our app from the perspective of an end-user.
Tools to use for E2E:
- PhantomJS is a headless, scriptable browser, used to mimic a user visiting our app on the web and clicking through
It is important to be aware of software vulnerabilities before we deploy our software. Scanning our code for vulnerabilities before deployment is absolutely necessary in a robust CI/CD workflow.
Code Smells are known programming patterns that tend to, though not always, lead to vulnerabilities and inefficiencies in our code.
Tools to analyze code against the most up-to-date list of vulnerabilities and code smells:
- SonarQube by SonarSource
- SonarCloud by SonarSource
Scalability and reliability should be part of our CI/CD process to ensure that our apps are prepared to meet future demand.
Infrastructure As Code (IaC) allows us to treat our CI/CD operations as software.
Kubernetes (K8s) is a container orchestration engine that is excellent at scalability. K8s can orchestrate the automatic running and stopping of containers as need ebbs and flows. K8s also balances the load of incoming requests between containers. Find more on K8s docs.
The smallest unit of a K8s environment is called a pod. A pod holds a single container or a collection of containers that share the same virtual network and all resources (IP address, storage, etc.).
Containers inside a pod can communicate w/one another as though as they are on the same host machine. This is very convenient bc it means our containers in a shared pod can always communicate w/one another on localhost.
Pods run on nodes. The best way to think of a node is as a machine. It could be a physical machine, a virtual machine on a laptop or on a cloud instance.
Nodes are conceptually held inside a cluster. Nodes on a K8s cluster do not have to be in the same location (cloud, on-prem, etc.). This is one of the main advantages of K8s. Your app has access to the entire computing power of a distributed network of machines.
The limits of K8s can be found in the amount of computing resources we give it to access to via nodes. Once the sum of the computing resources of all the nodes are used up, then K8s will have reached its limits.
We can gain insight into what is going on inside our cluster with a tool called kubectl. This is for troubleshooting purposes only. A functional K8s deployment is generally interacted w/via its cluster IP address.
There are 4 types of services:
- ClusterIP will be assigned an IP address that can be accessed using the service's name
- Think of a ClusterIP service as an app running on our cluster with an exposed port, a host name, and an IP type
- Names in our apps need to match the names in K8s, so that K8s can handle the intra-cluster communication for us
- NodePort allows for access to the service from outside the cluster
- Since each node is a "machine", it will have an IP address
- NodePort services can be accessed via the external IP address of one of the cluster nodes with the port that is assigned by K8s
- LoadBalancer
- ExternalName
K8s is configured with YAML config files called manifests.
GitOps is an implementation of IaC. GitOps uses Git as the source of truth for our infrastructure configuration. This allows us to quickly change, upgrade, and roll back changes in code.
There are multiple solutions when it comes time for the CD part of CI/CD. These include:
- JenkinsX
- GoCD
- Spinnaker
- ArgoCD