AUTOMATE
EVERTHING

What is CI/CD?

CI/CD stands for Continuous Integration and Continuous Delivery (or Deployment). It's a methodology that automates the process of integrating code changes, testing them, and delivering them to production.

The Three Pillars

• Continuous Integration (CI): Developers merge code changes frequently, usually multiple times per day. Each merge triggers automated builds and tests.
• Continuous Delivery (CD): Code changes are automatically prepared for release to production. A human manually approves the final deployment.
• Continuous Deployment: Every change that passes tests is automatically deployed to production. No human intervention needed.

Why Does It Matter?

# Without CI/CD:
Write code → Manually test → Hope it works → Deploy manually → 
Fix production bugs → Repeat

# With CI/CD:
Write code → Auto-test → Auto-deploy → Monitor → Fix if needed → Repeat

The CI/CD Pipeline

┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐
│  CODE   │──▶│  BUILD  │──▶│  TEST   │──▶│ STAGING │──▶│PRODUCtn │
│ COMMIT  │   │ COMPILE │   │   UNIT  │   │  DEPLOY │   │ DEPLOY  │
└─────────┘   └─────────┘   └─────────┘   └─────────┘   └─────────┘

Key Benefits

• Faster releases: From weeks to hours or minutes
• Fewer bugs: Automated testing catches issues early
• Consistency: Same process every time, no human error
• Feedback loops: Know immediately when something breaks
• Confidence: Deploy anytime, even on Fridays

The Foundation of CI/CD

Every CI/CD pipeline starts with version control. Your Git workflow directly impacts how effective your automation can be. Let's build on solid foundations.

Branching Strategies

Git Flow

main ──────┬─────────────────────────────
           ▲                         ▲
           │ merge                   │ merge
develop ──┴──────┬──────┬──────┬────┴────
                 feature feature feature

• main: Production-ready code
• develop: Integration branch
• feature/*: New features

GitHub Flow

main ────▶───▶───▶───▶───▶───▶───▶
         ▲   ▲   ▲   ▲   ▲   ▲   ▲
         │   │   │   │   │   │   │
         PR  PR  PR  PR  PR  PR  PR

• Everything from main can be deployed
• Create feature branches from main
• Open Pull Requests for discussion
• Merge after review and CI passes

Commit Messages That Matter

# Bad commits:
- fixed it
- updates

# Good commits (Conventional):
- feat: add user authentication
- fix: resolve memory leak in cache
- docs: update API documentation
- test: add unit tests for auth module

Pull Request Best Practices

• Small PRs: Under 400 lines. Easier to review.
• Clear description: What, why, how
• Link issues: Connect to tracking
• CI must pass: Don't merge broken code

What is Jenkins?

Jenkins is the most popular open-source automation server. It automates the building, testing, and deployment of software. With over 1,800 plugins, it integrates with virtually every tool in your stack.

Installing Jenkins

# On Ubuntu/Debian:
sudo apt update
sudo apt install openjdk-11-jdk -y
wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt update
sudo apt install jenkins -y
sudo systemctl start jenkins

# Access at: http://your-server:8080

Your First Jenkins Pipeline

pipeline {
    agent any
    
    stages {
        stage('Checkout') {
            steps {
                git 'https://github.com/your-org/your-app.git'
            }
        }
        
        stage('Build') {
            steps {
                sh 'npm install'
                sh 'npm run build'
            }
        }
        
        stage('Test') {
            steps {
                sh 'npm test'
            }
        }
        
        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                sh './deploy.sh production'
            }
        }
    }
}

Pipeline Structure

• pipeline: Top-level block defining the entire pipeline
• agent: Where to execute (any, specific node, Docker)
• stages: Collection of stages
• stage: Logical unit of work (Build, Test, Deploy)
• steps: Actual commands to execute

Native CI/CD for GitHub

GitHub Actions lets you automate, customize, and execute your software development workflows right in your repository. No external servers needed.

Basic Workflow Structure

# .github/workflows/ci.yml
name: CI Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run tests
        run: npm test
      
      - name: Build
        run: npm run build

Matrix Builds

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [16, 18, 20]
    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - name: Install dependencies
        run: npm ci
      - name: Run tests
        run: npm test

Deployment with Secrets

name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Deploy to server
        uses: appleboy/ssh-action@v1.0.0
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          key: ${{ secrets.SSH_KEY }}
          script: |
            cd /var/www/app
            git pull origin main
            docker-compose up -d --build

Choosing Your Strategy

How you deploy affects your availability, risk, and rollback capability. Let's explore the main strategies and when to use each.

Blue-Green Deployment

# Traffic switches instantly between two identical environments

[Load Balancer]──▶ [Blue: Production v1]    [Green: Idle v2]
                  
After testing green:
[Load Balancer]──────────▶ [Blue: Idle]    [Green: Production v2]

• Instant rollback possible
• Zero downtime deployments
• Requires double infrastructure

Canary Deployment

# Gradually shift traffic

10% ──────────▶ [New Version]
90% ──────────▶ [Old Version]

If metrics look good:
50% ──────────▶ [New Version]
50% ──────────▶ [Old Version]

Finally:
100% ─────────▶ [New Version]

• Test with real production traffic
• Gradual risk exposure
• Requires traffic management

Rolling Deployment

# Update instances one at a time

[v1] [v1] [v1] [v1]  →  [v2] [v1] [v1] [v1]
                       →  [v2] [v2] [v1] [v1]
                       →  [v2] [v2] [v2] [v1]
                       →  [v2] [v2] [v2] [v2]

• No additional infrastructure
• Simple to implement
• Rollback takes time

Feature Flags

# Toggle features without deploying new code
if (featureFlags.isEnabled('new-checkout')) {
    return renderNewCheckout();
} else {
    return renderOldCheckout();
}

Containerizing Your Pipeline

Docker provides consistent environments across development, testing, and production. Learn to leverage containers throughout your CI/CD pipeline.

Dockerfile Best Practices

# BAD: Large base image, multiple RUN commands
FROM ubuntu:20.04
RUN apt-get update
RUN apt-get install -y nodejs npm git
RUN git clone /repo
RUN npm install
RUN npm build

# GOOD: Minimal, layered, cached
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
CMD ["node", "index.js"]

Multi-Stage Builds

# Build stage - heavy tools
FROM golang:1.21 AS builder
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o app .

# Runtime stage - minimal
FROM alpine:3.18
RUN apk --no-cache add ca-certificates
COPY --from=builder /src/app /app
CMD ["/app"]

Building in CI

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        
      - name: Build image
        run: docker build -t myapp:${{ github.sha }} .
        
      - name: Run tests in container
        run: docker run myapp:${{ github.sha }} npm test

Image Scanning

# Trivy container scanning
- name: Run Trivy
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: 'myapp:${{ github.sha }}'
    format: 'sarif'
    output: 'trivy-results.sarif'

The Build Phase

The build phase transforms your source code into a deployable artifact. This includes compiling, bundling, and preparing your application for testing and deployment.

Multi-Stage Docker Builds

# Dockerfile for a Node.js app
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS runner
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
EXPOSE 3000
CMD ["node", "dist/index.js"]

Types of Tests

• Unit Tests: Test individual functions/modules in isolation
• Integration Tests: Test how components work together
• End-to-End Tests: Test the entire application flow
• Performance Tests: Load testing and benchmarking

Running Tests in CI

- name: Run tests
  run: npm test -- --coverage
  
- name: Upload test results
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: test-results
    path: coverage/

Test Reporting

# JUnit XML format is widely supported
npm test -- --junit-xml results.xml

# Coverage reports
npm test -- --coverage --json coverage-final.json

What Are Artifacts?

Build artifacts are the output of your build process—compiled binaries, Docker images, npm packages, or any file you need to deploy. Artifact management ensures you can reproduce any build.

Artifact Repositories

• JFrog Artifactory: Universal artifact repository
• GitHub Packages: GitHub's native package registry
• AWS S3: Simple object storage
• Docker Hub: For container images

Semantic Versioning

MAJOR.MINOR.PATCH
1.2.3

- 1.2.3 → 1.2.4 (patch: bug fixes)
- 1.2.3 → 1.3.0 (minor: new features)
- 1.2.3 → 2.0.0 (major: breaking changes)

Publishing Docker Images

- name: Login to Docker Hub
  uses: docker/login-action@v3
  with:
    username: ${{ secrets.DOCKERHUB_USERNAME }}
    password: ${{ secrets.DOCKERHUB_TOKEN }}
    
- name: Build and push
  uses: docker/build-push-action@v5
  with:
    push: true
    tags: myuser/myapp:latest

Artifact Retention

# GitHub Actions artifact retention
- name: Upload artifact
  uses: actions/upload-artifact@v4
  with:
    name: build
    path: dist/
    retention-days: 30

Code Your Infrastructure

IaC treats your infrastructure the same way you treat your application code: version control, code review, and automation. No more snowflake servers.

Terraform Basics

# main.tf
terraform {
  required_version = ">= 1.0"
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "web" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t3.micro"
  
  tags = {
    Name = "web-server"
  }
}

Terraform in CI/CD

- name: Setup Terraform
  uses: hashicorp/setup-terraform@v2
  
- name: Terraform Init
  run: terraform init
  
- name: Terraform Plan
  run: terraform plan -out=tfplan
  
- name: Terraform Apply
  if: github.ref == 'refs/heads/main'
  run: terraform apply tfplan

Ansible for Configuration

# playbook.yml
---
- name: Configure web servers
  hosts: webservers
  become: yes
  
  tasks:
    - name: Ensure nginx is installed
      apt:
        name: nginx
        state: present
        
    - name: Start nginx
      service:
        name: nginx
        state: started

GitOps Workflow

# Infrastructure repository structure
infrastructure/
├── terraform/
│   ├── prod/
│   └── staging/
└── ansible/
    └── playbooks/

Shift Left on Security

Security should be integrated throughout your pipeline, not added at the end. Let's build secure pipelines that catch vulnerabilities early.

SAST - Static Application Security Testing

# SonarCloud in GitHub Actions
- name: SonarCloud Scan
  uses: SonarSource/sonarcloud-github-action@master
  env:
    SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Secrets Scanning

# Gitleaks - Scan for secrets in code
- name: Run Gitleaks
  uses: gitleaks/gitleaks-action@v2
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Dependency Scanning

# npm audit in CI
- name: Security Audit
  run: |
    npm audit --audit-level=high

Container Scanning

# Trivy container scanning
- name: Run Trivy
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: 'myapp:${{ github.sha }}'
    format: 'sarif'
    output: 'trivy-results.sarif'

Secret Management

# Using HashiCorp Vault in CI
- name: Fetch secrets from Vault
  uses: hashicorp/vault-action@v2
  with:
    url: ${{ secrets.VAULT_ADDR }}
    token: ${{ secrets.VAULT_TOKEN }}

Observability in CI/CD

A pipeline isn't complete without observability. Track metrics, set up alerts, and create feedback loops that help your team improve.

Key Pipeline Metrics

• Build Success Rate: % of builds that pass
• Build Time: Average time from commit to artifact
• Deployment Frequency: How often you deploy
• Lead Time: Time from commit to production
• MTTR: Mean Time To Recovery

Alerting on Pipeline Failures

# Slack notification on failure
- name: Notify on failure
  if: failure()
  uses: 8398a7/action-slack@v3
  with:
    status: failure
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

Deployment Tracking

# Annotations in Datadog/Grafana
- name: Annotate deployment
  run: |
    curl -X POST "https://api.datadoghq.com/api/v1/monitor" \
      -H "DD-API-KEY: ${{ secrets.DD_API_KEY }}" \
      -d '{"title": "Deployed to production"}'

Continuous Feedback Loop

1. Automated tests run in staging
2. Canary deployment to 10% traffic
3. Monitor error rates and latency
4. If metrics look good → expand to 100%
5. If issues → automatic rollback
6. Alert team of all events

Enterprise-Grade CI/CD

Now let's explore advanced patterns used by high-performing engineering organizations. Take your pipeline to the next level.

Selective Pipeline Triggers

# Only run affected parts of pipeline
- name: Detect changes
  id: changes
  uses: dorny/paths-filter@v2
  with:
    filters: |
      api:
        - 'api/**'
      ui:
        - 'ui/**'

jobs:
  test-api:
    if: steps.changes.outputs.api == 'true'
    steps:
      - run: npm test --workspace=api

GitOps with ArgoCD

# ArgoCD continuously monitors your Git repo
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: production-app
spec:
  source:
    repoURL: https://github.com/org/manifests
    targetRevision: main
    path: production
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Self-Service Platforms

# Internal Developer Platform (IDP)
# Developers can provision resources via PR

# Backstage catalog example
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: my-service
spec:
  type: service
  lifecycle: production
  owner: platform-team

The CI/CD Maturity Model

Level 1: Manual - No automation
Level 2: Basic - Automated builds
Level 3: Intermediate - Automated tests
Level 4: Advanced - Automated deployments
Level 5: Elite - Self-healing, data-driven