GitHub Actions, one layer at a time: from six lines to a hardened pipeline

Start here — six lines that actually work

You want your tests to run when you push. That’s the entire ambition, and this is the entire solution:

# .github/workflows/ci.yml  —  the whole thing, to start
on: [push]                       # WHEN: run on every push
jobs:
  test:                          # WHAT: one job...
    runs-on: ubuntu-latest       #   ...on a fresh Ubuntu VM
    steps:                       #   ...running these commands in order
      - uses: actions/checkout@v4            # a prebuilt action: clone the repo
      - run: pip install -r requirements.txt
      - run: pytest

That is a real, working pipeline. Three keys carry the whole model: on: is when (the event), jobs: is what runs and where (each job gets its own machine), and steps: are the commands, in order — either a run: shell line or a uses: that pulls in a prebuilt action (here, checkout to clone your code). Push a commit and GitHub runs it, then reports pass or fail right on the commit and any PR.

Before we add anything, it’s worth seeing the full arc those six lines set in motion — from the event that fires to the result that lands back in the PR. Every layer after this just refines one part of this loop.

The cast of characters — every piece, named (and how many you get)

Before we pile on more layers, here is the whole cast and how it nests — the vocabulary trips people more than the concepts do. A workflow is one .yml file; what other tools call a pipeline, GitHub simply calls a workflow (there is no separate “pipeline” object). An action is not a workflow — it is a single reusable building block that a step pulls in with uses:. And the file you write is distinct from a workflow run, which is one execution of it.

“How many workflows can I run?”

Three different things are capped, so the answer depends on which you mean:

How the YAML keys fit together — and where with goes

A workflow file has a lot of keys, and they feel scattered until you notice the rule: every key belongs to exactly one level — the workflow, a job, or a step — and its level tells you what it’s allowed to do. Read the file as three nested scopes and the keys sort themselves.

The one that trips everyone is with:. It is the inputs to a uses: action, so it only appears on a uses: step, indented right under it. A run: step has no with: — you give it env: instead. The same pattern holds one level up: when you call a reusable workflow with uses:, you pass its inputs with with: too (and its secrets with secrets:).

“I need lint and a build too” — steps vs jobs

Real CI does more than one thing. Add a lint step and the shape of the problem appears: steps inside a job run one after another on the same machine, so a slow lint blocks your tests, and they can’t use different setups. When two things are independent, you don’t want them queued — you want them at once.

That’s the difference between a step and a job. Split independent work into separate jobs; jobs run in parallel by default, each on its own runner. When you do need order — build after tests pass — you say so with needs:.

jobs:
  lint:                          # a SECOND job — runs in PARALLEL with test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ruff check .
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: pytest
# Steps inside a job run in ORDER on one machine.
# Separate jobs run AT THE SAME TIME — unless you chain them with `needs:`.

So the hierarchy is just: a workflow wires an event to jobs (parallel machines), and each job runs steps (ordered commands) that call actions. Hold onto “each job is its own machine” — it’s about to bite.

“My deploy can’t find what build made” — jobs are isolated

You add a build job that produces ./dist, then a deploy job to ship it — and deploy fails: dist/ isn’t there. Nothing is broken. This is the single most important thing to internalize about GitHub Actions: every job runs on a brand-new VM, and its filesystem is thrown away when the job ends. Steps in one job share a disk; separate jobs share nothing.

So you move things across the gap deliberately. Files that a later job needs — a built package, a coverage report — go through artifacts (upload-artifact / download-artifact). Dependencies you want to reuse for speed go through a cache. Small string values (a version, an image tag) go through job outputs and are read via needs.build.outputs.*. Nothing else survives the crossing.

  build:
    needs: [lint, test]          # runs only after BOTH lint and test pass
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python -m build                 # produces ./dist
      - uses: actions/upload-artifact@v4      # hand ./dist to a later job
        with: { name: dist, path: dist/ }
  deploy:
    needs: build
    runs-on: ubuntu-latest       # a BRAND-NEW VM — build's ./dist is already gone
    steps:
      - uses: actions/download-artifact@v4    # the only way to get it back
        with: { name: dist }
      - run: ./deploy.sh

“I’ve pasted the same setup four times” — actions and reuse

By now checkout + setup-python + pip install appears in every job, and a sister repo wants the exact same pipeline. This is where the value compounds — and where two different tools get confused. The reusable unit is the action: the thing uses: calls. Bundle your repeated steps into a composite action and each job shrinks to two lines. To share the whole pipeline across repos, publish a reusable workflow and let callers invoke it by ref.

# .github/actions/setup/action.yml — bundle the repeated steps ONCE
name: setup
runs:
  using: composite
  steps:
    - uses: actions/setup-python@v4
      with: { python-version: "3.12", cache: pip }
    - run: pip install -r requirements.txt
      shell: bash                 # composite run: steps MUST set a shell

# then every job shrinks to two lines:
#   - uses: actions/checkout@v4
#   - uses: ./.github/actions/setup

# .github/workflows/reusable-ci.yml — share a WHOLE pipeline across repos
on:
  workflow_call:                  # this makes the workflow callable
    inputs:
      python-version: { type: string, default: "3.12" }
jobs:
  test: { runs-on: ubuntu-latest, steps: [ ... ] }

# a caller in ANY repo runs the whole thing:
# jobs:
#   ci:
#     uses: my-org/ci-templates/.github/workflows/reusable-ci.yml@<sha>
#     with: { python-version: "3.11" }

“It has to deploy — without me pasting a cloud key” — the token and OIDC

The deploy job needs credentials, and the instinct is to drop a long-lived AWS access key into repo secrets. That static key is exactly the thing that leaks. GitHub Actions gives you two better answers. First, every run already has an auto-generated GITHUB_TOKEN that expires when the job ends; its power is set by a permissions: block, per scope. Default it to read-only and grant up only where needed — the single highest-leverage hardening move.

Second, for the cloud, use OIDC: the job mints a short-lived token, and a trust policy in your cloud account swaps it for temporary credentials scoped to this repo and branch. No static keys live anywhere.

permissions:
  contents: read                 # least privilege for the WHOLE run; grant up per job
jobs:
  deploy:
    permissions:
      contents: read
      id-token: write            # let THIS job mint a short-lived OIDC token
    steps:
      - uses: aws-actions/configure-aws-credentials@v4   # swaps OIDC -> temp creds
        with:
          role-to-assume: arn:aws:iam::123456789012:role/gha-deploy
          aws-region: us-east-1
      - run: ./deploy.sh          # deploys with TEMPORARY creds — nothing stored in secrets

“Don’t auto-ship to prod” — Environments and approvals

Now every merge deploys, and you want a human in the loop for production. Change control is native, and it lives in Environments. An environment (say production) carries its own secrets plus protection rules: required reviewers (a manual approval gate), a wait timer, and branch restrictions. Point the deploy job at it and the job simply pauses until someone approves.

  deploy:
    needs: build
    environment: production       # protection rules apply here:
    runs-on: ubuntu-latest        #   required reviewers · wait timer · branch limits
    steps:
      - run: ./deploy.sh          # this job PAUSES until a reviewer approves

“Wait — my CI runs strangers’ code” — the fork trust line

Your workflow runs on pull requests, including from forks — that is, from people you’ve never met. The good news is the default is safe: a pull_request from a fork runs with a read-only token and no secrets, on purpose, so a drive-by PR can’t exfiltrate anything. The bad news is that almost every real compromise is one of three default-looking patterns.

They are: script injection — untrusted ${ github.event.* } interpolated into a run: block is shell execution; pull_request_target — it runs with a write token and secrets in the base repo’s context, so checking out and running the PR’s code hands your privileges to the author (the “pwn request”); and the unpinned action — a moveable tag can be re-pointed to new code. The fixes are cheap and worth making reflex:

# UNSAFE — a PR titled  $(rm -rf ~)  becomes a shell command on your runner:
#   - run: echo "Title is ${{ github.event.pull_request.title }}"

# SAFE — pass untrusted input through an env var; the shell treats it as data:
      - env:
          TITLE: ${{ github.event.pull_request.title }}
        run: echo "Title is $TITLE"

# And pin third-party actions to a full commit SHA — a tag can be re-pointed:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11  # v4.1.1

“It’s slow, and the bill is climbing” — matrix, concurrency, cache, cost

Three levers turn one job into fleet-scale CI without new machinery. A matrix fans a job across combinations (Python versions, OSes); set fail-fast: false to see every result instead of cancelling siblings on the first red. Concurrency cancels a run when newer commits land on the same PR (and, for deploys, does the opposite — one at a time, never cancel a live one). A cache skips repeated dependency installs.

  test:
    strategy:
      fail-fast: false                        # let ALL legs finish; don't cancel siblings
      matrix:
        python: ["3.10", "3.11", "3.12"]      # fan this job across versions
    runs-on: ubuntu-latest

# top level — cancel a run when new commits land on the same PR:
concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true                    # (for deploys, do the OPPOSITE: never cancel)

Then the bill. On public repos, standard runners are free. On private repos you pay per minute, rounded up, with an OS multiplier that dwarfs every other knob — roughly Linux ×1, Windows ×2, macOS ×10. So the biggest cost lever is which runner you pick; after that, caching and cancellation. Larger runners and artifact/cache storage are billed on top.

Step back — the whole thing, in one picture

You’ve now met every piece by hitting the wall that needs it. Here they are as one map: an event fires a workflow, which wires it to isolated jobs on runners, each running steps and actions; data crosses between jobs only through the artifact/cache/output corridor; a scoped token authorizes them; and an Environment gate can hold a deploy.

And four things stay true no matter how large your pipelines get, worth committing to memory:

• Jobs are isolated. Only artifacts, cache, and outputs cross between them.
• Fork PRs get no secrets — by design; don’t defeat it with pull_request_target.
• Actions triggered by GITHUB_TOKEN don’t recursively trigger workflows — a push made with the default token won’t kick off another run (loop prevention; surprises people expecting a follow-on).
• A tag is a moving target; a SHA is not. Pin third-party actions by commit.

The rest of the map — a reference shelf

The build above uses the pieces you reach for daily. The rest, kept compact so nothing’s lost:

Events beyond push / pull_request

Where data and config live

Runners

GitHub-hosted = a clean ephemeral VM per run (Ubuntu / Windows / macOS, plus larger runners with more vCPU/RAM/GPU). Self-hosted = your machines, chosen by label, autoscalable (ARC on Kubernetes) — but never attach a self-hosted runner to a public repo: a forked PR can run arbitrary code on it. Keep public-repo CI on ephemeral hosted runners.