From 68f1fd2c9a151ae23116e6539074cdb67013a499 Mon Sep 17 00:00:00 2001 From: continuist Date: Fri, 5 Sep 2025 19:34:14 -0400 Subject: [PATCH] Explain how to set env variables and secrets before use --- CI_CD_PIPELINE_SETUP_GUIDE.md | 168 ++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/CI_CD_PIPELINE_SETUP_GUIDE.md b/CI_CD_PIPELINE_SETUP_GUIDE.md index 075798c..a50fb58 100644 --- a/CI_CD_PIPELINE_SETUP_GUIDE.md +++ b/CI_CD_PIPELINE_SETUP_GUIDE.md @@ -940,6 +940,89 @@ sudo journalctl -u forgejo-runner.service -f --no-pager - Check network: Ensure the runner can reach your Forgejo instance - Restart service: `sudo systemctl restart forgejo-runner.service` +### Environment Variables for PiP Scripts + +Before proceeding with Section 7, you need to understand the environment variables used by the PiP (Podman-in-Podman) scripts. These variables control the behavior of `secure_pip_setup.sh` and `pip_ready.sh` and are automatically set in CI environments but may need manual configuration for local testing. + +#### Required Environment Variables + +**1. `PODMAN_CLIENT_IMG_DIGEST` (REQUIRED)** +- **Purpose**: Pinned image digest for the Podman client container used in PiP +- **Format**: Must be a full digest reference (e.g., `quay.io/podman/stable@sha256:...`) +- **How to obtain**: + ```bash + # Get Podman client image digest + podman manifest inspect quay.io/podman/stable:latest | jq -r '.manifests[0].digest' + # Result: quay.io/podman/stable@sha256:... + ``` +- **Security Importance**: Prevents supply chain attacks by ensuring only verified images are used + +**2. `RUN_ID` (Optional, Auto-detected)** +- **Purpose**: Unique identifier for each CI run to prevent container name conflicts +- **Default**: `$GITHUB_RUN_ID` in CI, `local` for manual runs +- **Usage**: Used to create unique container names like `ci-pip-123` + +**3. `PIP_NAME` (Optional, Auto-generated)** +- **Purpose**: Name of the PiP container for readiness checking +- **Default**: `ci-pip-${RUN_ID}` (e.g., `ci-pip-123`) +- **Usage**: Used by `pip_ready.sh` to monitor container readiness + +**4. `SOCKET_PATH` (Optional, Auto-detected)** +- **Purpose**: Path to the Podman UNIX socket for container communication +- **Default**: `$XDG_RUNTIME_DIR/podman-host/podman.sock` +- **How to verify**: + ```bash + # Check if socket exists + ls -la /run/user/$(id -u)/podman-host/podman.sock + # Should show: srw-rw---- 1 user user 0 ... /run/user/1000/podman-host/podman.sock + ``` + +**5. `WORKSPACE` (Optional, Auto-detected)** +- **Purpose**: Directory containing the application code for volume mounting +- **Default**: `$GITHUB_WORKSPACE` in CI, `$PWD` for manual runs +- **Usage**: Mounted as `/workspace` inside PiP containers + +**6. `PIP_UID` and `PIP_GID` (Optional)** +- **Purpose**: User and group IDs for the PiP container (security hardening) +- **Default**: `1000:1000` (non-root user) +- **Security Benefit**: Prevents privilege escalation by running as non-root + +**7. `TIMEOUT` and `SLEEP` (pip_ready.sh only)** +- **Purpose**: Control readiness probe timing +- **Defaults**: `TIMEOUT=30` (seconds), `SLEEP=2` (seconds between checks) +- **Usage**: Adjust for slower systems if needed + +#### Environment Setup in CI vs Local + +**In CI Environment (Automatic)**: +- All variables are automatically set by GitHub Actions/Forgejo +- `RUN_ID`, `WORKSPACE`, and other GitHub-specific variables are pre-configured +- Secrets like `PODMAN_CLIENT_IMG_DIGEST` come from repository secrets + +**For Local Testing (Manual Setup)**: +```bash +# Example local testing setup +export PODMAN_CLIENT_IMG_DIGEST="quay.io/podman/stable@sha256:..." +export RUN_ID="local-test" +export WORKSPACE="$(pwd)" + +# Run the scripts +./secure_pip_setup.sh +./pip_ready.sh +``` + +**Verification Commands**: +```bash +# Check current environment variables +env | grep -E "(RUN_ID|PIP_NAME|SOCKET_PATH|WORKSPACE|PODMAN)" + +# Test socket accessibility +ls -la "${SOCKET_PATH}" + +# Verify Podman client image exists +podman manifest inspect "${PODMAN_CLIENT_IMG_DIGEST%%@*}" +``` + ### Step 7: Set Up Ephemeral Podman-in-Podman (PiP) for Secure CI Operations #### 7.1 Secure Ephemeral PiP Container Setup @@ -1924,11 +2007,96 @@ Go to your Forgejo repository and add these secrets in **Settings → Secrets an - `REGISTRY_USERNAME`: Your Forgejo username for registry authentication - `REGISTRY_TOKEN`: Personal Access Token with `write:packages` scope for registry pushes - `SSH_PRIVATE_KEY`: SSH private key for production deployment access +- `SSH_KNOWN_HOSTS`: SSH known_hosts entry for production server +- `PODMAN_CLIENT_IMG_DIGEST`: Pinned Podman client image digest (e.g., `quay.io/podman/stable@sha256:...`) +- `RUST_IMG_DIGEST`: Pinned Rust image digest (e.g., `docker.io/library/rust@sha256:...`) +- `NODE_IMG_DIGEST`: Pinned Node.js image digest (e.g., `docker.io/library/node@sha256:...`) +- `POSTGRES_IMG_DIGEST`: Pinned PostgreSQL image digest (e.g., `docker.io/library/postgres@sha256:...`) **Optional Secrets (for enhanced security):** - `COSIGN_PRIVATE_KEY`: Private key for Cosign image signing - `COSIGN_PASSWORD`: Password for Cosign private key +#### How to Obtain Each Secret (with Purpose and Commands): + +**1. Image Digests (Security-Critical - Prevent Supply Chain Attacks):** + +- **`PODMAN_CLIENT_IMG_DIGEST`**: Used for secure ephemeral PiP containers in CI + ```bash + # Get Podman client image digest + podman manifest inspect quay.io/podman/stable:latest | jq -r '.manifests[0].digest' + # Result: quay.io/podman/stable@sha256:... + ``` + +- **`RUST_IMG_DIGEST`**: Used for Rust backend testing and building + ```bash + # Get Rust image digest + podman manifest inspect docker.io/library/rust:latest | jq -r '.manifests[0].digest' + ``` + +- **`NODE_IMG_DIGEST`**: Used for Node.js frontend testing and building + ```bash + # Get Node.js image digest + podman manifest inspect docker.io/library/node:latest | jq -r '.manifests[0].digest' + ``` + +- **`POSTGRES_IMG_DIGEST`**: Used for PostgreSQL database in integration tests + ```bash + # Get PostgreSQL image digest + podman manifest inspect docker.io/library/postgres:latest | jq -r '.manifests[0].digest' + ``` + +**2. SSH Keys (Secure Deployment Access):** + +- **`SSH_PRIVATE_KEY`**: Private key for CI to deploy to production server + ```bash + # Generate SSH key pair (if you don't have one) + ssh-keygen -t ed25519 -f ~/.ssh/ci_deploy_key -N "" + # Use content of: cat ~/.ssh/ci_deploy_key + ``` + +- **`SSH_KNOWN_HOSTS`**: Prevents MITM attacks during SSH deployment + ```bash + # Get production server's SSH fingerprint + ssh-keyscan -H YOUR_PRODUCTION_IP > known_hosts + # Use content of known_hosts file + ``` + +**3. Forgejo Registry Credentials (Image Storage):** + +- **`REGISTRY_HOST`**: Your Forgejo instance hostname (e.g., `git.example.com`) + *Purpose: Where to push/pull container images* + +- **`REGISTRY_USERNAME`**: Your Forgejo username + *Purpose: Authentication for registry pushes* + +- **`REGISTRY_TOKEN`**: Personal Access Token with `write:packages` scope + *Purpose: Secure authentication without password exposure* + *How to create: Forgejo → Settings → Applications → Generate New Token* + +**4. Application Configuration:** + +- **`CI_HOST`**: Your CI/CD Linode IP address + *Purpose: Testing environment configuration* + +- **`PRODUCTION_IP`**: Your Production Linode IP address + *Purpose: Deployment target* + +- **`PROD_DEPLOY_USER`**: Production deployment username (e.g., `prod-deploy`) + *Purpose: SSH user for deployment operations* + +- **`PROD_SERVICE_USER`**: Production service username (e.g., `prod-service`) + *Purpose: User that runs application services* + +- **`APP_NAME`**: Your application name (e.g., `sharenet`) + *Purpose: Image naming and directory structure* + +- **`POSTGRES_PASSWORD`**: Strong password for PostgreSQL database + ```bash + # Generate secure password + openssl rand -base64 32 + ``` + **Security Note**: All secrets are managed by Forgejo and never exposed in logs or environment variables. The ephemeral PiP approach ensures secrets are only used during execution and never persist.