From 7cc06431da0715830e50781f3cf6d70a13b2c8b3 Mon Sep 17 00:00:00 2001 From: continuist Date: Fri, 4 Jul 2025 22:51:36 -0400 Subject: [PATCH] Changed Part 2 to use Forgejo runner to clone application --- CI_CD_PIPELINE_SETUP_GUIDE.md | 153 ++++++++++------------------------ 1 file changed, 45 insertions(+), 108 deletions(-) diff --git a/CI_CD_PIPELINE_SETUP_GUIDE.md b/CI_CD_PIPELINE_SETUP_GUIDE.md index 84fa0ca..aa2dcca 100644 --- a/CI_CD_PIPELINE_SETUP_GUIDE.md +++ b/CI_CD_PIPELINE_SETUP_GUIDE.md @@ -56,10 +56,10 @@ This guide covers setting up a complete Continuous Integration/Continuous Deploy ## Quick Start 1. **Set up CI/CD Linode** (Steps 1-9) -2. **Set up Production Linode** (Steps 10-18) -3. **Configure SSH key exchange** (Step 16) -4. **Set up Forgejo repository secrets** (Step 19) -5. **Test the complete pipeline** (Step 20) +2. **Set up Production Linode** (Steps 10-16) +3. **Configure SSH key exchange** (Step 14) +4. **Set up Forgejo repository secrets** (Step 18) +5. **Test the complete pipeline** (Step 19) ## What's Included @@ -1320,94 +1320,9 @@ sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin sudo usermod -aG docker PROD_SERVICE_USER ``` -### Step 13: Configure Security +### Step 13: Configure Docker for Harbor Access -#### 13.1 Configure Firewall - -```bash -sudo ufw --force enable -sudo ufw default deny incoming -sudo ufw default allow outgoing -sudo ufw allow ssh -sudo ufw allow 80/tcp -sudo ufw allow 443/tcp -sudo ufw allow 3000/tcp -sudo ufw allow 3001/tcp -``` - -#### 13.2 Configure Fail2ban - -```bash -sudo systemctl enable fail2ban -sudo systemctl start fail2ban -``` - -### Step 14: Create Application Directory - -#### 14.1 Create Directory Structure - -```bash -sudo mkdir -p /opt/APP_NAME -sudo chown PROD_SERVICE_USER:PROD_SERVICE_USER /opt/APP_NAME -``` - -**Note**: Replace `APP_NAME` with your actual application name. This directory name can be controlled via the `APP_NAME` secret in your Forgejo repository settings. If you set the `APP_NAME` secret to `myapp`, the deployment directory will be `/opt/myapp`. - -#### 14.2 Create SSL Directory (Optional - for domain users) - -```bash -sudo mkdir -p /opt/APP_NAME/nginx/ssl -sudo chown PROD_SERVICE_USER:PROD_SERVICE_USER /opt/APP_NAME/nginx/ssl -``` - -### Step 15: Clone Repository and Set Up Application Files - -#### 15.1 Switch to PROD_SERVICE_USER User - -```bash -sudo su - PROD_SERVICE_USER -``` - -#### 15.2 Clone Repository - -```bash -cd /opt/APP_NAME -git clone https://your-forgejo-instance/your-username/APP_NAME.git . -``` - -**Important**: The repository includes a pre-configured `nginx/nginx.conf` file that handles both SSL and non-SSL scenarios, with proper security headers, rate limiting, and CORS configuration. This file will be automatically used by the Docker Compose setup. - -**Important**: The repository also includes a pre-configured `.forgejo/workflows/ci.yml` file that handles the complete CI/CD pipeline including testing, building, and deployment. This workflow is already set up to work with the private registry and production deployment. - -**Note**: Replace `your-forgejo-instance` and `your-username/APP_NAME` with your actual Forgejo instance URL and repository path. - -#### 15.3 Create Environment File - -The repository doesn't include a `.env.example` file for security reasons. The CI/CD pipeline will create the `.env` file dynamically during deployment. However, for manual testing or initial setup, you can create a basic `.env` file: - -```bash -cat > /opt/APP_NAME/.env << 'EOF' -# Production Environment Variables -POSTGRES_PASSWORD=your_secure_password_here -REGISTRY=YOUR_CI_CD_IP:8080 -IMAGE_NAME=APP_NAME -IMAGE_TAG=latest - -# Database Configuration -POSTGRES_DB=sharenet -POSTGRES_USER=sharenet -DATABASE_URL=postgresql://sharenet:your_secure_password_here@postgres:5432/sharenet - -# Application Configuration -NODE_ENV=production -RUST_LOG=info -RUST_BACKTRACE=1 -EOF -``` - -**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address and `your_secure_password_here` with a strong password. - -#### 15.4 Configure Docker for Harbor Access +**Important**: The Production Linode needs to be able to pull Docker images from the Harbor registry on the CI/CD Linode. ```bash # Add the CI/CD Harbor registry to Docker's insecure registries @@ -1424,9 +1339,9 @@ sudo systemctl restart docker **Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. -### Step 16: Set Up SSH Key Authentication +### Step 14: Set Up SSH Key Authentication -#### 16.1 Add CI/CD Public Key +#### 14.1 Add CI/CD Public Key ```bash # Create .ssh directory for PROD_SERVICE_USER @@ -1440,7 +1355,7 @@ chmod 600 ~/.ssh/authorized_keys **Important**: Replace `YOUR_CI_CD_PUBLIC_KEY` with the public key from the CI/CD Linode (the output from `cat ~/.ssh/id_ed25519.pub` on the CI/CD Linode). -#### 16.2 Test SSH Connection +#### 14.2 Test SSH Connection From the CI/CD Linode, test the SSH connection: @@ -1450,11 +1365,11 @@ ssh production **Expected output**: You should be able to SSH to the production server without a password prompt. -### Step 17: Set Up Forgejo Runner for Production Deployment +### Step 15: Set Up Forgejo Runner for Production Deployment **Important**: The Production Linode needs a Forgejo runner to execute the deployment job from the CI/CD workflow. This runner will pull images from Harbor and deploy using `docker-compose.prod.yml`. -#### 17.1 Install Forgejo Runner +#### 15.1 Install Forgejo Runner ```bash # Download the latest Forgejo runner @@ -1470,7 +1385,7 @@ sudo mv forgejo-runner /usr/bin/forgejo-runner forgejo-runner --version ``` -#### 17.2 Set Up Runner Directory for PROD_SERVICE_USER +#### 15.2 Set Up Runner Directory for PROD_SERVICE_USER ```bash # Create runner directory owned by PROD_SERVICE_USER @@ -1478,14 +1393,14 @@ sudo mkdir -p /opt/forgejo-runner sudo chown PROD_SERVICE_USER:PROD_SERVICE_USER /opt/forgejo-runner ``` -#### 17.3 Get Registration Token +#### 15.3 Get Registration Token 1. Go to your Forgejo repository 2. Navigate to **Settings → Actions → Runners** 3. Click **"New runner"** 4. Copy the registration token -#### 17.4 Register the Production Runner +#### 15.4 Register the Production Runner ```bash # Switch to PROD_SERVICE_USER @@ -1505,9 +1420,9 @@ sudo chown PROD_SERVICE_USER:PROD_SERVICE_USER /opt/forgejo-runner/.runner sudo chmod 600 /opt/forgejo-runner/.runner ``` -**Important**: Replace `your-forgejo-instance` with your actual Forgejo instance URL and `YOUR_REGISTRATION_TOKEN` with the token you copied from Step 17.3. +**Important**: Replace `your-forgejo-instance` with your actual Forgejo instance URL and `YOUR_REGISTRATION_TOKEN` with the token you copied from Step 15.3. -#### 17.5 Create Systemd Service +#### 15.5 Create Systemd Service ```bash # Create systemd service file @@ -1538,7 +1453,7 @@ sudo systemctl start forgejo-runner.service sudo systemctl status forgejo-runner.service ``` -#### 17.6 Test Runner Configuration +#### 15.6 Test Runner Configuration ```bash # Check if the runner is running @@ -1565,7 +1480,7 @@ sudo journalctl -u forgejo-runner.service -f --no-pager The production runner will automatically handle the deployment process when you push to the main branch. -#### 17.7 Understanding the Production Docker Compose Setup +#### 15.7 Understanding the Production Docker Compose Setup The `docker-compose.prod.yml` file is specifically designed for production deployment and differs from development setups: @@ -1590,16 +1505,38 @@ The `docker-compose.prod.yml` file is specifically designed for production deplo 4. Waits for all services to be healthy 5. Verifies the deployment was successful -### Step 18: Test Production Setup +### Step 16: Configure Security -#### 18.1 Test Docker Installation +#### 16.1 Configure Firewall + +```bash +sudo ufw --force enable +sudo ufw default deny incoming +sudo ufw default allow outgoing +sudo ufw allow ssh +sudo ufw allow 80/tcp +sudo ufw allow 443/tcp +sudo ufw allow 3000/tcp +sudo ufw allow 3001/tcp +``` + +#### 16.2 Configure Fail2ban + +```bash +sudo systemctl enable fail2ban +sudo systemctl start fail2ban +``` + +### Step 17: Test Production Setup + +#### 17.1 Test Docker Installation ```bash docker --version docker compose --version ``` -#### 18.2 Test Harbor Access +#### 17.2 Test Harbor Access ```bash # Test pulling an image from the CI/CD Harbor registry @@ -1608,14 +1545,14 @@ docker pull YOUR_CI_CD_IP:8080/public/backend:latest **Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. -#### 18.3 Test Application Deployment +#### 17.3 Test Application Deployment ```bash cd /opt/APP_NAME docker compose up -d ``` -#### 18.4 Verify Application Status +#### 17.4 Verify Application Status ```bash docker compose ps