Update prod configuration fail2ban and ufw steps

Clean up CI guide
2025-09-06 12:08:55 -04:00 · 2025-09-06 10:35:23 -04:00
1 changed files with 203 additions and 289 deletions
--- a/CI_CD_PIPELINE_SETUP_GUIDE.md
+++ b/CI_CD_PIPELINE_SETUP_GUIDE.md
@ -1207,7 +1207,7 @@ podman exec ci-pip-local podman run --rm \
  sh -c "cargo test --lib -- --test-threads=1"

 # Run backend integration tests with real database
-podman exec ci-pip podman run --rm \
+podman exec ci-pip-local podman run --rm \
  -v $(pwd)/backend:/workspace \
  -w /workspace \
  -e DATABASE_URL=postgres://testuser:testpassword@localhost:5432/testdb \
@ -1252,32 +1252,6 @@ The CI/CD pipeline uses ephemeral PiP containers with this secure workflow:
 - 🛡️ **Network isolation**: PiP containers have no external network
 - 🛡️ **Ephemeral execution**: Fresh environment every time

-#### 7.2 Configure PiP for Forgejo Container Registry
-
-```bash
-# Navigate to the application directory
-cd /opt/APP_NAME
-
-# Copy client certificates to PiP container
-# Forgejo Container Registry authentication is handled through environment variables
-
-# Test Forgejo Container Registry connectivity from PiP
-podman exec ci-pip podman pull alpine:latest
-podman exec ci-pip podman tag alpine:latest your-forgejo-domain.com/your-username/APP_NAME/test:latest
-
-# Log in to Forgejo Container Registry
-podman exec ci-pip podman login git.gcdo.org -u YOUR_USERNAME -p YOUR_PERSONAL_ACCESS_TOKEN
-
-# Push the image
-podman exec ci-pip podman push your-forgejo-domain.com/your-username/APP_NAME/test:latest
-
-# Test unauthenticated pull from standard port 443
-podman exec ci-pip podman pull your-forgejo-domain.com/your-username/APP_NAME/test:latest
-
-# Clean up test images
-podman exec ci-pip podman rmi your-forgejo-domain.com/your-username/APP_NAME/test:latest
-```
-
 #### 7.3 Set Up Workspace Directory

 **Important**: The CI workflow needs a workspace directory for code checkout. This directory will be used by the Forgejo Actions runner.
@ -1335,7 +1309,7 @@ The Forgejo Container Registry setup uses the built-in registry functionality, p
 - **Tests connectivity**: Verifies DinD can pull, tag, and push images to Forgejo Container Registry
 - **Validates setup**: Ensures the complete CI/CD pipeline will work

-#### 6.4 CI/CD Workflow Architecture with Ephemeral PiP
+#### 7.4 CI/CD Workflow Architecture with Ephemeral PiP

 The CI/CD pipeline uses ephemeral Podman-in-Podman containers with a secure four-stage approach:

@ -1383,31 +1357,7 @@ The CI/CD pipeline uses ephemeral Podman-in-Podman containers with a secure four
 - ✅ **Comprehensive Coverage**: Unit + integration tests
 - ✅ **Isolated Execution**: Each test run completely independent

-**Testing DinD Setup:**
-
-```bash
-# Test DinD functionality
-docker exec ci-dind docker run --rm alpine:latest echo "DinD is working!"
-
-# Test Forgejo Container Registry integration
-docker exec ci-dind docker pull alpine:latest
-docker exec ci-dind docker tag alpine:latest YOUR_CI_CD_IP:4443/APP_NAME/dind-test:latest
-docker exec ci-dind docker push YOUR_CI_CD_IP:4443/APP_NAME/dind-test:latest
-
-# Test unauthenticated pull
-docker exec ci-dind docker pull YOUR_CI_CD_IP/APP_NAME/dind-test:latest
-
-# Clean up test
-docker exec ci-dind docker rmi YOUR_CI_CD_IP:4443/APP_NAME/dind-test:latest
-docker exec ci-dind docker rmi YOUR_CI_CD_IP/APP_NAME/dind-test:latest
-```
-
-**Expected Output**:
- DinD container should be running and accessible
- Docker commands should work inside DinD
- Forgejo Container Registry push/pull should work from DinD
-
-#### 6.5 Production Deployment Architecture
+#### 7.5 Production Deployment Architecture

 The production deployment uses a separate pod configuration (`prod-pod.yaml`) that pulls built images from the Forgejo Container Registry and deploys the complete application stack.

@ -1430,35 +1380,6 @@ The production deployment uses a separate pod configuration (`prod-pod.yaml`) th
 - **🛡️ Rollback Capability**: Can easily rollback to previous image versions
 - **📊 Health Monitoring**: Built-in health checks for all services

-#### 6.6 Monitoring Script
-
-**Important**: The repository includes a pre-configured monitoring script in the `scripts/` directory that can be used for both CI/CD and production monitoring.
-
-**Repository Script**:
- `scripts/monitor.sh` - Comprehensive monitoring script with support for both CI/CD and production environments
-
-**To use the repository monitoring script**:
-```bash
-# The repository is already cloned at /opt/APP_NAME/
-cd /opt/APP_NAME
-
-# Make the script executable
-chmod +x scripts/monitor.sh
-
-# Test CI/CD monitoring
-./scripts/monitor.sh --type ci-cd
-
-# Test production monitoring (if you have a production setup)
-./scripts/monitor.sh --type production
-```
-
-**Note**: The repository script is more comprehensive and includes proper error handling, colored output, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate monitoring information.
-
-**Security Model**:
- **Forgejo Registry**: Integrated authentication and authorization
- **SSH**: Restricted to your IP addresses
- **All other ports**: Blocked
-
 ### Step 8: Security Verification and Testing

 #### 8.1 Security Audit - Verify NO Exposure
@ -1514,15 +1435,15 @@ podman exec ci-pip podman search alpine

 ## Part 2: Production Linode Setup

-### Step 10: Initial System Setup
+### Step 9: Initial System Setup

-#### 10.1 Update the System
+#### 9.1 Update the System

 ```bash
 sudo apt update && sudo apt upgrade -y
 ```

-#### 10.2 Configure Timezone
+#### 9.2 Configure Timezone

 ```bash
 # Configure timezone interactively
@ -1536,7 +1457,7 @@ date

 **Expected output**: After selecting your timezone, the `date` command should show the current date and time in your selected timezone.

-#### 10.3 Configure /etc/hosts
+#### 9.3 Configure /etc/hosts

 ```bash
 # Add localhost entries for both IPv4 and IPv6
@ -1557,7 +1478,7 @@ cat /etc/hosts

 **Expected output**: The `/etc/hosts` file should show entries for `127.0.0.1`, `::1`, and your Linode's actual IP addresses all mapping to `localhost`.

-#### 10.4 Install Essential Packages
+#### 9.4 Install Essential Packages

 ```bash
 sudo apt install -y \
@ -1575,7 +1496,99 @@ sudo apt install -y \
    python3-certbot-nginx
 ```

-#### 10.5 Secure SSH Configuration
+#### 9.5 Configure Firewall and Fail2ban FIRST - Before Any Services
+
+**SECURITY FIRST**: Configure firewall and intrusion prevention BEFORE any services are installed or exposed to prevent attackers from exploiting open ports.
+
+```bash
+# Configure secure firewall defaults
+sudo ufw --force enable
+sudo ufw default deny incoming
+sudo ufw default allow outgoing
+sudo ufw allow ssh
+sudo ufw allow 80/tcp
+sudo ufw allow 443/tcp
+```
+
+**Security Note**: We only allow ports 80 and 443 for external access. The application services (backend on 3001, frontend on 3000) are only accessible through the Nginx reverse proxy, which provides better security and SSL termination.
+
+**Fail2ban Configuration**:
+
+**Fail2ban** is an intrusion prevention system that monitors logs and automatically blocks IP addresses showing malicious behavior.
+
+```bash
+# Install fail2ban (if not already installed)
+sudo apt install -y fail2ban
+
+# Create a custom jail configuration
+sudo tee /etc/fail2ban/jail.local > /dev/null << 'EOF'
+[DEFAULT]
+# Ban time in seconds (24 hours)
+bantime = 86400
+# Find time in seconds (10 minutes)
+findtime = 600
+# Max retries before ban
+maxretry = 3
+# Ban action (use ufw since we're using ufw firewall)
+banaction = ufw
+# Log level
+loglevel = INFO
+# Log target
+logtarget = /var/log/fail2ban.log
+
+# SSH protection
+[sshd]
+enabled = true
+port = ssh
+filter = sshd
+logpath = /var/log/auth.log
+maxretry = 3
+
+# Note: Nginx protection is handled by the firewall and application-level security
+# Docker containers are isolated, and Nginx logs are not directly accessible to fail2ban
+# Web attack protection is provided by:
+# 1. UFW firewall (ports 80/443 only)
+# 2. Nginx security headers and rate limiting
+# 3. Application-level input validation
+EOF
+
+# Enable and start fail2ban
+sudo systemctl enable fail2ban
+sudo systemctl start fail2ban
+
+# Verify fail2ban is running
+sudo systemctl status fail2ban
+
+# Check current jails
+sudo fail2ban-client status
+```
+
+**What this does**:
+- **SSH Protection**: Blocks IPs that fail SSH login 3 times in 10 minutes
+- **24-hour bans**: Banned IPs are blocked for 24 hours
+- **Automatic monitoring**: Continuously watches SSH logs
+
+**Web Security Note**: Since Nginx runs in a Docker container, web attack protection is handled by:
+- **UFW Firewall**: Only allows ports 80/443 (no direct access to app services)
+- **Nginx Security**: Built-in rate limiting and security headers
+- **Application Security**: Input validation in the backend/frontend code
+
+**Monitoring Fail2ban**:
+```bash
+# Check banned IPs
+sudo fail2ban-client status sshd
+
+# Unban an IP if needed
+sudo fail2ban-client set sshd unbanip IP_ADDRESS
+
+# View fail2ban logs
+sudo tail -f /var/log/fail2ban.log
+
+# Check all active jails
+sudo fail2ban-client status
+```
+
+#### 9.6 Secure SSH Configuration

 **Critical Security Step**: After setting up SSH key authentication, you must disable password authentication and root login to secure your Production server.

@ -1722,9 +1735,9 @@ sudo sshd -t
 sudo systemctl restart ssh
 ```

-### Step 11: Create Users
+### Step 10: Create Users

-#### 11.1 Create the PROD_SERVICE_USER User
+#### 10.1 Create the PROD_SERVICE_USER User

 ```bash
 # Create dedicated group for the production service account
@ -1735,7 +1748,7 @@ sudo useradd -r -g PROD_SERVICE_USER -s /bin/bash -m -d /home/PROD_SERVICE_USER
 echo "PROD_SERVICE_USER:$(openssl rand -base64 32)" | sudo chpasswd
 ```

-#### 11.2 Verify Users
+#### 10.2 Verify Users

 ```bash
 sudo su - PROD_SERVICE_USER
@ -1749,9 +1762,9 @@ pwd
 exit
 ```

-### Step 12: Install Podman
+### Step 11: Install Podman

-#### 12.1 Install Podman
+#### 11.1 Install Podman

 ```bash
 # Install Podman and related tools
@ -1761,14 +1774,54 @@ sudo apt install -y podman
 podman --version
 ```

-#### 12.2 Configure Podman for Production Service Account
+#### 11.2 Configure Podman for Production Service Account Only
+
+**Security Restriction**: Configure Podman to only be usable by PROD_SERVICE_USER to prevent unauthorized container operations.

 ```bash
-# Podman runs rootless by default, no group membership needed
-# The subuid/subgid ranges configured in Step 1.5 enable rootless operation
+# Create podman group for controlled access
+sudo groupadd podman
+
+# Add PROD_SERVICE_USER to podman group
+sudo usermod -aG podman PROD_SERVICE_USER
+
+# Configure Podman socket permissions (restrict to podman group)
+sudo mkdir -p /etc/containers
+sudo tee /etc/containers/containers.conf > /dev/null << 'EOF'
+[engine]
+events_logger = "file"
+
+[network]
+dns_bind_port = 53
+
+[engine.runtimes]
+runc = [
+    "/usr/bin/runc",
+]
+
+[engine.socket_group]
+group = "podman"
+EOF
+
+# Configure user namespace for rootless operation (if not already done)
+echo 'kernel.unprivileged_userns_clone=1' | sudo tee -a /etc/sysctl.conf
+sudo sysctl -p
+
+# Verify PROD_SERVICE_USER can access Podman
+sudo -u PROD_SERVICE_USER podman --version
+
+# Test that other users cannot access Podman socket
+sudo -u PROD_DEPLOY_USER podman --version || echo "Good: PROD_DEPLOY_USER cannot access Podman"
 ```

-#### 12.4 Create Application Directory
+**What this does**:
+- Creates dedicated `podman` group for controlled access
+- Restricts Podman socket access to only members of the `podman` group  
+- Ensures only PROD_SERVICE_USER can execute Podman commands
+- Prevents PROD_DEPLOY_USER and other users from running containers
+- Maintains rootless operation for security
+
+#### 11.4 Create Application Directory

 ```bash
 # Create application directory for deployment
@ -1785,33 +1838,11 @@ ls -la /opt/APP_NAME
 - Sets proper ownership for the PROD_SERVICE_USER
 - Ensures the directory exists before the CI workflow runs

-### Step 13: Configure Podman for Forgejo Container Registry Access
-
-**Important**: The Production Linode needs to be able to pull images from the Forgejo Container Registry. Authentication is handled through the CI/CD pipeline configuration.
-
-```bash
-# Change to the PROD_SERVICE_USER
-sudo su - PROD_SERVICE_USER
-
-# Test Forgejo Container Registry access will be verified during deployment
-# Images are pulled from the configured REGISTRY_HOST during the CI/CD process
-
-# Change back to PROD_DEPLOY_USER
-exit
-```
-
-**Note**: Production deployments pull images from Forgejo Container Registry using the configured authentication.
-
-**What this does**:
- **Tests Forgejo Container Registry access**: Registry access is verified during the deployment process
- **Integrated authentication**: Forgejo handles authentication automatically
- **Simple setup**: No complex certificate management required
-
-### Step 14: Set Up Forgejo Runner for Production Deployment
+### Step 12: Set Up Forgejo Runner for Production Deployment

 **Important**: The Production Linode needs a Forgejo runner to execute the deployment job from the CI/CD workflow. This runner will pull images from Forgejo Container Registry and deploy using the production pod configuration.

-#### 14.1 Download Runner
+#### 12.1 Download Runner

 **Important**: Run this step as the **PROD_DEPLOY_USER** (not root or PROD_SERVICE_USER). The PROD_DEPLOY_USER handles deployment tasks including downloading and installing the Forgejo runner.

@ -1848,14 +1879,14 @@ sudo mv forgejo-runner-${VERSION#v}-linux-amd64 /usr/bin/forgejo-runner

 **Production Recommendation**: Use version pinning in production environments to ensure consistency and avoid unexpected breaking changes.

-#### 14.2 Get Registration Token
+#### 12.2 Get Registration Token

 1. Go to your Forgejo repository
 2. Navigate to **Settings → Actions → Runners**
 3. Click **"New runner"**
 4. Copy the registration token

-#### 14.3 Register the Production Runner
+#### 12.3 Register the Production Runner

 **Step 1: Register the Runner**

@ -1903,7 +1934,7 @@ sudo chmod 600 /etc/forgejo-runner/.runner
 - Registers the runner with your Forgejo instance
 - Sets up the runner with appropriate labels for production deployment

-#### 14.4 Create Systemd Service
+#### 12.4 Create Systemd Service

 ```bash
 # Create systemd service file
@ -1933,7 +1964,7 @@ sudo systemctl start forgejo-runner.service
 sudo systemctl status forgejo-runner.service
 ```

-#### 14.5 Test Runner Configuration
+#### 12.5 Test Runner Configuration

 ```bash
 # Check if the runner is running
@ -1967,7 +1998,7 @@ When the workflow runs, it will:

 The production runner will automatically handle the deployment process when you push to the main branch.

-#### 14.6 Understanding the Production Pod Setup
+#### 12.6 Understanding the Production Pod Setup

 The `prod-pod.yaml` file is specifically designed for production deployment and uses Kubernetes pod specifications:

@ -1992,115 +2023,22 @@ The `prod-pod.yaml` file is specifically designed for production deployment and
 4. Waits for all services to be healthy
 5. Verifies the deployment was successful

-### Step 15: Configure Security

-#### 15.1 Configure Firewall
+
+### Step 13: Test Production Setup
+
+#### 13.1 Test Podman Installation

 ```bash
-sudo ufw --force enable
-sudo ufw default deny incoming
-sudo ufw default allow outgoing
-sudo ufw allow ssh
-sudo ufw allow 80/tcp
-sudo ufw allow 443/tcp
+# Test Podman installation (run as PROD_SERVICE_USER)
+sudo -u PROD_SERVICE_USER podman --version
 ```

-**Security Note**: We only allow ports 80 and 443 for external access. The application services (backend on 3001, frontend on 3000) are only accessible through the Nginx reverse proxy, which provides better security and SSL termination.
-
-#### 15.2 Configure Fail2ban
-
-**Fail2ban** is an intrusion prevention system that monitors logs and automatically blocks IP addresses showing malicious behavior.
+#### 13.2 Test Forgejo Container Registry Access

 ```bash
-# Install fail2ban (if not already installed)
-sudo apt install -y fail2ban
-
-# Create a custom jail configuration
-sudo tee /etc/fail2ban/jail.local > /dev/null << 'EOF'
-[DEFAULT]
-# Ban time in seconds (24 hours)
-bantime = 86400
-# Find time in seconds (10 minutes)
-findtime = 600
-# Max retries before ban
-maxretry = 3
-# Ban action (use ufw since we're using ufw firewall)
-banaction = ufw
-# Log level
-loglevel = INFO
-# Log target
-logtarget = /var/log/fail2ban.log
-
-# SSH protection
-[sshd]
-enabled = true
-port = ssh
-filter = sshd
-logpath = /var/log/auth.log
-maxretry = 3
-
-# Note: Nginx protection is handled by the firewall and application-level security
-# Docker containers are isolated, and Nginx logs are not directly accessible to fail2ban
-# Web attack protection is provided by:
-# 1. UFW firewall (ports 80/443 only)
-# 2. Nginx security headers and rate limiting
-# 3. Application-level input validation
-EOF
-
-# Enable and start fail2ban
-sudo systemctl enable fail2ban
-sudo systemctl start fail2ban
-
-# Verify fail2ban is running
-sudo systemctl status fail2ban
-
-# Check current jails
-sudo fail2ban-client status
-```
-
-**What this does**:
- **SSH Protection**: Blocks IPs that fail SSH login 3 times in 10 minutes
- **24-hour bans**: Banned IPs are blocked for 24 hours
- **Automatic monitoring**: Continuously watches SSH logs
-
-**Web Security Note**: Since Nginx runs in a Docker container, web attack protection is handled by:
- **UFW Firewall**: Only allows ports 80/443 (no direct access to app services)
- **Nginx Security**: Built-in rate limiting and security headers
- **Application Security**: Input validation in the backend/frontend code
-
-**Monitoring Fail2ban**:
-```bash
-# Check banned IPs
-sudo fail2ban-client status sshd
-
-# Unban an IP if needed
-sudo fail2ban-client set sshd unbanip IP_ADDRESS
-
-# View fail2ban logs
-sudo tail -f /var/log/fail2ban.log
-
-# Check all active jails
-sudo fail2ban-client status
-
-**Why This Matters for Production**:
- **Your server is exposed**: The Production Linode is accessible from the internet
- **Automated attacks**: Bots constantly scan for vulnerable servers
- **Resource protection**: Prevents attackers from consuming CPU/memory
- **Security layers**: Works with the firewall to provide defense in depth
-
-### Step 16: Test Production Setup
-
-#### 16.1 Test Podman Installation
-
-```bash
-podman --version
-```
-
-#### 16.2 Test Forgejo Container Registry Access
-
-```bash
-# Test pulling an image from Forgejo Container Registry
-podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
+# Test pulling an image from Forgejo Container Registry (run as PROD_SERVICE_USER)
+sudo -u PROD_SERVICE_USER podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
 ```

 **Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address.
@ -2113,17 +2051,12 @@ podman pull YOUR_CI_CD_IP/APP_NAME/test:latest

 ## Part 3: Final Configuration and Testing

-### Step 17: Configure Forgejo Repository Secrets
+### Step 14: Configure Forgejo Repository Secrets

 Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**:

 **Required Secrets:**
- `CI_HOST`: Your CI/CD Linode IP address (used for testing environment)
- `PRODUCTION_IP`: Your Production Linode IP address
- `PROD_DEPLOY_USER`: The production deployment user name (e.g., `prod-deploy`)
- `PROD_SERVICE_USER`: The production service user name (e.g., `prod-service`)
 - `APP_NAME`: Your application name (e.g., `sharenet`)
- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database
 - `REGISTRY_HOST`: Your Forgejo instance's registry URL
 - `REGISTRY_USERNAME`: Your Forgejo username for registry authentication
 - `REGISTRY_TOKEN`: Personal Access Token with `write:packages` scope for registry pushes
@ -2134,10 +2067,6 @@ Go to your Forgejo repository and add these secrets in **Settings → Secrets an
 - `NODE_IMG_DIGEST`: Pinned Node.js image digest (e.g., `docker.io/library/node@sha256:...`)
 - `POSTGRES_IMG_DIGEST`: Pinned PostgreSQL image digest (e.g., `docker.io/library/postgres@sha256:...`)

-**Optional Secrets (for enhanced security):**
- `COSIGN_PRIVATE_KEY`: Private key for Cosign image signing
- `COSIGN_PASSWORD`: Password for Cosign private key
-
 #### How to Obtain Each Secret (with Purpose and Commands):

 **1. Image Digests (Security-Critical - Prevent Supply Chain Attacks):**
@ -2208,42 +2137,22 @@ Go to your Forgejo repository and add these secrets in **Settings → Secrets an

 **4. Application Configuration:**

- **`CI_HOST`**: Your CI/CD Linode IP address
-  *Purpose: Testing environment configuration*
-
- **`PRODUCTION_IP`**: Your Production Linode IP address  
-  *Purpose: Deployment target*
-
- **`PROD_DEPLOY_USER`**: Production deployment username (e.g., `prod-deploy`)
-  *Purpose: SSH user for deployment operations*
-
- **`PROD_SERVICE_USER`**: Production service username (e.g., `prod-service`)
-  *Purpose: User that runs application services*
-
 - **`APP_NAME`**: Your application name (e.g., `sharenet`)
  *Purpose: Image naming and directory structure*

- **`POSTGRES_PASSWORD`**: Strong password for PostgreSQL database
-  ```bash
-  # Generate secure password
-  openssl rand -base64 32
-  ```
-
 **Security Note**: All secrets are managed by Forgejo and never exposed in logs or environment variables. The ephemeral PiP approach ensures secrets are only used during execution and never persist.

-
-
 **Note**: This setup uses custom Dockerfiles for testing environments with base images. The CI pipeline automatically checks if base images exist in Forgejo Container Registry and pulls them from Docker Hub only when needed, eliminating rate limiting issues and providing better control over the testing environment.

-### Step 18: Test Complete Pipeline
+### Step 15: Test Complete Pipeline

-#### 18.1 Trigger a Test Build
+#### 15.1 Trigger a Test Build

 1. **Make a small change** to your repository (e.g., update a comment or add a test file)
 2. **Commit and push** the changes to trigger the CI/CD pipeline
 3. **Monitor the build** in your Forgejo repository → Actions tab

-#### 18.2 Verify Pipeline Steps
+#### 15.2 Verify Pipeline Steps

 The pipeline should execute these steps in order:

@ -2256,7 +2165,7 @@ The pipeline should execute these steps in order:
 7. **Push to Registry**: Push images to Forgejo Container Registry from DinD
 8. **Deploy to Production**: Deploy to production server

-#### 18.3 Check Forgejo Container Registry
+#### 15.3 Check Forgejo Container Registry

 ```bash
 # On CI/CD Linode
@ -2277,7 +2186,7 @@ curl -k https://YOUR_CI_CD_IP:4443/v2/_catalog
 # Expected: This should return authentication error without credentials
 ```

-#### 18.4 Verify Production Deployment
+#### 15.4 Verify Production Deployment

 ```bash
 # On Production Linode
@ -2295,16 +2204,16 @@ podman logs sharenet-production-pod-backend
 podman logs sharenet-production-pod-frontend
 ```

-#### 18.5 Test Application Functionality
+#### 15.5 Test Application Functionality

 1. **Frontend**: Visit your production URL (IP address)
 2. **Backend API**: Test API endpoints
 3. **Database**: Verify database connections
 4. **Logs**: Check for any errors in application logs

-### Step 19: Final Verification
+### Step 16: Final Verification

-#### 19.1 Security Check
+#### 16.1 Security Check

 ```bash
 # Check firewall status
@ -2317,7 +2226,7 @@ sudo systemctl status fail2ban
 sudo grep "PasswordAuthentication" /etc/ssh/sshd_config
 ```

-#### 19.2 Performance Check
+#### 16.2 Performance Check

 ```bash
 # Check system resources
@ -2330,7 +2239,7 @@ df -h
 docker system df
 ```

-#### 19.3 Backup Verification
+#### 16.3 Backup Verification

 ```bash
 # Test backup script
@ -2341,16 +2250,16 @@ cd /opt/APP_NAME
 ./scripts/backup.sh
 ```

-### Step 20: Documentation and Maintenance
+### Step 17: Documentation and Maintenance

-#### 20.1 Update Documentation
+#### 17.1 Update Documentation

 1. **Update README.md** with deployment information
 2. **Document environment variables** and their purposes
 3. **Create troubleshooting guide** for common issues
 4. **Document backup and restore procedures**

-#### 20.2 Set Up Monitoring Alerts
+#### 17.2 Set Up Monitoring Alerts

 ```bash
 # Set up monitoring cron job
@ -2360,7 +2269,7 @@ cd /opt/APP_NAME
 tail -f /tmp/monitor.log
 ```

-#### 20.3 Regular Maintenance Tasks
+#### 17.3 Regular Maintenance Tasks

 **Daily:**
 - Check application logs for errors
@ -2392,15 +2301,20 @@ Before using this setup, ensure that:

 ### Repository Secrets

-The CI pipeline expects the following secrets to be configured in your Forgejo repository:
+The CI pipeline uses the following secrets. Only the required secrets are needed for current CI operations:

-| Secret Name | Description | Example |
-|-------------|-------------|---------|
-| `REGISTRY_HOST` | Your Forgejo instance hostname | `forgejo.example.com` |
-| `REGISTRY_USERNAME` | Bot or owner account username | `ci-bot` |
-| `REGISTRY_TOKEN` | PAT with `write:packages` scope | `gto_...` |
-| `COSIGN_PRIVATE_KEY` | (Optional) Private key for image signing | `-----BEGIN PRIVATE KEY-----...` |
-| `COSIGN_PASSWORD` | (Optional) Password for Cosign private key | `your-password` |
+| Secret Name | Description | Example | Required |
+|-------------|-------------|---------|----------|
+| `APP_NAME` | Your application name | `sharenet` | ✅ Yes |
+| `REGISTRY_HOST` | Your Forgejo instance hostname | `forgejo.example.com` | ✅ Yes |
+| `REGISTRY_USERNAME` | Bot or owner account username | `ci-bot` | ✅ Yes |
+| `REGISTRY_TOKEN` | PAT with `write:packages` scope | `gto_...` | ✅ Yes |
+| `SSH_PRIVATE_KEY` | SSH private key for deployment | `-----BEGIN OPENSSH PRIVATE KEY-----...` | ✅ Yes |
+| `SSH_KNOWN_HOSTS` | SSH known_hosts entry | `production-ip ecdsa-sha2-nistp256...` | ✅ Yes |
+| `PODMAN_CLIENT_IMG_DIGEST` | Pinned Podman client image digest | `quay.io/podman/stable@sha256:...` | ✅ Yes |
+| `RUST_IMG_DIGEST` | Pinned Rust image digest | `docker.io/library/rust@sha256:...` | ✅ Yes |
+| `NODE_IMG_DIGEST` | Pinned Node.js image digest | `docker.io/library/node@sha256:...` | ✅ Yes |
+| `POSTGRES_IMG_DIGEST` | Pinned PostgreSQL image digest | `docker.io/library/postgres@sha256:...` | ✅ Yes |

 ### How It Works

@ -2419,11 +2333,11 @@ The CI pipeline has been updated to:
 Since the repository is public, applications can pull images anonymously:

 ```bash
-# Pull by tag
-podman pull forgejo.example.com/owner/repo/backend:latest
+# Pull by tag (run as PROD_SERVICE_USER in production)
+sudo -u PROD_SERVICE_USER podman pull forgejo.example.com/owner/repo/backend:latest

-# Pull by digest
-podman pull forgejo.example.com/owner/repo/backend@sha256:abc123...
+# Pull by digest (run as PROD_SERVICE_USER in production)
+sudo -u PROD_SERVICE_USER podman pull forgejo.example.com/owner/repo/backend@sha256:abc123...
 ```

 #### Image Naming Convention
@ -2468,12 +2382,12 @@ For example:
 # Test registry access
 podman login REGISTRY_HOST -u REGISTRY_USERNAME -p REGISTRY_TOKEN

-# List available images
-podman search REGISTRY_HOST/OWNER_REPO
+# List available images (run as PROD_SERVICE_USER in production)
+sudo -u PROD_SERVICE_USER podman search REGISTRY_HOST/OWNER_REPO

-# Pull and verify image
-podman pull REGISTRY_HOST/OWNER_REPO/backend:TAG
-podman image inspect REGISTRY_HOST/OWNER_REPO/backend:TAG
+# Pull and verify image (run as PROD_SERVICE_USER in production)
+sudo -u PROD_SERVICE_USER podman pull REGISTRY_HOST/OWNER_REPO/backend:TAG
+sudo -u PROD_SERVICE_USER podman image inspect REGISTRY_HOST/OWNER_REPO/backend:TAG
 ```

 ---
Author	SHA1	Message	Date
continuist	1ea4dc32e5	Update prod configuration fail2ban and ufw steps Some checks are pending CI/CD Pipeline with Secure Ephemeral PiP / test-backend (push) Waiting to run Details CI/CD Pipeline with Secure Ephemeral PiP / test-frontend (push) Blocked by required conditions Details CI/CD Pipeline with Secure Ephemeral PiP / build-backend (push) Blocked by required conditions Details CI/CD Pipeline with Secure Ephemeral PiP / build-frontend (push) Blocked by required conditions Details	2025-09-06 12:08:55 -04:00
continuist	b0234f13b5	Clean up CI guide	2025-09-06 10:35:23 -04:00