devteam/sharenet
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: devteam:1ea4dc32e55312725ac80ddb5b294f07780d7221
Branches Tags
devteam:main
..
compare: devteam:6e0d66a200b901d1f89fa6cd1ca26f2ae456abe2
Branches Tags
devteam:main

No commits in common. "1ea4dc32e55312725ac80ddb5b294f07780d7221" and "6e0d66a200b901d1f89fa6cd1ca26f2ae456abe2" have entirely different histories.
1ea4dc32e5 ... 6e0d66a200

1 changed files with 289 additions and 203 deletions
Show stats Expand all files Collapse all files

492
CI_CD_PIPELINE_SETUP_GUIDE.md
View file

@ -1207,7 +1207,7 @@ podman exec ci-pip-local podman run --rm \
sh -c "cargo test --lib -- --test-threads=1" sh -c "cargo test --lib -- --test-threads=1"
# Run backend integration tests with real database # Run backend integration tests with real database
podman exec ci-pip-local podman run --rm \ podman exec ci-pip podman run --rm \
-v $(pwd)/backend:/workspace \ -v $(pwd)/backend:/workspace \
-w /workspace \ -w /workspace \
-e DATABASE_URL=postgres://testuser:testpassword@localhost:5432/testdb \ -e DATABASE_URL=postgres://testuser:testpassword@localhost:5432/testdb \
@ -1252,6 +1252,32 @@ The CI/CD pipeline uses ephemeral PiP containers with this secure workflow:
- 🛡️ **Network isolation**: PiP containers have no external network - 🛡️ **Network isolation**: PiP containers have no external network
- 🛡️ **Ephemeral execution**: Fresh environment every time - 🛡️ **Ephemeral execution**: Fresh environment every time
#### 7.2 Configure PiP for Forgejo Container Registry
```bash
# Navigate to the application directory
cd /opt/APP_NAME
# Copy client certificates to PiP container
# Forgejo Container Registry authentication is handled through environment variables
# Test Forgejo Container Registry connectivity from PiP
podman exec ci-pip podman pull alpine:latest
podman exec ci-pip podman tag alpine:latest your-forgejo-domain.com/your-username/APP_NAME/test:latest
# Log in to Forgejo Container Registry
podman exec ci-pip podman login git.gcdo.org -u YOUR_USERNAME -p YOUR_PERSONAL_ACCESS_TOKEN
# Push the image
podman exec ci-pip podman push your-forgejo-domain.com/your-username/APP_NAME/test:latest
# Test unauthenticated pull from standard port 443
podman exec ci-pip podman pull your-forgejo-domain.com/your-username/APP_NAME/test:latest
# Clean up test images
podman exec ci-pip podman rmi your-forgejo-domain.com/your-username/APP_NAME/test:latest
```
#### 7.3 Set Up Workspace Directory #### 7.3 Set Up Workspace Directory
**Important**: The CI workflow needs a workspace directory for code checkout. This directory will be used by the Forgejo Actions runner. **Important**: The CI workflow needs a workspace directory for code checkout. This directory will be used by the Forgejo Actions runner.
@ -1309,7 +1335,7 @@ The Forgejo Container Registry setup uses the built-in registry functionality, p
- **Tests connectivity**: Verifies DinD can pull, tag, and push images to Forgejo Container Registry - **Tests connectivity**: Verifies DinD can pull, tag, and push images to Forgejo Container Registry
- **Validates setup**: Ensures the complete CI/CD pipeline will work - **Validates setup**: Ensures the complete CI/CD pipeline will work
#### 7.4 CI/CD Workflow Architecture with Ephemeral PiP #### 6.4 CI/CD Workflow Architecture with Ephemeral PiP
The CI/CD pipeline uses ephemeral Podman-in-Podman containers with a secure four-stage approach: The CI/CD pipeline uses ephemeral Podman-in-Podman containers with a secure four-stage approach:
@ -1357,7 +1383,31 @@ The CI/CD pipeline uses ephemeral Podman-in-Podman containers with a secure four
- ✅ **Comprehensive Coverage**: Unit + integration tests - ✅ **Comprehensive Coverage**: Unit + integration tests
- ✅ **Isolated Execution**: Each test run completely independent - ✅ **Isolated Execution**: Each test run completely independent
#### 7.5 Production Deployment Architecture **Testing DinD Setup:**
```bash
# Test DinD functionality
docker exec ci-dind docker run --rm alpine:latest echo "DinD is working!"
# Test Forgejo Container Registry integration
docker exec ci-dind docker pull alpine:latest
docker exec ci-dind docker tag alpine:latest YOUR_CI_CD_IP:4443/APP_NAME/dind-test:latest
docker exec ci-dind docker push YOUR_CI_CD_IP:4443/APP_NAME/dind-test:latest
# Test unauthenticated pull
docker exec ci-dind docker pull YOUR_CI_CD_IP/APP_NAME/dind-test:latest
# Clean up test
docker exec ci-dind docker rmi YOUR_CI_CD_IP:4443/APP_NAME/dind-test:latest
docker exec ci-dind docker rmi YOUR_CI_CD_IP/APP_NAME/dind-test:latest
```
**Expected Output**:
- DinD container should be running and accessible
- Docker commands should work inside DinD
- Forgejo Container Registry push/pull should work from DinD
#### 6.5 Production Deployment Architecture
The production deployment uses a separate pod configuration (`prod-pod.yaml`) that pulls built images from the Forgejo Container Registry and deploys the complete application stack. The production deployment uses a separate pod configuration (`prod-pod.yaml`) that pulls built images from the Forgejo Container Registry and deploys the complete application stack.
@ -1380,6 +1430,35 @@ The production deployment uses a separate pod configuration (`prod-pod.yaml`) th
- **🛡️ Rollback Capability**: Can easily rollback to previous image versions - **🛡️ Rollback Capability**: Can easily rollback to previous image versions
- **📊 Health Monitoring**: Built-in health checks for all services - **📊 Health Monitoring**: Built-in health checks for all services
#### 6.6 Monitoring Script
**Important**: The repository includes a pre-configured monitoring script in the `scripts/` directory that can be used for both CI/CD and production monitoring.
**Repository Script**:
- `scripts/monitor.sh` - Comprehensive monitoring script with support for both CI/CD and production environments
**To use the repository monitoring script**:
```bash
# The repository is already cloned at /opt/APP_NAME/
cd /opt/APP_NAME
# Make the script executable
chmod +x scripts/monitor.sh
# Test CI/CD monitoring
./scripts/monitor.sh --type ci-cd
# Test production monitoring (if you have a production setup)
./scripts/monitor.sh --type production
```
**Note**: The repository script is more comprehensive and includes proper error handling, colored output, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate monitoring information.
**Security Model**:
- **Forgejo Registry**: Integrated authentication and authorization
- **SSH**: Restricted to your IP addresses
- **All other ports**: Blocked
### Step 8: Security Verification and Testing ### Step 8: Security Verification and Testing
#### 8.1 Security Audit - Verify NO Exposure #### 8.1 Security Audit - Verify NO Exposure
@ -1435,15 +1514,15 @@ podman exec ci-pip podman search alpine
## Part 2: Production Linode Setup ## Part 2: Production Linode Setup
### Step 9: Initial System Setup ### Step 10: Initial System Setup
#### 9.1 Update the System #### 10.1 Update the System
```bash ```bash
sudo apt update && sudo apt upgrade -y sudo apt update && sudo apt upgrade -y
``` ```
#### 9.2 Configure Timezone #### 10.2 Configure Timezone
```bash ```bash
# Configure timezone interactively # Configure timezone interactively
@ -1457,7 +1536,7 @@ date
**Expected output**: After selecting your timezone, the `date` command should show the current date and time in your selected timezone. **Expected output**: After selecting your timezone, the `date` command should show the current date and time in your selected timezone.
#### 9.3 Configure /etc/hosts #### 10.3 Configure /etc/hosts
```bash ```bash
# Add localhost entries for both IPv4 and IPv6 # Add localhost entries for both IPv4 and IPv6
@ -1478,7 +1557,7 @@ cat /etc/hosts
**Expected output**: The `/etc/hosts` file should show entries for `127.0.0.1`, `::1`, and your Linode's actual IP addresses all mapping to `localhost`. **Expected output**: The `/etc/hosts` file should show entries for `127.0.0.1`, `::1`, and your Linode's actual IP addresses all mapping to `localhost`.
#### 9.4 Install Essential Packages #### 10.4 Install Essential Packages
```bash ```bash
sudo apt install -y \ sudo apt install -y \
@ -1496,99 +1575,7 @@ sudo apt install -y \
python3-certbot-nginx python3-certbot-nginx
``` ```
#### 9.5 Configure Firewall and Fail2ban FIRST - Before Any Services #### 10.5 Secure SSH Configuration
**SECURITY FIRST**: Configure firewall and intrusion prevention BEFORE any services are installed or exposed to prevent attackers from exploiting open ports.
```bash
# Configure secure firewall defaults
sudo ufw --force enable
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
```
**Security Note**: We only allow ports 80 and 443 for external access. The application services (backend on 3001, frontend on 3000) are only accessible through the Nginx reverse proxy, which provides better security and SSL termination.
**Fail2ban Configuration**:
**Fail2ban** is an intrusion prevention system that monitors logs and automatically blocks IP addresses showing malicious behavior.
```bash
# Install fail2ban (if not already installed)
sudo apt install -y fail2ban
# Create a custom jail configuration
sudo tee /etc/fail2ban/jail.local > /dev/null << 'EOF'
[DEFAULT]
# Ban time in seconds (24 hours)
bantime = 86400
# Find time in seconds (10 minutes)
findtime = 600
# Max retries before ban
maxretry = 3
# Ban action (use ufw since we're using ufw firewall)
banaction = ufw
# Log level
loglevel = INFO
# Log target
logtarget = /var/log/fail2ban.log
# SSH protection
[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
# Note: Nginx protection is handled by the firewall and application-level security
# Docker containers are isolated, and Nginx logs are not directly accessible to fail2ban
# Web attack protection is provided by:
# 1. UFW firewall (ports 80/443 only)
# 2. Nginx security headers and rate limiting
# 3. Application-level input validation
EOF
# Enable and start fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
# Verify fail2ban is running
sudo systemctl status fail2ban
# Check current jails
sudo fail2ban-client status
```
**What this does**:
- **SSH Protection**: Blocks IPs that fail SSH login 3 times in 10 minutes
- **24-hour bans**: Banned IPs are blocked for 24 hours
- **Automatic monitoring**: Continuously watches SSH logs
**Web Security Note**: Since Nginx runs in a Docker container, web attack protection is handled by:
- **UFW Firewall**: Only allows ports 80/443 (no direct access to app services)
- **Nginx Security**: Built-in rate limiting and security headers
- **Application Security**: Input validation in the backend/frontend code
**Monitoring Fail2ban**:
```bash
# Check banned IPs
sudo fail2ban-client status sshd
# Unban an IP if needed
sudo fail2ban-client set sshd unbanip IP_ADDRESS
# View fail2ban logs
sudo tail -f /var/log/fail2ban.log
# Check all active jails
sudo fail2ban-client status
```
#### 9.6 Secure SSH Configuration
**Critical Security Step**: After setting up SSH key authentication, you must disable password authentication and root login to secure your Production server. **Critical Security Step**: After setting up SSH key authentication, you must disable password authentication and root login to secure your Production server.
@ -1735,9 +1722,9 @@ sudo sshd -t
sudo systemctl restart ssh sudo systemctl restart ssh
``` ```
### Step 10: Create Users ### Step 11: Create Users
#### 10.1 Create the PROD_SERVICE_USER User #### 11.1 Create the PROD_SERVICE_USER User
```bash ```bash
# Create dedicated group for the production service account # Create dedicated group for the production service account
@ -1748,7 +1735,7 @@ sudo useradd -r -g PROD_SERVICE_USER -s /bin/bash -m -d /home/PROD_SERVICE_USER
echo "PROD_SERVICE_USER:$(openssl rand -base64 32)" | sudo chpasswd echo "PROD_SERVICE_USER:$(openssl rand -base64 32)" | sudo chpasswd
``` ```
#### 10.2 Verify Users #### 11.2 Verify Users
```bash ```bash
sudo su - PROD_SERVICE_USER sudo su - PROD_SERVICE_USER
@ -1762,9 +1749,9 @@ pwd
exit exit
``` ```
### Step 11: Install Podman ### Step 12: Install Podman
#### 11.1 Install Podman #### 12.1 Install Podman
```bash ```bash
# Install Podman and related tools # Install Podman and related tools
@ -1774,54 +1761,14 @@ sudo apt install -y podman
podman --version podman --version
``` ```
#### 11.2 Configure Podman for Production Service Account Only #### 12.2 Configure Podman for Production Service Account
**Security Restriction**: Configure Podman to only be usable by PROD_SERVICE_USER to prevent unauthorized container operations.
```bash ```bash
# Create podman group for controlled access # Podman runs rootless by default, no group membership needed
sudo groupadd podman # The subuid/subgid ranges configured in Step 1.5 enable rootless operation
# Add PROD_SERVICE_USER to podman group
sudo usermod -aG podman PROD_SERVICE_USER
# Configure Podman socket permissions (restrict to podman group)
sudo mkdir -p /etc/containers
sudo tee /etc/containers/containers.conf > /dev/null << 'EOF'
[engine]
events_logger = "file"
[network]
dns_bind_port = 53
[engine.runtimes]
runc = [
"/usr/bin/runc",
]
[engine.socket_group]
group = "podman"
EOF
# Configure user namespace for rootless operation (if not already done)
echo 'kernel.unprivileged_userns_clone=1' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
# Verify PROD_SERVICE_USER can access Podman
sudo -u PROD_SERVICE_USER podman --version
# Test that other users cannot access Podman socket
sudo -u PROD_DEPLOY_USER podman --version || echo "Good: PROD_DEPLOY_USER cannot access Podman"
``` ```
**What this does**: #### 12.4 Create Application Directory
- Creates dedicated `podman` group for controlled access
- Restricts Podman socket access to only members of the `podman` group
- Ensures only PROD_SERVICE_USER can execute Podman commands
- Prevents PROD_DEPLOY_USER and other users from running containers
- Maintains rootless operation for security
#### 11.4 Create Application Directory
```bash ```bash
# Create application directory for deployment # Create application directory for deployment
@ -1838,11 +1785,33 @@ ls -la /opt/APP_NAME
- Sets proper ownership for the PROD_SERVICE_USER - Sets proper ownership for the PROD_SERVICE_USER
- Ensures the directory exists before the CI workflow runs - Ensures the directory exists before the CI workflow runs
### Step 12: Set Up Forgejo Runner for Production Deployment ### Step 13: Configure Podman for Forgejo Container Registry Access
**Important**: The Production Linode needs to be able to pull images from the Forgejo Container Registry. Authentication is handled through the CI/CD pipeline configuration.
```bash
# Change to the PROD_SERVICE_USER
sudo su - PROD_SERVICE_USER
# Test Forgejo Container Registry access will be verified during deployment
# Images are pulled from the configured REGISTRY_HOST during the CI/CD process
# Change back to PROD_DEPLOY_USER
exit
```
**Note**: Production deployments pull images from Forgejo Container Registry using the configured authentication.
**What this does**:
- **Tests Forgejo Container Registry access**: Registry access is verified during the deployment process
- **Integrated authentication**: Forgejo handles authentication automatically
- **Simple setup**: No complex certificate management required
### Step 14: Set Up Forgejo Runner for Production Deployment
**Important**: The Production Linode needs a Forgejo runner to execute the deployment job from the CI/CD workflow. This runner will pull images from Forgejo Container Registry and deploy using the production pod configuration. **Important**: The Production Linode needs a Forgejo runner to execute the deployment job from the CI/CD workflow. This runner will pull images from Forgejo Container Registry and deploy using the production pod configuration.
#### 12.1 Download Runner #### 14.1 Download Runner
**Important**: Run this step as the **PROD_DEPLOY_USER** (not root or PROD_SERVICE_USER). The PROD_DEPLOY_USER handles deployment tasks including downloading and installing the Forgejo runner. **Important**: Run this step as the **PROD_DEPLOY_USER** (not root or PROD_SERVICE_USER). The PROD_DEPLOY_USER handles deployment tasks including downloading and installing the Forgejo runner.
@ -1879,14 +1848,14 @@ sudo mv forgejo-runner-${VERSION#v}-linux-amd64 /usr/bin/forgejo-runner
**Production Recommendation**: Use version pinning in production environments to ensure consistency and avoid unexpected breaking changes. **Production Recommendation**: Use version pinning in production environments to ensure consistency and avoid unexpected breaking changes.
#### 12.2 Get Registration Token #### 14.2 Get Registration Token
1. Go to your Forgejo repository 1. Go to your Forgejo repository
2. Navigate to **Settings → Actions → Runners** 2. Navigate to **Settings → Actions → Runners**
3. Click **"New runner"** 3. Click **"New runner"**
4. Copy the registration token 4. Copy the registration token
#### 12.3 Register the Production Runner #### 14.3 Register the Production Runner
**Step 1: Register the Runner** **Step 1: Register the Runner**
@ -1934,7 +1903,7 @@ sudo chmod 600 /etc/forgejo-runner/.runner
- Registers the runner with your Forgejo instance - Registers the runner with your Forgejo instance
- Sets up the runner with appropriate labels for production deployment - Sets up the runner with appropriate labels for production deployment
#### 12.4 Create Systemd Service #### 14.4 Create Systemd Service
```bash ```bash
# Create systemd service file # Create systemd service file
@ -1964,7 +1933,7 @@ sudo systemctl start forgejo-runner.service
sudo systemctl status forgejo-runner.service sudo systemctl status forgejo-runner.service
``` ```
#### 12.5 Test Runner Configuration #### 14.5 Test Runner Configuration
```bash ```bash
# Check if the runner is running # Check if the runner is running
@ -1998,7 +1967,7 @@ When the workflow runs, it will:
The production runner will automatically handle the deployment process when you push to the main branch. The production runner will automatically handle the deployment process when you push to the main branch.
#### 12.6 Understanding the Production Pod Setup #### 14.6 Understanding the Production Pod Setup
The `prod-pod.yaml` file is specifically designed for production deployment and uses Kubernetes pod specifications: The `prod-pod.yaml` file is specifically designed for production deployment and uses Kubernetes pod specifications:
@ -2023,22 +1992,115 @@ The `prod-pod.yaml` file is specifically designed for production deployment and
4. Waits for all services to be healthy 4. Waits for all services to be healthy
5. Verifies the deployment was successful 5. Verifies the deployment was successful
### Step 15: Configure Security
#### 15.1 Configure Firewall
### Step 13: Test Production Setup
#### 13.1 Test Podman Installation
```bash ```bash
# Test Podman installation (run as PROD_SERVICE_USER) sudo ufw --force enable
sudo -u PROD_SERVICE_USER podman --version sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
``` ```
#### 13.2 Test Forgejo Container Registry Access **Security Note**: We only allow ports 80 and 443 for external access. The application services (backend on 3001, frontend on 3000) are only accessible through the Nginx reverse proxy, which provides better security and SSL termination.
#### 15.2 Configure Fail2ban
**Fail2ban** is an intrusion prevention system that monitors logs and automatically blocks IP addresses showing malicious behavior.
```bash ```bash
# Test pulling an image from Forgejo Container Registry (run as PROD_SERVICE_USER) # Install fail2ban (if not already installed)
sudo -u PROD_SERVICE_USER podman pull YOUR_CI_CD_IP/APP_NAME/test:latest sudo apt install -y fail2ban
# Create a custom jail configuration
sudo tee /etc/fail2ban/jail.local > /dev/null << 'EOF'
[DEFAULT]
# Ban time in seconds (24 hours)
bantime = 86400
# Find time in seconds (10 minutes)
findtime = 600
# Max retries before ban
maxretry = 3
# Ban action (use ufw since we're using ufw firewall)
banaction = ufw
# Log level
loglevel = INFO
# Log target
logtarget = /var/log/fail2ban.log
# SSH protection
[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
# Note: Nginx protection is handled by the firewall and application-level security
# Docker containers are isolated, and Nginx logs are not directly accessible to fail2ban
# Web attack protection is provided by:
# 1. UFW firewall (ports 80/443 only)
# 2. Nginx security headers and rate limiting
# 3. Application-level input validation
EOF
# Enable and start fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
# Verify fail2ban is running
sudo systemctl status fail2ban
# Check current jails
sudo fail2ban-client status
```
**What this does**:
- **SSH Protection**: Blocks IPs that fail SSH login 3 times in 10 minutes
- **24-hour bans**: Banned IPs are blocked for 24 hours
- **Automatic monitoring**: Continuously watches SSH logs
**Web Security Note**: Since Nginx runs in a Docker container, web attack protection is handled by:
- **UFW Firewall**: Only allows ports 80/443 (no direct access to app services)
- **Nginx Security**: Built-in rate limiting and security headers
- **Application Security**: Input validation in the backend/frontend code
**Monitoring Fail2ban**:
```bash
# Check banned IPs
sudo fail2ban-client status sshd
# Unban an IP if needed
sudo fail2ban-client set sshd unbanip IP_ADDRESS
# View fail2ban logs
sudo tail -f /var/log/fail2ban.log
# Check all active jails
sudo fail2ban-client status
**Why This Matters for Production**:
- **Your server is exposed**: The Production Linode is accessible from the internet
- **Automated attacks**: Bots constantly scan for vulnerable servers
- **Resource protection**: Prevents attackers from consuming CPU/memory
- **Security layers**: Works with the firewall to provide defense in depth
### Step 16: Test Production Setup
#### 16.1 Test Podman Installation
```bash
podman --version
```
#### 16.2 Test Forgejo Container Registry Access
```bash
# Test pulling an image from Forgejo Container Registry
podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
``` ```
**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. **Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address.
@ -2051,12 +2113,17 @@ sudo -u PROD_SERVICE_USER podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
## Part 3: Final Configuration and Testing ## Part 3: Final Configuration and Testing
### Step 14: Configure Forgejo Repository Secrets ### Step 17: Configure Forgejo Repository Secrets
Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**: Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**:
**Required Secrets:** **Required Secrets:**
- `CI_HOST`: Your CI/CD Linode IP address (used for testing environment)
- `PRODUCTION_IP`: Your Production Linode IP address
- `PROD_DEPLOY_USER`: The production deployment user name (e.g., `prod-deploy`)
- `PROD_SERVICE_USER`: The production service user name (e.g., `prod-service`)
- `APP_NAME`: Your application name (e.g., `sharenet`) - `APP_NAME`: Your application name (e.g., `sharenet`)
- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database
- `REGISTRY_HOST`: Your Forgejo instance's registry URL - `REGISTRY_HOST`: Your Forgejo instance's registry URL
- `REGISTRY_USERNAME`: Your Forgejo username for registry authentication - `REGISTRY_USERNAME`: Your Forgejo username for registry authentication
- `REGISTRY_TOKEN`: Personal Access Token with `write:packages` scope for registry pushes - `REGISTRY_TOKEN`: Personal Access Token with `write:packages` scope for registry pushes
@ -2067,6 +2134,10 @@ Go to your Forgejo repository and add these secrets in **Settings → Secrets an
- `NODE_IMG_DIGEST`: Pinned Node.js image digest (e.g., `docker.io/library/node@sha256:...`) - `NODE_IMG_DIGEST`: Pinned Node.js image digest (e.g., `docker.io/library/node@sha256:...`)
- `POSTGRES_IMG_DIGEST`: Pinned PostgreSQL image digest (e.g., `docker.io/library/postgres@sha256:...`) - `POSTGRES_IMG_DIGEST`: Pinned PostgreSQL image digest (e.g., `docker.io/library/postgres@sha256:...`)
**Optional Secrets (for enhanced security):**
- `COSIGN_PRIVATE_KEY`: Private key for Cosign image signing
- `COSIGN_PASSWORD`: Password for Cosign private key
#### How to Obtain Each Secret (with Purpose and Commands): #### How to Obtain Each Secret (with Purpose and Commands):
**1. Image Digests (Security-Critical - Prevent Supply Chain Attacks):** **1. Image Digests (Security-Critical - Prevent Supply Chain Attacks):**
@ -2137,22 +2208,42 @@ Go to your Forgejo repository and add these secrets in **Settings → Secrets an
**4. Application Configuration:** **4. Application Configuration:**
- **`CI_HOST`**: Your CI/CD Linode IP address
*Purpose: Testing environment configuration*
- **`PRODUCTION_IP`**: Your Production Linode IP address
*Purpose: Deployment target*
- **`PROD_DEPLOY_USER`**: Production deployment username (e.g., `prod-deploy`)
*Purpose: SSH user for deployment operations*
- **`PROD_SERVICE_USER`**: Production service username (e.g., `prod-service`)
*Purpose: User that runs application services*
- **`APP_NAME`**: Your application name (e.g., `sharenet`) - **`APP_NAME`**: Your application name (e.g., `sharenet`)
*Purpose: Image naming and directory structure* *Purpose: Image naming and directory structure*
- **`POSTGRES_PASSWORD`**: Strong password for PostgreSQL database
```bash
# Generate secure password
openssl rand -base64 32
```
**Security Note**: All secrets are managed by Forgejo and never exposed in logs or environment variables. The ephemeral PiP approach ensures secrets are only used during execution and never persist. **Security Note**: All secrets are managed by Forgejo and never exposed in logs or environment variables. The ephemeral PiP approach ensures secrets are only used during execution and never persist.
**Note**: This setup uses custom Dockerfiles for testing environments with base images. The CI pipeline automatically checks if base images exist in Forgejo Container Registry and pulls them from Docker Hub only when needed, eliminating rate limiting issues and providing better control over the testing environment. **Note**: This setup uses custom Dockerfiles for testing environments with base images. The CI pipeline automatically checks if base images exist in Forgejo Container Registry and pulls them from Docker Hub only when needed, eliminating rate limiting issues and providing better control over the testing environment.
### Step 15: Test Complete Pipeline ### Step 18: Test Complete Pipeline
#### 15.1 Trigger a Test Build #### 18.1 Trigger a Test Build
1. **Make a small change** to your repository (e.g., update a comment or add a test file) 1. **Make a small change** to your repository (e.g., update a comment or add a test file)
2. **Commit and push** the changes to trigger the CI/CD pipeline 2. **Commit and push** the changes to trigger the CI/CD pipeline
3. **Monitor the build** in your Forgejo repository → Actions tab 3. **Monitor the build** in your Forgejo repository → Actions tab
#### 15.2 Verify Pipeline Steps #### 18.2 Verify Pipeline Steps
The pipeline should execute these steps in order: The pipeline should execute these steps in order:
@ -2165,7 +2256,7 @@ The pipeline should execute these steps in order:
7. **Push to Registry**: Push images to Forgejo Container Registry from DinD 7. **Push to Registry**: Push images to Forgejo Container Registry from DinD
8. **Deploy to Production**: Deploy to production server 8. **Deploy to Production**: Deploy to production server
#### 15.3 Check Forgejo Container Registry #### 18.3 Check Forgejo Container Registry
```bash ```bash
# On CI/CD Linode # On CI/CD Linode
@ -2186,7 +2277,7 @@ curl -k https://YOUR_CI_CD_IP:4443/v2/_catalog
# Expected: This should return authentication error without credentials # Expected: This should return authentication error without credentials
``` ```
#### 15.4 Verify Production Deployment #### 18.4 Verify Production Deployment
```bash ```bash
# On Production Linode # On Production Linode
@ -2204,16 +2295,16 @@ podman logs sharenet-production-pod-backend
podman logs sharenet-production-pod-frontend podman logs sharenet-production-pod-frontend
``` ```
#### 15.5 Test Application Functionality #### 18.5 Test Application Functionality
1. **Frontend**: Visit your production URL (IP address) 1. **Frontend**: Visit your production URL (IP address)
2. **Backend API**: Test API endpoints 2. **Backend API**: Test API endpoints
3. **Database**: Verify database connections 3. **Database**: Verify database connections
4. **Logs**: Check for any errors in application logs 4. **Logs**: Check for any errors in application logs
### Step 16: Final Verification ### Step 19: Final Verification
#### 16.1 Security Check #### 19.1 Security Check
```bash ```bash
# Check firewall status # Check firewall status
@ -2226,7 +2317,7 @@ sudo systemctl status fail2ban
sudo grep "PasswordAuthentication" /etc/ssh/sshd_config sudo grep "PasswordAuthentication" /etc/ssh/sshd_config
``` ```
#### 16.2 Performance Check #### 19.2 Performance Check
```bash ```bash
# Check system resources # Check system resources
@ -2239,7 +2330,7 @@ df -h
docker system df docker system df
``` ```
#### 16.3 Backup Verification #### 19.3 Backup Verification
```bash ```bash
# Test backup script # Test backup script
@ -2250,16 +2341,16 @@ cd /opt/APP_NAME
./scripts/backup.sh ./scripts/backup.sh
``` ```
### Step 17: Documentation and Maintenance ### Step 20: Documentation and Maintenance
#### 17.1 Update Documentation #### 20.1 Update Documentation
1. **Update README.md** with deployment information 1. **Update README.md** with deployment information
2. **Document environment variables** and their purposes 2. **Document environment variables** and their purposes
3. **Create troubleshooting guide** for common issues 3. **Create troubleshooting guide** for common issues
4. **Document backup and restore procedures** 4. **Document backup and restore procedures**
#### 17.2 Set Up Monitoring Alerts #### 20.2 Set Up Monitoring Alerts
```bash ```bash
# Set up monitoring cron job # Set up monitoring cron job
@ -2269,7 +2360,7 @@ cd /opt/APP_NAME
tail -f /tmp/monitor.log tail -f /tmp/monitor.log
``` ```
#### 17.3 Regular Maintenance Tasks #### 20.3 Regular Maintenance Tasks
**Daily:** **Daily:**
- Check application logs for errors - Check application logs for errors
@ -2301,20 +2392,15 @@ Before using this setup, ensure that:
### Repository Secrets ### Repository Secrets
The CI pipeline uses the following secrets. Only the required secrets are needed for current CI operations: The CI pipeline expects the following secrets to be configured in your Forgejo repository:
| Secret Name | Description | Example | Required | | Secret Name | Description | Example |
|-------------|-------------|---------|----------| |-------------|-------------|---------|
| `APP_NAME` | Your application name | `sharenet` | ✅ Yes | | `REGISTRY_HOST` | Your Forgejo instance hostname | `forgejo.example.com` |
| `REGISTRY_HOST` | Your Forgejo instance hostname | `forgejo.example.com` | ✅ Yes | | `REGISTRY_USERNAME` | Bot or owner account username | `ci-bot` |
| `REGISTRY_USERNAME` | Bot or owner account username | `ci-bot` | ✅ Yes | | `REGISTRY_TOKEN` | PAT with `write:packages` scope | `gto_...` |
| `REGISTRY_TOKEN` | PAT with `write:packages` scope | `gto_...` | ✅ Yes | | `COSIGN_PRIVATE_KEY` | (Optional) Private key for image signing | `-----BEGIN PRIVATE KEY-----...` |
| `SSH_PRIVATE_KEY` | SSH private key for deployment | `-----BEGIN OPENSSH PRIVATE KEY-----...` | ✅ Yes | | `COSIGN_PASSWORD` | (Optional) Password for Cosign private key | `your-password` |
| `SSH_KNOWN_HOSTS` | SSH known_hosts entry | `production-ip ecdsa-sha2-nistp256...` | ✅ Yes |
| `PODMAN_CLIENT_IMG_DIGEST` | Pinned Podman client image digest | `quay.io/podman/stable@sha256:...` | ✅ Yes |
| `RUST_IMG_DIGEST` | Pinned Rust image digest | `docker.io/library/rust@sha256:...` | ✅ Yes |
| `NODE_IMG_DIGEST` | Pinned Node.js image digest | `docker.io/library/node@sha256:...` | ✅ Yes |
| `POSTGRES_IMG_DIGEST` | Pinned PostgreSQL image digest | `docker.io/library/postgres@sha256:...` | ✅ Yes |
### How It Works ### How It Works
@ -2333,11 +2419,11 @@ The CI pipeline has been updated to:
Since the repository is public, applications can pull images anonymously: Since the repository is public, applications can pull images anonymously:
```bash ```bash
# Pull by tag (run as PROD_SERVICE_USER in production) # Pull by tag
sudo -u PROD_SERVICE_USER podman pull forgejo.example.com/owner/repo/backend:latest podman pull forgejo.example.com/owner/repo/backend:latest
# Pull by digest (run as PROD_SERVICE_USER in production) # Pull by digest
sudo -u PROD_SERVICE_USER podman pull forgejo.example.com/owner/repo/backend@sha256:abc123... podman pull forgejo.example.com/owner/repo/backend@sha256:abc123...
``` ```
#### Image Naming Convention #### Image Naming Convention
@ -2382,12 +2468,12 @@ For example:
# Test registry access # Test registry access
podman login REGISTRY_HOST -u REGISTRY_USERNAME -p REGISTRY_TOKEN podman login REGISTRY_HOST -u REGISTRY_USERNAME -p REGISTRY_TOKEN
# List available images (run as PROD_SERVICE_USER in production) # List available images
sudo -u PROD_SERVICE_USER podman search REGISTRY_HOST/OWNER_REPO podman search REGISTRY_HOST/OWNER_REPO
# Pull and verify image (run as PROD_SERVICE_USER in production) # Pull and verify image
sudo -u PROD_SERVICE_USER podman pull REGISTRY_HOST/OWNER_REPO/backend:TAG podman pull REGISTRY_HOST/OWNER_REPO/backend:TAG
sudo -u PROD_SERVICE_USER podman image inspect REGISTRY_HOST/OWNER_REPO/backend:TAG podman image inspect REGISTRY_HOST/OWNER_REPO/backend:TAG
``` ```
--- ---