Add Part 3 back in

CI_CD_PIPELINE_SETUP_GUIDE.md

@@ -1326,3 +1326,367 @@ curl http://localhost:3001/health
 ---
 
 ## Part 3: Final Configuration and Testing
+
+### Step 20: Configure Forgejo Repository Secrets
+
+#### 20.1 Required Repository Secrets
+
+Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**:
+
+**Required Secrets:**
+- `CI_CD_IP`: Your CI/CD Linode IP address
+- `PRODUCTION_IP`: Your Production Linode IP address
+- `DEPLOY_USER`: The deployment user name (e.g., `deploy`, `ci`, `admin`)
+- `SERVICE_USER`: The service user name (e.g., `appuser`, `service`, `app`)
+- `APP_NAME`: Your application name (e.g., `sharenet`, `myapp`)
+- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database
+
+**Optional Secrets (for domain users):**
+- `DOMAIN`: Your domain name (e.g., `example.com`)
+- `EMAIL`: Your email for SSL certificate notifications
+
+#### 20.2 Configure Forgejo Actions Runner
+
+##### 20.2.1 Get Runner Token
+
+1. Go to your Forgejo repository
+2. Navigate to **Settings → Actions → Runners**
+3. Click **"New runner"**
+4. Copy the registration token
+
+##### 20.2.2 Configure Runner
+
+```bash
+# Switch to DEPLOY_USER on CI/CD Linode
+sudo su - DEPLOY_USER
+
+# Get the registration token from your Forgejo repository
+# Go to Settings → Actions → Runners → New runner
+# Copy the registration token
+
+# Configure the runner
+forgejo-runner register \
+  --instance https://your-forgejo-instance \
+  --token YOUR_TOKEN \
+  --name "ci-cd-runner" \
+  --labels "ubuntu-latest,docker" \
+  --no-interactive
+```
+
+##### 20.2.3 Start Runner
+
+```bash
+sudo systemctl start forgejo-runner.service
+sudo systemctl status forgejo-runner.service
+```
+
+##### 20.2.4 Test Runner Configuration
+
+```bash
+# Check if the runner is running
+sudo systemctl status forgejo-runner.service
+
+# Check runner logs
+sudo journalctl -u forgejo-runner.service -f --no-pager
+
+# Test runner connectivity (in a separate terminal)
+forgejo-runner list
+
+# Verify runner appears in Forgejo
+# Go to your Forgejo repository → Settings → Actions → Runners
+# You should see your runner listed as "ci-cd-runner" with status "Online"
+```
+
+**Expected Output**:
+- `systemctl status` should show "active (running)"
+- `forgejo-runner list` should show your runner
+- Forgejo web interface should show the runner as online
+
+**If something goes wrong**:
+- Check logs: `sudo journalctl -u forgejo-runner.service -f`
+- Verify token: Make sure the registration token is correct
+- Check network: Ensure the runner can reach your Forgejo instance
+- Restart service: `sudo systemctl restart forgejo-runner.service`
+
+### Step 21: Set Up Monitoring and Cleanup
+
+#### 21.1 Monitoring Script
+
+**Important**: The repository includes a pre-configured monitoring script in the `scripts/` directory that can be used for both CI/CD and production monitoring.
+
+**Repository Script**:
+- `scripts/monitor.sh` - Comprehensive monitoring script with support for both CI/CD and production environments
+
+**To use the repository monitoring script**:
+```bash
+# The repository is already cloned at /opt/APP_NAME/
+cd /opt/APP_NAME
+
+# Make the script executable
+chmod +x scripts/monitor.sh
+
+# Test CI/CD monitoring
+./scripts/monitor.sh --type ci-cd
+
+# Test production monitoring (if you have a production setup)
+./scripts/monitor.sh --type production
+```
+
+**Note**: The repository script is more comprehensive and includes proper error handling, colored output, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate monitoring information.
+
+#### 21.2 Cleanup Script
+
+**Important**: The repository includes a pre-configured cleanup script in the `scripts/` directory that can be used for both CI/CD and production cleanup operations.
+
+**Repository Script**:
+- `scripts/cleanup.sh` - Comprehensive cleanup script with support for both CI/CD and production environments
+
+**To use the repository cleanup script**:
+```bash
+# The repository is already cloned at /opt/APP_NAME/
+cd /opt/APP_NAME
+
+# Make the script executable
+chmod +x scripts/cleanup.sh
+
+# Test CI/CD cleanup (dry run first)
+./scripts/cleanup.sh --type ci-cd --dry-run
+
+# Run CI/CD cleanup
+./scripts/cleanup.sh --type ci-cd
+
+# Test production cleanup (dry run first)
+./scripts/cleanup.sh --type production --dry-run
+```
+
+**Note**: The repository script is more comprehensive and includes proper error handling, colored output, dry-run mode, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate cleanup operations.
+
+#### 21.3 Test Cleanup Script
+
+```bash
+# Create some test images to clean up
+docker pull alpine:latest
+docker pull nginx:latest
+docker tag alpine:latest test-cleanup:latest
+docker tag nginx:latest test-cleanup2:latest
+
+# Test cleanup with dry run first
+./scripts/cleanup.sh --type ci-cd --dry-run
+
+# Run the cleanup script
+./scripts/cleanup.sh --type ci-cd
+
+# Verify cleanup worked
+echo "Checking remaining images:"
+docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"
+
+echo "Checking remaining volumes:"
+docker volume ls
+
+echo "Checking remaining networks:"
+docker network ls
+```
+
+**Expected Output**:
+- Cleanup script should run without errors
+- Test images should be removed
+- System should report cleanup completion
+- Remaining images should be minimal (only actively used ones)
+
+**If something goes wrong**:
+- Check script permissions: `ls -la scripts/cleanup.sh`
+- Verify Docker access: `docker ps`
+- Check registry access: `cd /opt/APP_NAME/registry && docker compose ps`
+- Run manually: `bash -x scripts/cleanup.sh`
+
+#### 21.4 Set Up Automated Cleanup
+
+```bash
+# Create a cron job to run cleanup daily at 3 AM using the repository script
+(crontab -l 2>/dev/null; echo "0 3 * * * cd /opt/APP_NAME && ./scripts/cleanup.sh --type ci-cd >> /tmp/cleanup.log 2>&1") | crontab -
+
+# Verify the cron job was added
+crontab -l
+```
+
+**What this does:**
+- **Runs automatically**: The cleanup script runs every day at 3:00 AM
+- **Frequency**: Daily cleanup to prevent disk space issues
+- **Logging**: All cleanup output is logged to `/tmp/cleanup.log`
+- **What it cleans**: Unused Docker images, volumes, networks, and registry images
+
+### Step 22: Test Complete Pipeline
+
+#### 22.1 Trigger a Test Build
+
+1. **Make a small change** to your repository (e.g., update a comment or add a test file)
+2. **Commit and push** the changes to trigger the CI/CD pipeline
+3. **Monitor the build** in your Forgejo repository → Actions tab
+
+#### 22.2 Verify Pipeline Steps
+
+The pipeline should execute these steps in order:
+
+1. **Checkout**: Clone the repository
+2. **Test Backend**: Run backend tests
+3. **Test Frontend**: Run frontend tests
+4. **Build Backend**: Build backend Docker image
+5. **Build Frontend**: Build frontend Docker image
+6. **Push to Registry**: Push images to your private registry
+7. **Deploy to Production**: Deploy to production server
+
+#### 22.3 Check Registry
+
+```bash
+# On CI/CD Linode
+cd /opt/APP_NAME/registry
+
+# Check if new images were pushed
+curl -k https://localhost:8080/v2/_catalog
+
+# Check specific repository tags
+curl -k https://localhost:8080/v2/APP_NAME/backend/tags/list
+curl -k https://localhost:8080/v2/APP_NAME/frontend/tags/list
+```
+
+#### 22.4 Verify Production Deployment
+
+```bash
+# On Production Linode
+cd /opt/APP_NAME
+
+# Check if containers are running with new images
+docker compose ps
+
+# Check application health
+curl http://localhost:3000
+curl http://localhost:3001/health
+
+# Check container logs for any errors
+docker compose logs backend
+docker compose logs frontend
+```
+
+#### 22.5 Test Application Functionality
+
+1. **Frontend**: Visit your production URL (IP or domain)
+2. **Backend API**: Test API endpoints
+3. **Database**: Verify database connections
+4. **Logs**: Check for any errors in application logs
+
+### Step 23: Set Up SSL/TLS (Optional - Domain Users)
+
+#### 23.1 Install SSL Certificate
+
+If you have a domain pointing to your Production Linode:
+
+```bash
+# On Production Linode
+sudo certbot --nginx -d your-domain.com
+
+# Verify certificate
+sudo certbot certificates
+```
+
+#### 23.2 Configure Auto-Renewal
+
+```bash
+# Test auto-renewal
+sudo certbot renew --dry-run
+
+# Add to crontab for automatic renewal
+sudo crontab -e
+# Add this line:
+# 0 12 * * * /usr/bin/certbot renew --quiet
+```
+
+### Step 24: Final Verification
+
+#### 24.1 Security Check
+
+```bash
+# Check firewall status
+sudo ufw status
+
+# Check fail2ban status
+sudo systemctl status fail2ban
+
+# Check SSH access (should be key-based only)
+sudo grep "PasswordAuthentication" /etc/ssh/sshd_config
+```
+
+#### 24.2 Performance Check
+
+```bash
+# Check system resources
+htop
+
+# Check disk usage
+df -h
+
+# Check Docker disk usage
+docker system df
+```
+
+#### 24.3 Backup Verification
+
+```bash
+# Test backup script
+cd /opt/APP_NAME
+./scripts/backup.sh --dry-run
+
+# Run actual backup
+./scripts/backup.sh
+```
+
+### Step 25: Documentation and Maintenance
+
+#### 25.1 Update Documentation
+
+1. **Update README.md** with deployment information
+2. **Document environment variables** and their purposes
+3. **Create troubleshooting guide** for common issues
+4. **Document backup and restore procedures**
+
+#### 25.2 Set Up Monitoring Alerts
+
+```bash
+# Set up monitoring cron job
+(crontab -l 2>/dev/null; echo "*/5 * * * * cd /opt/APP_NAME && ./scripts/monitor.sh --type production >> /tmp/monitor.log 2>&1") | crontab -
+
+# Check monitoring logs
+tail -f /tmp/monitor.log
+```
+
+#### 25.3 Regular Maintenance Tasks
+
+**Daily:**
+- Check application logs for errors
+- Monitor system resources
+- Verify backup completion
+
+**Weekly:**
+- Review security logs
+- Update system packages
+- Test backup restoration
+
+**Monthly:**
+- Review and rotate logs
+- Update SSL certificates
+- Review and update documentation
+
+---
+
+## 🎉 Congratulations!
+
+You have successfully set up a complete CI/CD pipeline with:
+
+- ✅ **Automated testing** on every code push
+- ✅ **Docker image building** and registry storage
+- ✅ **Automated deployment** to production
+- ✅ **Health monitoring** and logging
+- ✅ **Backup and cleanup** automation
+- ✅ **Security hardening** with proper user separation
+- ✅ **SSL/TLS support** for production (optional)
+
+Your application is now ready for continuous deployment with proper security, monitoring, and maintenance procedures in place!