diff --git a/CI_CD_PIPELINE_SETUP_GUIDE.md b/CI_CD_PIPELINE_SETUP_GUIDE.md index e951b0d..f4f748b 100644 --- a/CI_CD_PIPELINE_SETUP_GUIDE.md +++ b/CI_CD_PIPELINE_SETUP_GUIDE.md @@ -1325,4 +1325,368 @@ curl http://localhost:3001/health --- -## Part 3: Final Configuration and Testing \ No newline at end of file +## Part 3: Final Configuration and Testing + +### Step 20: Configure Forgejo Repository Secrets + +#### 20.1 Required Repository Secrets + +Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**: + +**Required Secrets:** +- `CI_CD_IP`: Your CI/CD Linode IP address +- `PRODUCTION_IP`: Your Production Linode IP address +- `DEPLOY_USER`: The deployment user name (e.g., `deploy`, `ci`, `admin`) +- `SERVICE_USER`: The service user name (e.g., `appuser`, `service`, `app`) +- `APP_NAME`: Your application name (e.g., `sharenet`, `myapp`) +- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database + +**Optional Secrets (for domain users):** +- `DOMAIN`: Your domain name (e.g., `example.com`) +- `EMAIL`: Your email for SSL certificate notifications + +#### 20.2 Configure Forgejo Actions Runner + +##### 20.2.1 Get Runner Token + +1. Go to your Forgejo repository +2. Navigate to **Settings → Actions → Runners** +3. Click **"New runner"** +4. Copy the registration token + +##### 20.2.2 Configure Runner + +```bash +# Switch to DEPLOY_USER on CI/CD Linode +sudo su - DEPLOY_USER + +# Get the registration token from your Forgejo repository +# Go to Settings → Actions → Runners → New runner +# Copy the registration token + +# Configure the runner +forgejo-runner register \ + --instance https://your-forgejo-instance \ + --token YOUR_TOKEN \ + --name "ci-cd-runner" \ + --labels "ubuntu-latest,docker" \ + --no-interactive +``` + +##### 20.2.3 Start Runner + +```bash +sudo systemctl start forgejo-runner.service +sudo systemctl status forgejo-runner.service +``` + +##### 20.2.4 Test Runner Configuration + +```bash +# Check if the runner is running +sudo systemctl status forgejo-runner.service + +# Check runner logs +sudo journalctl -u forgejo-runner.service -f --no-pager + +# Test runner connectivity (in a separate terminal) +forgejo-runner list + +# Verify runner appears in Forgejo +# Go to your Forgejo repository → Settings → Actions → Runners +# You should see your runner listed as "ci-cd-runner" with status "Online" +``` + +**Expected Output**: +- `systemctl status` should show "active (running)" +- `forgejo-runner list` should show your runner +- Forgejo web interface should show the runner as online + +**If something goes wrong**: +- Check logs: `sudo journalctl -u forgejo-runner.service -f` +- Verify token: Make sure the registration token is correct +- Check network: Ensure the runner can reach your Forgejo instance +- Restart service: `sudo systemctl restart forgejo-runner.service` + +### Step 21: Set Up Monitoring and Cleanup + +#### 21.1 Monitoring Script + +**Important**: The repository includes a pre-configured monitoring script in the `scripts/` directory that can be used for both CI/CD and production monitoring. + +**Repository Script**: +- `scripts/monitor.sh` - Comprehensive monitoring script with support for both CI/CD and production environments + +**To use the repository monitoring script**: +```bash +# The repository is already cloned at /opt/APP_NAME/ +cd /opt/APP_NAME + +# Make the script executable +chmod +x scripts/monitor.sh + +# Test CI/CD monitoring +./scripts/monitor.sh --type ci-cd + +# Test production monitoring (if you have a production setup) +./scripts/monitor.sh --type production +``` + +**Note**: The repository script is more comprehensive and includes proper error handling, colored output, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate monitoring information. + +#### 21.2 Cleanup Script + +**Important**: The repository includes a pre-configured cleanup script in the `scripts/` directory that can be used for both CI/CD and production cleanup operations. + +**Repository Script**: +- `scripts/cleanup.sh` - Comprehensive cleanup script with support for both CI/CD and production environments + +**To use the repository cleanup script**: +```bash +# The repository is already cloned at /opt/APP_NAME/ +cd /opt/APP_NAME + +# Make the script executable +chmod +x scripts/cleanup.sh + +# Test CI/CD cleanup (dry run first) +./scripts/cleanup.sh --type ci-cd --dry-run + +# Run CI/CD cleanup +./scripts/cleanup.sh --type ci-cd + +# Test production cleanup (dry run first) +./scripts/cleanup.sh --type production --dry-run +``` + +**Note**: The repository script is more comprehensive and includes proper error handling, colored output, dry-run mode, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate cleanup operations. + +#### 21.3 Test Cleanup Script + +```bash +# Create some test images to clean up +docker pull alpine:latest +docker pull nginx:latest +docker tag alpine:latest test-cleanup:latest +docker tag nginx:latest test-cleanup2:latest + +# Test cleanup with dry run first +./scripts/cleanup.sh --type ci-cd --dry-run + +# Run the cleanup script +./scripts/cleanup.sh --type ci-cd + +# Verify cleanup worked +echo "Checking remaining images:" +docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" + +echo "Checking remaining volumes:" +docker volume ls + +echo "Checking remaining networks:" +docker network ls +``` + +**Expected Output**: +- Cleanup script should run without errors +- Test images should be removed +- System should report cleanup completion +- Remaining images should be minimal (only actively used ones) + +**If something goes wrong**: +- Check script permissions: `ls -la scripts/cleanup.sh` +- Verify Docker access: `docker ps` +- Check registry access: `cd /opt/APP_NAME/registry && docker compose ps` +- Run manually: `bash -x scripts/cleanup.sh` + +#### 21.4 Set Up Automated Cleanup + +```bash +# Create a cron job to run cleanup daily at 3 AM using the repository script +(crontab -l 2>/dev/null; echo "0 3 * * * cd /opt/APP_NAME && ./scripts/cleanup.sh --type ci-cd >> /tmp/cleanup.log 2>&1") | crontab - + +# Verify the cron job was added +crontab -l +``` + +**What this does:** +- **Runs automatically**: The cleanup script runs every day at 3:00 AM +- **Frequency**: Daily cleanup to prevent disk space issues +- **Logging**: All cleanup output is logged to `/tmp/cleanup.log` +- **What it cleans**: Unused Docker images, volumes, networks, and registry images + +### Step 22: Test Complete Pipeline + +#### 22.1 Trigger a Test Build + +1. **Make a small change** to your repository (e.g., update a comment or add a test file) +2. **Commit and push** the changes to trigger the CI/CD pipeline +3. **Monitor the build** in your Forgejo repository → Actions tab + +#### 22.2 Verify Pipeline Steps + +The pipeline should execute these steps in order: + +1. **Checkout**: Clone the repository +2. **Test Backend**: Run backend tests +3. **Test Frontend**: Run frontend tests +4. **Build Backend**: Build backend Docker image +5. **Build Frontend**: Build frontend Docker image +6. **Push to Registry**: Push images to your private registry +7. **Deploy to Production**: Deploy to production server + +#### 22.3 Check Registry + +```bash +# On CI/CD Linode +cd /opt/APP_NAME/registry + +# Check if new images were pushed +curl -k https://localhost:8080/v2/_catalog + +# Check specific repository tags +curl -k https://localhost:8080/v2/APP_NAME/backend/tags/list +curl -k https://localhost:8080/v2/APP_NAME/frontend/tags/list +``` + +#### 22.4 Verify Production Deployment + +```bash +# On Production Linode +cd /opt/APP_NAME + +# Check if containers are running with new images +docker compose ps + +# Check application health +curl http://localhost:3000 +curl http://localhost:3001/health + +# Check container logs for any errors +docker compose logs backend +docker compose logs frontend +``` + +#### 22.5 Test Application Functionality + +1. **Frontend**: Visit your production URL (IP or domain) +2. **Backend API**: Test API endpoints +3. **Database**: Verify database connections +4. **Logs**: Check for any errors in application logs + +### Step 23: Set Up SSL/TLS (Optional - Domain Users) + +#### 23.1 Install SSL Certificate + +If you have a domain pointing to your Production Linode: + +```bash +# On Production Linode +sudo certbot --nginx -d your-domain.com + +# Verify certificate +sudo certbot certificates +``` + +#### 23.2 Configure Auto-Renewal + +```bash +# Test auto-renewal +sudo certbot renew --dry-run + +# Add to crontab for automatic renewal +sudo crontab -e +# Add this line: +# 0 12 * * * /usr/bin/certbot renew --quiet +``` + +### Step 24: Final Verification + +#### 24.1 Security Check + +```bash +# Check firewall status +sudo ufw status + +# Check fail2ban status +sudo systemctl status fail2ban + +# Check SSH access (should be key-based only) +sudo grep "PasswordAuthentication" /etc/ssh/sshd_config +``` + +#### 24.2 Performance Check + +```bash +# Check system resources +htop + +# Check disk usage +df -h + +# Check Docker disk usage +docker system df +``` + +#### 24.3 Backup Verification + +```bash +# Test backup script +cd /opt/APP_NAME +./scripts/backup.sh --dry-run + +# Run actual backup +./scripts/backup.sh +``` + +### Step 25: Documentation and Maintenance + +#### 25.1 Update Documentation + +1. **Update README.md** with deployment information +2. **Document environment variables** and their purposes +3. **Create troubleshooting guide** for common issues +4. **Document backup and restore procedures** + +#### 25.2 Set Up Monitoring Alerts + +```bash +# Set up monitoring cron job +(crontab -l 2>/dev/null; echo "*/5 * * * * cd /opt/APP_NAME && ./scripts/monitor.sh --type production >> /tmp/monitor.log 2>&1") | crontab - + +# Check monitoring logs +tail -f /tmp/monitor.log +``` + +#### 25.3 Regular Maintenance Tasks + +**Daily:** +- Check application logs for errors +- Monitor system resources +- Verify backup completion + +**Weekly:** +- Review security logs +- Update system packages +- Test backup restoration + +**Monthly:** +- Review and rotate logs +- Update SSL certificates +- Review and update documentation + +--- + +## 🎉 Congratulations! + +You have successfully set up a complete CI/CD pipeline with: + +- ✅ **Automated testing** on every code push +- ✅ **Docker image building** and registry storage +- ✅ **Automated deployment** to production +- ✅ **Health monitoring** and logging +- ✅ **Backup and cleanup** automation +- ✅ **Security hardening** with proper user separation +- ✅ **SSL/TLS support** for production (optional) + +Your application is now ready for continuous deployment with proper security, monitoring, and maintenance procedures in place! \ No newline at end of file