diff --git a/CI_CD_PIPELINE_SETUP_GUIDE.md b/CI_CD_PIPELINE_SETUP_GUIDE.md index f4f748b..717a381 100644 --- a/CI_CD_PIPELINE_SETUP_GUIDE.md +++ b/CI_CD_PIPELINE_SETUP_GUIDE.md @@ -406,110 +406,9 @@ pwd exit ``` -### Step 3: Install Docker +### Step 3: Clone Repository for Registry Configuration -#### 3.1 Add Docker Repository - -```bash -curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg -echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null -sudo apt update -``` - -#### 3.2 Install Docker Packages - -```bash -sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin -``` - -#### 3.3 Configure Docker for Service Account - -```bash -sudo usermod -aG docker SERVICE_USER -``` - -### Step 4: Set Up Docker Registry - -#### 4.1 Create Registry Directory - -```bash -sudo mkdir -p /opt/registry -sudo chown SERVICE_USER:SERVICE_USER /opt/registry -``` - -#### 4.2 Create Registry Configuration - -```bash -# Switch to SERVICE_USER (registry directory owner) -sudo su - SERVICE_USER - -# The registry configuration file is already available in the repository -# at /opt/APP_NAME/registry/config.yml -# No manual configuration is needed - we'll use the file directly from the repository - -# Exit SERVICE_USER shell -exit -``` - -**What this configuration does:** -- **HTTPS Enabled**: Uses TLS certificates for secure communication -- **Public Read Access**: Anyone can pull images without authentication -- **Authenticated Push**: Only authenticated users can push images -- **Security Headers**: Protects against common web vulnerabilities -- **CORS Headers**: Allows the registry UI to access the registry API with all necessary headers -- **Delete Enabled**: Allows deletion of images for cleanup operations - -**Security Note**: The configuration file is version controlled in the repository and will be used directly by the registry container. - -#### 4.2.1 Generate SSL Certificates - -```bash -# Switch to SERVICE_USER (registry directory owner) -sudo su - SERVICE_USER - -# Create system SSL directory for registry certificates -sudo mkdir -p /etc/ssl/registry - -# Get your actual IP address -YOUR_ACTUAL_IP=$(curl -4 -s ifconfig.me) -echo "Your IP address is: $YOUR_ACTUAL_IP" - -# Generate self-signed certificate with actual IP in system directory -sudo openssl req -x509 -newkey rsa:4096 -keyout /etc/ssl/registry/registry.key -out /etc/ssl/registry/registry.crt -days 365 -nodes -subj "/C=US/ST=State/L=City/O=Organization/CN=$YOUR_ACTUAL_IP" - -# Set proper permissions -sudo chmod 600 /etc/ssl/registry/registry.key -sudo chmod 644 /etc/ssl/registry/registry.crt - -# Exit SERVICE_USER shell -exit -``` - -**Important**: The certificate is now generated in the system SSL directory `/etc/ssl/registry/` with your actual CI/CD Linode IP address automatically. - -#### 4.3 Create Authentication File - -```bash -# Switch to SERVICE_USER (registry directory owner) -sudo su - SERVICE_USER - -# Create system auth directory for registry authentication -sudo mkdir -p /etc/registry/auth - -# Create htpasswd file for authentication (required for push operations only) -sudo htpasswd -Bbn push-user "$(openssl rand -base64 32)" > /tmp/auth.htpasswd -sudo mv /tmp/auth.htpasswd /etc/registry/auth/auth.htpasswd - -# Exit SERVICE_USER shell -exit -``` - -**What this does**: Creates user credentials for registry authentication in the system auth directory. -- `push-user`: Can push images (used by CI/CD pipeline for deployments) - -**Note**: Pull operations are public and don't require authentication, but push operations require these credentials. - -#### 4.3.1 Clone Repository for Registry Configuration +#### 3.1 Clone Repository for Registry Configuration ```bash # Switch to DEPLOY_USER (who has sudo access) @@ -536,7 +435,86 @@ exit - SERVICE_USER owns all the files for security - Registry configuration files are now available at `/opt/APP_NAME/registry/` -#### 4.4 Create Docker Compose for Registry +### Step 4: Install Docker + +#### 4.1 Add Docker Repository + +```bash +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +sudo apt update +``` + +#### 4.2 Install Docker Packages + +```bash +sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin +``` + +#### 4.3 Configure Docker for Service Account + +```bash +sudo usermod -aG docker SERVICE_USER +``` + +### Step 5: Set Up Docker Registry + +#### 5.1 Create Registry Directory + +```bash +sudo mkdir -p /opt/registry +sudo chown SERVICE_USER:SERVICE_USER /opt/registry +``` + +#### 5.2 Generate SSL Certificates + +```bash +# Switch to SERVICE_USER (registry directory owner) +sudo su - SERVICE_USER + +# Create system SSL directory for registry certificates +sudo mkdir -p /etc/ssl/registry + +# Get your actual IP address +YOUR_ACTUAL_IP=$(curl -4 -s ifconfig.me) +echo "Your IP address is: $YOUR_ACTUAL_IP" + +# Generate self-signed certificate with actual IP in system directory +sudo openssl req -x509 -newkey rsa:4096 -keyout /etc/ssl/registry/registry.key -out /etc/ssl/registry/registry.crt -days 365 -nodes -subj "/C=US/ST=State/L=City/O=Organization/CN=$YOUR_ACTUAL_IP" + +# Set proper permissions +sudo chmod 600 /etc/ssl/registry/registry.key +sudo chmod 644 /etc/ssl/registry/registry.crt + +# Exit SERVICE_USER shell +exit +``` + +**Important**: The certificate is now generated in the system SSL directory `/etc/ssl/registry/` with your actual CI/CD Linode IP address automatically. + +#### 5.3 Create Authentication File + +```bash +# Switch to SERVICE_USER (registry directory owner) +sudo su - SERVICE_USER + +# Create system auth directory for registry authentication +sudo mkdir -p /etc/registry/auth + +# Create htpasswd file for authentication (required for push operations only) +sudo htpasswd -Bbn push-user "$(openssl rand -base64 32)" > /tmp/auth.htpasswd +sudo mv /tmp/auth.htpasswd /etc/registry/auth/auth.htpasswd + +# Exit SERVICE_USER shell +exit +``` + +**What this does**: Creates user credentials for registry authentication in the system auth directory. +- `push-user`: Can push images (used by CI/CD pipeline for deployments) + +**Note**: Pull operations are public and don't require authentication, but push operations require these credentials. + +#### 5.4 Create Docker Compose for Registry ```bash # Switch to SERVICE_USER (registry directory owner) @@ -550,9 +528,9 @@ sudo su - SERVICE_USER exit ``` -**Important**: The repository should be cloned in the previous step (4.3.1) to `/opt/APP_NAME/`. The registry configuration files are used directly from the repository. +**Important**: The repository should be cloned in the previous step (3.1) to `/opt/APP_NAME/`. The registry configuration files are used directly from the repository. -#### 4.4.1 Update Configuration with Actual IP Address +#### 5.5 Update Configuration with Actual IP Address ```bash # Switch to SERVICE_USER (registry directory owner) @@ -574,14 +552,14 @@ exit **Important**: This step replaces all instances of `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address in both the docker-compose.yml and nginx.conf files in the repository. -#### 4.5 Install Required Tools +#### 5.6 Install Required Tools ```bash # Install htpasswd utility sudo apt install -y apache2-utils ``` -#### 4.6 Start Registry +#### 5.7 Start Registry ```bash # Switch to SERVICE_USER (registry directory owner) @@ -594,7 +572,7 @@ docker compose up -d exit ``` -#### 4.7 Test Registry Setup +#### 5.8 Test Registry Setup ```bash # Check if containers are running @@ -675,9 +653,9 @@ exit - Check Docker daemon config: `cat /etc/docker/daemon.json` - Restart registry: `docker compose restart` -### Step 5: Configure Docker for Registry Access +### Step 6: Configure Docker for Registry Access -#### 5.1 Configure Docker for Registry Access +#### 6.1 Configure Docker for Registry Access ```bash # Get the push user credentials @@ -703,7 +681,7 @@ EOF **Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. -#### 5.2 Restart Docker +#### 6.2 Restart Docker ```bash sudo systemctl restart docker @@ -745,15 +723,15 @@ For other machines to pull images, they only need: # No authentication needed for pulls ``` -### Step 6: Set Up SSH for Production Communication +### Step 7: Set Up SSH for Production Communication -#### 6.1 Generate SSH Key Pair +#### 7.1 Generate SSH Key Pair ```bash ssh-keygen -t ed25519 -C "ci-cd-server" -f ~/.ssh/id_ed25519 -N "" ``` -#### 6.2 Create SSH Config +#### 7.2 Create SSH Config ```bash cat > ~/.ssh/config << 'EOF' @@ -768,9 +746,9 @@ EOF chmod 600 ~/.ssh/config ``` -### Step 7: Install Forgejo Actions Runner +### Step 8: Install Forgejo Actions Runner -#### 7.1 Download Runner +#### 8.1 Download Runner ```bash cd ~ @@ -779,7 +757,7 @@ chmod +x forgejo-runner-0.2.11-linux-amd64 sudo mv forgejo-runner-0.2.11-linux-amd64 /usr/local/bin/forgejo-runner ``` -#### 7.2 Create Systemd Service +#### 8.2 Create Systemd Service ```bash sudo tee /etc/systemd/system/forgejo-runner.service > /dev/null << 'EOF' @@ -800,14 +778,14 @@ WantedBy=multi-user.target EOF ``` -#### 7.3 Enable Service +#### 8.3 Enable Service ```bash sudo systemctl daemon-reload sudo systemctl enable forgejo-runner.service ``` -#### 7.4 Test Runner Configuration +#### 8.4 Test Runner Configuration ```bash # Check if the runner is running @@ -835,582 +813,9 @@ forgejo-runner list - Check network: Ensure the runner can reach your Forgejo instance - Restart service: `sudo systemctl restart forgejo-runner.service` -### Step 8: Set Up Monitoring and Cleanup +### Step 9: Set Up Monitoring and Cleanup -#### 8.1 Monitoring Script - -**Important**: The repository includes a pre-configured monitoring script in the `scripts/` directory that can be used for both CI/CD and production monitoring. - -**Repository Script**: -- `scripts/monitor.sh` - Comprehensive monitoring script with support for both CI/CD and production environments - -**To use the repository monitoring script**: -```bash -# Clone the repository if not already done -git clone https://your-forgejo-instance/your-username/APP_NAME.git /tmp/monitoring-setup -cd /tmp/monitoring-setup - -# Make the script executable -chmod +x scripts/monitor.sh - -# Test CI/CD monitoring -./scripts/monitor.sh --type ci-cd - -# Test production monitoring (if you have a production setup) -./scripts/monitor.sh --type production - -# Clean up -cd / -rm -rf /tmp/monitoring-setup -``` - -**Alternative: Create a local copy for convenience**: -```bash -# Copy the script to your home directory for easy access -cp /tmp/monitoring-setup/scripts/monitor.sh ~/monitor.sh -chmod +x ~/monitor.sh - -# Test the local copy -~/monitor.sh --type ci-cd -``` - -**Note**: The repository script is more comprehensive and includes proper error handling, colored output, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate monitoring information. - -#### 8.2 Cleanup Script - -**Important**: The repository includes a pre-configured cleanup script in the `scripts/` directory that can be used for both CI/CD and production cleanup operations. - -**Repository Script**: -- `scripts/cleanup.sh` - Comprehensive cleanup script with support for both CI/CD and production environments - -**To use the repository cleanup script**: -```bash -# Clone the repository if not already done -git clone https://your-forgejo-instance/your-username/APP_NAME.git /tmp/cleanup-setup -cd /tmp/cleanup-setup - -# Make the script executable -chmod +x scripts/cleanup.sh - -# Test CI/CD cleanup (dry run first) -./scripts/cleanup.sh --type ci-cd --dry-run - -# Run CI/CD cleanup -./scripts/cleanup.sh --type ci-cd - -# Test production cleanup (dry run first) -./scripts/cleanup.sh --type production --dry-run - -# Clean up -cd / -rm -rf /tmp/cleanup-setup -``` - -**Alternative: Create a local copy for convenience**: -```bash -# Copy the script to your home directory for easy access -cp /tmp/cleanup-setup/scripts/cleanup.sh ~/cleanup.sh -chmod +x ~/cleanup.sh - -# Test the local copy (dry run) -~/cleanup.sh --type ci-cd --dry-run -``` - -**Note**: The repository script is more comprehensive and includes proper error handling, colored output, dry-run mode, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate cleanup operations. - -#### 8.3 Test Cleanup Script - -```bash -# Create some test images to clean up -docker pull alpine:latest -docker pull nginx:latest -docker tag alpine:latest test-cleanup:latest -docker tag nginx:latest test-cleanup2:latest - -# Test cleanup with dry run first -./scripts/cleanup.sh --type ci-cd --dry-run - -# Run the cleanup script -./scripts/cleanup.sh --type ci-cd - -# Verify cleanup worked -echo "Checking remaining images:" -docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" - -echo "Checking remaining volumes:" -docker volume ls - -echo "Checking remaining networks:" -docker network ls -``` - -**Expected Output**: -- Cleanup script should run without errors -- Test images should be removed -- System should report cleanup completion -- Remaining images should be minimal (only actively used ones) - -**If something goes wrong**: -- Check script permissions: `ls -la scripts/cleanup.sh` -- Verify Docker access: `docker ps` -- Check registry access: `cd /opt/registry && docker compose ps` -- Run manually: `bash -x scripts/cleanup.sh` - -#### 8.4 Set Up Automated Cleanup - -```bash -# Create a cron job to run cleanup daily at 3 AM using the repository script -(crontab -l 2>/dev/null; echo "0 3 * * * cd /tmp/cleanup-setup && ./scripts/cleanup.sh --type ci-cd >> /tmp/cleanup.log 2>&1") | crontab - - -# Verify the cron job was added -crontab -l -``` - -**What this does:** -- **Runs automatically**: The cleanup script runs every day at 3:00 AM -- **Frequency**: Daily cleanup to prevent disk space issues -- **Logging**: All cleanup output is logged to `/tmp/cleanup.log` -- **What it cleans**: Unused Docker images, volumes, networks, and registry images - -**Alternative: Use a local copy for automated cleanup**: -```bash -# If you created a local copy, use that instead -(crontab -l 2>/dev/null; echo "0 3 * * * ~/cleanup.sh --type ci-cd >> ~/cleanup.log 2>&1") | crontab - -``` - -### Step 9: Configure Firewall - -```bash -sudo ufw --force enable -sudo ufw default deny incoming -sudo ufw default allow outgoing -sudo ufw allow ssh -sudo ufw allow 5000/tcp # Docker registry (public read access) -sudo ufw allow 8080/tcp # Registry UI (public read access) -``` - -**Security Model**: -- **Port 5000 (Registry)**: Public read access, authenticated write access -- **Port 8080 (UI)**: Public read access for browsing images -- **SSH**: Restricted to your IP addresses -- **All other ports**: Blocked - -### Step 10: Test CI/CD Setup - -#### 10.1 Test Docker Installation - -```bash -docker --version -docker compose --version -``` - -#### 10.2 Check Registry Status - -```bash -cd /opt/registry -docker compose ps -``` - -#### 10.3 Test Registry Access - -```bash -curl http://localhost:5000/v2/_catalog -``` - -#### 10.4 Get Public Key for Production Server - -```bash -cat ~/.ssh/id_ed25519.pub -``` - -**Important**: Copy this public key - you'll need it for the production server setup. - ---- - -## Part 2: Production Linode Setup - -### Step 11: Initial System Setup - -#### 11.1 Update the System - -```bash -sudo apt update && sudo apt upgrade -y -``` - -#### 11.2 Configure Timezone - -```bash -# Configure timezone interactively -sudo dpkg-reconfigure tzdata - -# Verify timezone setting -date -``` - -**What this does**: Opens an interactive dialog to select your timezone. Navigate through the menus to choose your preferred timezone (e.g., UTC, America/New_York, Europe/London, Asia/Tokyo). - -**Expected output**: After selecting your timezone, the `date` command should show the current date and time in your selected timezone. - -#### 11.3 Configure /etc/hosts - -```bash -# Add localhost entries for both IPv4 and IPv6 -echo "127.0.0.1 localhost" | sudo tee -a /etc/hosts -echo "::1 localhost ip6-localhost ip6-loopback" | sudo tee -a /etc/hosts -echo "YOUR_PRODUCTION_IPV4_ADDRESS localhost" | sudo tee -a /etc/hosts -echo "YOUR_PRODUCTION_IPV6_ADDRESS localhost" | sudo tee -a /etc/hosts - -# Verify the configuration -cat /etc/hosts -``` - -**What this does**: -- Adds localhost entries for both IPv4 and IPv6 addresses to `/etc/hosts` -- Ensures proper localhost resolution for both IPv4 and IPv6 - -**Important**: Replace `YOUR_PRODUCTION_IPV4_ADDRESS` and `YOUR_PRODUCTION_IPV6_ADDRESS` with the actual IPv4 and IPv6 addresses of your Production Linode obtained from your Linode dashboard. - -**Expected output**: The `/etc/hosts` file should show entries for `127.0.0.1`, `::1`, and your Linode's actual IP addresses all mapping to `localhost`. - -#### 11.4 Install Essential Packages - -```bash -sudo apt install -y \ - curl \ - wget \ - git \ - ca-certificates \ - apt-transport-https \ - software-properties-common \ - ufw \ - fail2ban \ - htop \ - nginx \ - certbot \ - python3-certbot-nginx -``` - -### Step 12: Create Users - -#### 12.1 Create the SERVICE_USER User - -```bash -# Create dedicated group for the service account -sudo groupadd -r SERVICE_USER - -# Create service account user with dedicated group -sudo useradd -r -g SERVICE_USER -s /bin/bash -m -d /home/SERVICE_USER SERVICE_USER -echo "SERVICE_USER:$(openssl rand -base64 32)" | sudo chpasswd -``` - -#### 12.2 Create the DEPLOY_USER User - -```bash -# Create deployment user -sudo useradd -m -s /bin/bash DEPLOY_USER -sudo usermod -aG sudo DEPLOY_USER -echo "DEPLOY_USER:$(openssl rand -base64 32)" | sudo chpasswd -``` - -#### 12.3 Verify Users - -```bash -sudo su - SERVICE_USER -whoami -pwd -exit - -sudo su - DEPLOY_USER -whoami -pwd -exit -``` - -### Step 13: Install Docker - -#### 13.1 Add Docker Repository - -```bash -curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg -echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null -sudo apt update -``` - -#### 13.2 Install Docker Packages - -```bash -sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin -``` - -#### 13.3 Configure Docker for Service Account - -```bash -sudo usermod -aG docker SERVICE_USER -``` - -### Step 14: Install Docker Compose - -```bash -sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose -sudo chmod +x /usr/local/bin/docker-compose -``` - -### Step 15: Configure Security - -#### 15.1 Configure Firewall - -```bash -sudo ufw --force enable -sudo ufw default deny incoming -sudo ufw default allow outgoing -sudo ufw allow ssh -sudo ufw allow 80/tcp -sudo ufw allow 443/tcp -sudo ufw allow 3000/tcp -sudo ufw allow 3001/tcp -``` - -#### 15.2 Configure Fail2ban - -```bash -sudo systemctl enable fail2ban -sudo systemctl start fail2ban -``` - -### Step 16: Create Application Directory - -#### 16.1 Create Directory Structure - -```bash -sudo mkdir -p /opt/APP_NAME -sudo chown SERVICE_USER:SERVICE_USER /opt/APP_NAME -``` - -**Note**: Replace `APP_NAME` with your actual application name. This directory name can be controlled via the `APP_NAME` secret in your Forgejo repository settings. If you set the `APP_NAME` secret to `myapp`, the deployment directory will be `/opt/myapp`. - -#### 16.2 Create SSL Directory (Optional - for domain users) - -```bash -sudo mkdir -p /opt/APP_NAME/nginx/ssl -sudo chown SERVICE_USER:SERVICE_USER /opt/APP_NAME/nginx/ssl -``` - -### Step 17: Clone Repository and Set Up Application Files - -#### 17.1 Switch to SERVICE_USER User - -```bash -sudo su - SERVICE_USER -``` - -#### 17.2 Clone Repository - -```bash -cd /opt/APP_NAME -git clone https://your-forgejo-instance/your-username/APP_NAME.git . -``` - -**Important**: The repository includes a pre-configured `nginx/nginx.conf` file that handles both SSL and non-SSL scenarios, with proper security headers, rate limiting, and CORS configuration. This file will be automatically used by the Docker Compose setup. - -**Important**: The repository also includes a pre-configured `.forgejo/workflows/ci.yml` file that handles the complete CI/CD pipeline including testing, building, and deployment. This workflow is already set up to work with the private registry and production deployment. - -**Note**: Replace `your-forgejo-instance` and `your-username/APP_NAME` with your actual Forgejo instance URL and repository path. - -#### 17.3 Create Environment File - -The repository doesn't include a `.env.example` file for security reasons. The CI/CD pipeline will create the `.env` file dynamically during deployment. However, for manual testing or initial setup, you can create a basic `.env` file: - -```bash -cat > /opt/APP_NAME/.env << 'EOF' -# Production Environment Variables -POSTGRES_PASSWORD=your_secure_password_here -REGISTRY=YOUR_CI_CD_IP:5000 -IMAGE_NAME=APP_NAME -IMAGE_TAG=latest - -# Database Configuration -POSTGRES_DB=sharenet -POSTGRES_USER=sharenet -DATABASE_URL=postgresql://sharenet:your_secure_password_here@postgres:5432/sharenet - -# Application Configuration -NODE_ENV=production -RUST_LOG=info -RUST_BACKTRACE=1 -EOF -``` - -**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address and `your_secure_password_here` with a strong password. - -#### 17.4 Configure Docker for Registry Access - -```bash -# Add the CI/CD registry to Docker's insecure registries -sudo mkdir -p /etc/docker -sudo tee /etc/docker/daemon.json << EOF -{ - "insecure-registries": ["YOUR_CI_CD_IP:5000"] -} -EOF - -# Restart Docker to apply changes -sudo systemctl restart docker -``` - -**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. - -### Step 18: Set Up SSH Key Authentication - -#### 18.1 Add CI/CD Public Key - -```bash -# Create .ssh directory for SERVICE_USER -mkdir -p ~/.ssh -chmod 700 ~/.ssh - -# Add the CI/CD public key (copy from CI/CD Linode) -echo "YOUR_CI_CD_PUBLIC_KEY" >> ~/.ssh/authorized_keys -chmod 600 ~/.ssh/authorized_keys -``` - -**Important**: Replace `YOUR_CI_CD_PUBLIC_KEY` with the public key from the CI/CD Linode (the output from `cat ~/.ssh/id_ed25519.pub` on the CI/CD Linode). - -#### 18.2 Test SSH Connection - -From the CI/CD Linode, test the SSH connection: - -```bash -ssh production -``` - -**Expected output**: You should be able to SSH to the production server without a password prompt. - -### Step 19: Test Production Setup - -#### 19.1 Test Docker Installation - -```bash -docker --version -docker compose --version -``` - -#### 19.2 Test Registry Access - -```bash -# Test pulling an image from the CI/CD registry -docker pull YOUR_CI_CD_IP:5000/APP_NAME/backend:latest -``` - -**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. - -#### 19.3 Test Application Deployment - -```bash -cd /opt/APP_NAME -docker compose up -d -``` - -#### 19.4 Verify Application Status - -```bash -docker compose ps -curl http://localhost:3000 -curl http://localhost:3001/health -``` - -**Expected Output**: -- All containers should be running -- Frontend should be accessible on port 3000 -- Backend health check should return 200 OK - ---- - -## Part 3: Final Configuration and Testing - -### Step 20: Configure Forgejo Repository Secrets - -#### 20.1 Required Repository Secrets - -Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**: - -**Required Secrets:** -- `CI_CD_IP`: Your CI/CD Linode IP address -- `PRODUCTION_IP`: Your Production Linode IP address -- `DEPLOY_USER`: The deployment user name (e.g., `deploy`, `ci`, `admin`) -- `SERVICE_USER`: The service user name (e.g., `appuser`, `service`, `app`) -- `APP_NAME`: Your application name (e.g., `sharenet`, `myapp`) -- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database - -**Optional Secrets (for domain users):** -- `DOMAIN`: Your domain name (e.g., `example.com`) -- `EMAIL`: Your email for SSL certificate notifications - -#### 20.2 Configure Forgejo Actions Runner - -##### 20.2.1 Get Runner Token - -1. Go to your Forgejo repository -2. Navigate to **Settings → Actions → Runners** -3. Click **"New runner"** -4. Copy the registration token - -##### 20.2.2 Configure Runner - -```bash -# Switch to DEPLOY_USER on CI/CD Linode -sudo su - DEPLOY_USER - -# Get the registration token from your Forgejo repository -# Go to Settings → Actions → Runners → New runner -# Copy the registration token - -# Configure the runner -forgejo-runner register \ - --instance https://your-forgejo-instance \ - --token YOUR_TOKEN \ - --name "ci-cd-runner" \ - --labels "ubuntu-latest,docker" \ - --no-interactive -``` - -##### 20.2.3 Start Runner - -```bash -sudo systemctl start forgejo-runner.service -sudo systemctl status forgejo-runner.service -``` - -##### 20.2.4 Test Runner Configuration - -```bash -# Check if the runner is running -sudo systemctl status forgejo-runner.service - -# Check runner logs -sudo journalctl -u forgejo-runner.service -f --no-pager - -# Test runner connectivity (in a separate terminal) -forgejo-runner list - -# Verify runner appears in Forgejo -# Go to your Forgejo repository → Settings → Actions → Runners -# You should see your runner listed as "ci-cd-runner" with status "Online" -``` - -**Expected Output**: -- `systemctl status` should show "active (running)" -- `forgejo-runner list` should show your runner -- Forgejo web interface should show the runner as online - -**If something goes wrong**: -- Check logs: `sudo journalctl -u forgejo-runner.service -f` -- Verify token: Make sure the registration token is correct -- Check network: Ensure the runner can reach your Forgejo instance -- Restart service: `sudo systemctl restart forgejo-runner.service` - -### Step 21: Set Up Monitoring and Cleanup - -#### 21.1 Monitoring Script +#### 9.1 Monitoring Script **Important**: The repository includes a pre-configured monitoring script in the `scripts/` directory that can be used for both CI/CD and production monitoring. @@ -1434,7 +839,7 @@ chmod +x scripts/monitor.sh **Note**: The repository script is more comprehensive and includes proper error handling, colored output, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate monitoring information. -#### 21.2 Cleanup Script +#### 9.2 Cleanup Script **Important**: The repository includes a pre-configured cleanup script in the `scripts/` directory that can be used for both CI/CD and production cleanup operations. @@ -1461,7 +866,7 @@ chmod +x scripts/cleanup.sh **Note**: The repository script is more comprehensive and includes proper error handling, colored output, dry-run mode, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate cleanup operations. -#### 21.3 Test Cleanup Script +#### 9.3 Test Cleanup Script ```bash # Create some test images to clean up @@ -1499,7 +904,7 @@ docker network ls - Check registry access: `cd /opt/APP_NAME/registry && docker compose ps` - Run manually: `bash -x scripts/cleanup.sh` -#### 21.4 Set Up Automated Cleanup +#### 9.4 Set Up Automated Cleanup ```bash # Create a cron job to run cleanup daily at 3 AM using the repository script @@ -1515,15 +920,552 @@ crontab -l - **Logging**: All cleanup output is logged to `/tmp/cleanup.log` - **What it cleans**: Unused Docker images, volumes, networks, and registry images -### Step 22: Test Complete Pipeline +### Step 10: Configure Firewall -#### 22.1 Trigger a Test Build +```bash +sudo ufw --force enable +sudo ufw default deny incoming +sudo ufw default allow outgoing +sudo ufw allow ssh +sudo ufw allow 5000/tcp # Docker registry (public read access) +sudo ufw allow 8080/tcp # Registry UI (public read access) +``` + +**Security Model**: +- **Port 5000 (Registry)**: Public read access, authenticated write access +- **Port 8080 (UI)**: Public read access for browsing images +- **SSH**: Restricted to your IP addresses +- **All other ports**: Blocked + +### Step 11: Test CI/CD Setup + +#### 11.1 Test Docker Installation + +```bash +docker --version +docker compose --version +``` + +#### 11.2 Check Registry Status + +```bash +cd /opt/registry +docker compose ps +``` + +#### 11.3 Test Registry Access + +```bash +curl http://localhost:5000/v2/_catalog +``` + +#### 11.4 Get Public Key for Production Server + +```bash +cat ~/.ssh/id_ed25519.pub +``` + +**Important**: Copy this public key - you'll need it for the production server setup. + +--- + +## Part 2: Production Linode Setup + +### Step 12: Initial System Setup + +#### 12.1 Update the System + +```bash +sudo apt update && sudo apt upgrade -y +``` + +#### 12.2 Configure Timezone + +```bash +# Configure timezone interactively +sudo dpkg-reconfigure tzdata + +# Verify timezone setting +date +``` + +**What this does**: Opens an interactive dialog to select your timezone. Navigate through the menus to choose your preferred timezone (e.g., UTC, America/New_York, Europe/London, Asia/Tokyo). + +**Expected output**: After selecting your timezone, the `date` command should show the current date and time in your selected timezone. + +#### 12.3 Configure /etc/hosts + +```bash +# Add localhost entries for both IPv4 and IPv6 +echo "127.0.0.1 localhost" | sudo tee -a /etc/hosts +echo "::1 localhost ip6-localhost ip6-loopback" | sudo tee -a /etc/hosts +echo "YOUR_PRODUCTION_IPV4_ADDRESS localhost" | sudo tee -a /etc/hosts +echo "YOUR_PRODUCTION_IPV6_ADDRESS localhost" | sudo tee -a /etc/hosts + +# Verify the configuration +cat /etc/hosts +``` + +**What this does**: +- Adds localhost entries for both IPv4 and IPv6 addresses to `/etc/hosts` +- Ensures proper localhost resolution for both IPv4 and IPv6 + +**Important**: Replace `YOUR_PRODUCTION_IPV4_ADDRESS` and `YOUR_PRODUCTION_IPV6_ADDRESS` with the actual IPv4 and IPv6 addresses of your Production Linode obtained from your Linode dashboard. + +**Expected output**: The `/etc/hosts` file should show entries for `127.0.0.1`, `::1`, and your Linode's actual IP addresses all mapping to `localhost`. + +#### 12.4 Install Essential Packages + +```bash +sudo apt install -y \ + curl \ + wget \ + git \ + ca-certificates \ + apt-transport-https \ + software-properties-common \ + ufw \ + fail2ban \ + htop \ + nginx \ + certbot \ + python3-certbot-nginx +``` + +### Step 13: Create Users + +#### 13.1 Create the SERVICE_USER User + +```bash +# Create dedicated group for the service account +sudo groupadd -r SERVICE_USER + +# Create service account user with dedicated group +sudo useradd -r -g SERVICE_USER -s /bin/bash -m -d /home/SERVICE_USER SERVICE_USER +echo "SERVICE_USER:$(openssl rand -base64 32)" | sudo chpasswd +``` + +#### 13.2 Create the DEPLOY_USER User + +```bash +# Create deployment user +sudo useradd -m -s /bin/bash DEPLOY_USER +sudo usermod -aG sudo DEPLOY_USER +echo "DEPLOY_USER:$(openssl rand -base64 32)" | sudo chpasswd +``` + +#### 13.3 Verify Users + +```bash +sudo su - SERVICE_USER +whoami +pwd +exit + +sudo su - DEPLOY_USER +whoami +pwd +exit +``` + +### Step 14: Install Docker + +#### 14.1 Add Docker Repository + +```bash +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +sudo apt update +``` + +#### 14.2 Install Docker Packages + +```bash +sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin +``` + +#### 14.3 Configure Docker for Service Account + +```bash +sudo usermod -aG docker SERVICE_USER +``` + +### Step 15: Install Docker Compose + +```bash +sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose +sudo chmod +x /usr/local/bin/docker-compose +``` + +### Step 16: Configure Security + +#### 16.1 Configure Firewall + +```bash +sudo ufw --force enable +sudo ufw default deny incoming +sudo ufw default allow outgoing +sudo ufw allow ssh +sudo ufw allow 80/tcp +sudo ufw allow 443/tcp +sudo ufw allow 3000/tcp +sudo ufw allow 3001/tcp +``` + +#### 16.2 Configure Fail2ban + +```bash +sudo systemctl enable fail2ban +sudo systemctl start fail2ban +``` + +### Step 17: Create Application Directory + +#### 17.1 Create Directory Structure + +```bash +sudo mkdir -p /opt/APP_NAME +sudo chown SERVICE_USER:SERVICE_USER /opt/APP_NAME +``` + +**Note**: Replace `APP_NAME` with your actual application name. This directory name can be controlled via the `APP_NAME` secret in your Forgejo repository settings. If you set the `APP_NAME` secret to `myapp`, the deployment directory will be `/opt/myapp`. + +#### 17.2 Create SSL Directory (Optional - for domain users) + +```bash +sudo mkdir -p /opt/APP_NAME/nginx/ssl +sudo chown SERVICE_USER:SERVICE_USER /opt/APP_NAME/nginx/ssl +``` + +### Step 18: Clone Repository and Set Up Application Files + +#### 18.1 Switch to SERVICE_USER User + +```bash +sudo su - SERVICE_USER +``` + +#### 18.2 Clone Repository + +```bash +cd /opt/APP_NAME +git clone https://your-forgejo-instance/your-username/APP_NAME.git . +``` + +**Important**: The repository includes a pre-configured `nginx/nginx.conf` file that handles both SSL and non-SSL scenarios, with proper security headers, rate limiting, and CORS configuration. This file will be automatically used by the Docker Compose setup. + +**Important**: The repository also includes a pre-configured `.forgejo/workflows/ci.yml` file that handles the complete CI/CD pipeline including testing, building, and deployment. This workflow is already set up to work with the private registry and production deployment. + +**Note**: Replace `your-forgejo-instance` and `your-username/APP_NAME` with your actual Forgejo instance URL and repository path. + +#### 18.3 Create Environment File + +The repository doesn't include a `.env.example` file for security reasons. The CI/CD pipeline will create the `.env` file dynamically during deployment. However, for manual testing or initial setup, you can create a basic `.env` file: + +```bash +cat > /opt/APP_NAME/.env << 'EOF' +# Production Environment Variables +POSTGRES_PASSWORD=your_secure_password_here +REGISTRY=YOUR_CI_CD_IP:5000 +IMAGE_NAME=APP_NAME +IMAGE_TAG=latest + +# Database Configuration +POSTGRES_DB=sharenet +POSTGRES_USER=sharenet +DATABASE_URL=postgresql://sharenet:your_secure_password_here@postgres:5432/sharenet + +# Application Configuration +NODE_ENV=production +RUST_LOG=info +RUST_BACKTRACE=1 +EOF +``` + +**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address and `your_secure_password_here` with a strong password. + +#### 18.4 Configure Docker for Registry Access + +```bash +# Add the CI/CD registry to Docker's insecure registries +sudo mkdir -p /etc/docker +sudo tee /etc/docker/daemon.json << EOF +{ + "insecure-registries": ["YOUR_CI_CD_IP:5000"] +} +EOF + +# Restart Docker to apply changes +sudo systemctl restart docker +``` + +**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. + +### Step 19: Set Up SSH Key Authentication + +#### 19.1 Add CI/CD Public Key + +```bash +# Create .ssh directory for SERVICE_USER +mkdir -p ~/.ssh +chmod 700 ~/.ssh + +# Add the CI/CD public key (copy from CI/CD Linode) +echo "YOUR_CI_CD_PUBLIC_KEY" >> ~/.ssh/authorized_keys +chmod 600 ~/.ssh/authorized_keys +``` + +**Important**: Replace `YOUR_CI_CD_PUBLIC_KEY` with the public key from the CI/CD Linode (the output from `cat ~/.ssh/id_ed25519.pub` on the CI/CD Linode). + +#### 19.2 Test SSH Connection + +From the CI/CD Linode, test the SSH connection: + +```bash +ssh production +``` + +**Expected output**: You should be able to SSH to the production server without a password prompt. + +### Step 20: Test Production Setup + +#### 20.1 Test Docker Installation + +```bash +docker --version +docker compose --version +``` + +#### 20.2 Test Registry Access + +```bash +# Test pulling an image from the CI/CD registry +docker pull YOUR_CI_CD_IP:5000/APP_NAME/backend:latest +``` + +**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. + +#### 20.3 Test Application Deployment + +```bash +cd /opt/APP_NAME +docker compose up -d +``` + +#### 20.4 Verify Application Status + +```bash +docker compose ps +curl http://localhost:3000 +curl http://localhost:3001/health +``` + +**Expected Output**: +- All containers should be running +- Frontend should be accessible on port 3000 +- Backend health check should return 200 OK + +--- + +## Part 3: Final Configuration and Testing + +### Step 21: Configure Forgejo Repository Secrets + +#### 21.1 Required Repository Secrets + +Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**: + +**Required Secrets:** +- `CI_CD_IP`: Your CI/CD Linode IP address +- `PRODUCTION_IP`: Your Production Linode IP address +- `DEPLOY_USER`: The deployment user name (e.g., `deploy`, `ci`, `admin`) +- `SERVICE_USER`: The service user name (e.g., `appuser`, `service`, `app`) +- `APP_NAME`: Your application name (e.g., `sharenet`, `myapp`) +- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database + +**Optional Secrets (for domain users):** +- `DOMAIN`: Your domain name (e.g., `example.com`) +- `EMAIL`: Your email for SSL certificate notifications + +#### 21.2 Configure Forgejo Actions Runner + +##### 21.2.1 Get Runner Token + +1. Go to your Forgejo repository +2. Navigate to **Settings → Actions → Runners** +3. Click **"New runner"** +4. Copy the registration token + +##### 21.2.2 Configure Runner + +```bash +# Switch to DEPLOY_USER on CI/CD Linode +sudo su - DEPLOY_USER + +# Get the registration token from your Forgejo repository +# Go to Settings → Actions → Runners → New runner +# Copy the registration token + +# Configure the runner +forgejo-runner register \ + --instance https://your-forgejo-instance \ + --token YOUR_TOKEN \ + --name "ci-cd-runner" \ + --labels "ubuntu-latest,docker" \ + --no-interactive +``` + +##### 21.2.3 Start Runner + +```bash +sudo systemctl start forgejo-runner.service +sudo systemctl status forgejo-runner.service +``` + +##### 21.2.4 Test Runner Configuration + +```bash +# Check if the runner is running +sudo systemctl status forgejo-runner.service + +# Check runner logs +sudo journalctl -u forgejo-runner.service -f --no-pager + +# Test runner connectivity (in a separate terminal) +forgejo-runner list + +# Verify runner appears in Forgejo +# Go to your Forgejo repository → Settings → Actions → Runners +# You should see your runner listed as "ci-cd-runner" with status "Online" +``` + +**Expected Output**: +- `systemctl status` should show "active (running)" +- `forgejo-runner list` should show your runner +- Forgejo web interface should show the runner as online + +**If something goes wrong**: +- Check logs: `sudo journalctl -u forgejo-runner.service -f` +- Verify token: Make sure the registration token is correct +- Check network: Ensure the runner can reach your Forgejo instance +- Restart service: `sudo systemctl restart forgejo-runner.service` + +### Step 22: Set Up Monitoring and Cleanup + +#### 22.1 Monitoring Script + +**Important**: The repository includes a pre-configured monitoring script in the `scripts/` directory that can be used for both CI/CD and production monitoring. + +**Repository Script**: +- `scripts/monitor.sh` - Comprehensive monitoring script with support for both CI/CD and production environments + +**To use the repository monitoring script**: +```bash +# The repository is already cloned at /opt/APP_NAME/ +cd /opt/APP_NAME + +# Make the script executable +chmod +x scripts/monitor.sh + +# Test CI/CD monitoring +./scripts/monitor.sh --type ci-cd + +# Test production monitoring (if you have a production setup) +./scripts/monitor.sh --type production +``` + +**Note**: The repository script is more comprehensive and includes proper error handling, colored output, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate monitoring information. + +#### 22.2 Cleanup Script + +**Important**: The repository includes a pre-configured cleanup script in the `scripts/` directory that can be used for both CI/CD and production cleanup operations. + +**Repository Script**: +- `scripts/cleanup.sh` - Comprehensive cleanup script with support for both CI/CD and production environments + +**To use the repository cleanup script**: +```bash +# The repository is already cloned at /opt/APP_NAME/ +cd /opt/APP_NAME + +# Make the script executable +chmod +x scripts/cleanup.sh + +# Test CI/CD cleanup (dry run first) +./scripts/cleanup.sh --type ci-cd --dry-run + +# Run CI/CD cleanup +./scripts/cleanup.sh --type ci-cd + +# Test production cleanup (dry run first) +./scripts/cleanup.sh --type production --dry-run +``` + +**Note**: The repository script is more comprehensive and includes proper error handling, colored output, dry-run mode, and support for both CI/CD and production environments. It automatically detects the environment and provides appropriate cleanup operations. + +#### 22.3 Test Cleanup Script + +```bash +# Create some test images to clean up +docker pull alpine:latest +docker pull nginx:latest +docker tag alpine:latest test-cleanup:latest +docker tag nginx:latest test-cleanup2:latest + +# Test cleanup with dry run first +./scripts/cleanup.sh --type ci-cd --dry-run + +# Run the cleanup script +./scripts/cleanup.sh --type ci-cd + +# Verify cleanup worked +echo "Checking remaining images:" +docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" + +echo "Checking remaining volumes:" +docker volume ls + +echo "Checking remaining networks:" +docker network ls +``` + +**Expected Output**: +- Cleanup script should run without errors +- Test images should be removed +- System should report cleanup completion +- Remaining images should be minimal (only actively used ones) + +**If something goes wrong**: +- Check script permissions: `ls -la scripts/cleanup.sh` +- Verify Docker access: `docker ps` +- Check registry access: `cd /opt/APP_NAME/registry && docker compose ps` +- Run manually: `bash -x scripts/cleanup.sh` + +#### 22.4 Set Up Automated Cleanup + +```bash +# Create a cron job to run cleanup daily at 3 AM using the repository script +(crontab -l 2>/dev/null; echo "0 3 * * * cd /opt/APP_NAME && ./scripts/cleanup.sh --type ci-cd >> /tmp/cleanup.log 2>&1") | crontab - + +# Verify the cron job was added +crontab -l +``` + +**What this does:** +- **Runs automatically**: The cleanup script runs every day at 3:00 AM +- **Frequency**: Daily cleanup to prevent disk space issues +- **Logging**: All cleanup output is logged to `/tmp/cleanup.log` +- **What it cleans**: Unused Docker images, volumes, networks, and registry images + +### Step 23: Test Complete Pipeline + +#### 23.1 Trigger a Test Build 1. **Make a small change** to your repository (e.g., update a comment or add a test file) 2. **Commit and push** the changes to trigger the CI/CD pipeline 3. **Monitor the build** in your Forgejo repository → Actions tab -#### 22.2 Verify Pipeline Steps +#### 23.2 Verify Pipeline Steps The pipeline should execute these steps in order: @@ -1535,7 +1477,7 @@ The pipeline should execute these steps in order: 6. **Push to Registry**: Push images to your private registry 7. **Deploy to Production**: Deploy to production server -#### 22.3 Check Registry +#### 23.3 Check Registry ```bash # On CI/CD Linode @@ -1549,7 +1491,7 @@ curl -k https://localhost:8080/v2/APP_NAME/backend/tags/list curl -k https://localhost:8080/v2/APP_NAME/frontend/tags/list ``` -#### 22.4 Verify Production Deployment +#### 23.4 Verify Production Deployment ```bash # On Production Linode @@ -1567,16 +1509,16 @@ docker compose logs backend docker compose logs frontend ``` -#### 22.5 Test Application Functionality +#### 23.5 Test Application Functionality 1. **Frontend**: Visit your production URL (IP or domain) 2. **Backend API**: Test API endpoints 3. **Database**: Verify database connections 4. **Logs**: Check for any errors in application logs -### Step 23: Set Up SSL/TLS (Optional - Domain Users) +### Step 24: Set Up SSL/TLS (Optional - Domain Users) -#### 23.1 Install SSL Certificate +#### 24.1 Install SSL Certificate If you have a domain pointing to your Production Linode: @@ -1588,7 +1530,7 @@ sudo certbot --nginx -d your-domain.com sudo certbot certificates ``` -#### 23.2 Configure Auto-Renewal +#### 24.2 Configure Auto-Renewal ```bash # Test auto-renewal @@ -1600,9 +1542,9 @@ sudo crontab -e # 0 12 * * * /usr/bin/certbot renew --quiet ``` -### Step 24: Final Verification +### Step 25: Final Verification -#### 24.1 Security Check +#### 25.1 Security Check ```bash # Check firewall status @@ -1615,7 +1557,7 @@ sudo systemctl status fail2ban sudo grep "PasswordAuthentication" /etc/ssh/sshd_config ``` -#### 24.2 Performance Check +#### 25.2 Performance Check ```bash # Check system resources @@ -1628,7 +1570,7 @@ df -h docker system df ``` -#### 24.3 Backup Verification +#### 25.3 Backup Verification ```bash # Test backup script @@ -1639,16 +1581,16 @@ cd /opt/APP_NAME ./scripts/backup.sh ``` -### Step 25: Documentation and Maintenance +### Step 26: Documentation and Maintenance -#### 25.1 Update Documentation +#### 26.1 Update Documentation 1. **Update README.md** with deployment information 2. **Document environment variables** and their purposes 3. **Create troubleshooting guide** for common issues 4. **Document backup and restore procedures** -#### 25.2 Set Up Monitoring Alerts +#### 26.2 Set Up Monitoring Alerts ```bash # Set up monitoring cron job @@ -1658,7 +1600,7 @@ cd /opt/APP_NAME tail -f /tmp/monitor.log ``` -#### 25.3 Regular Maintenance Tasks +#### 26.3 Regular Maintenance Tasks **Daily:** - Check application logs for errors