Change from docker to podman and add security hardening
Some checks failed
CI/CD Pipeline (Fully Isolated DinD) / Run Tests (DinD) (push) Has been cancelled
CI/CD Pipeline (Fully Isolated DinD) / Build and Push Docker Images (DinD) (push) Has been cancelled
CI/CD Pipeline (Fully Isolated DinD) / Deploy to Production (push) Has been cancelled

This commit is contained in:
continuist 2025-08-18 23:03:06 -04:00
parent 0b4fb89e77
commit 98c5fb948f
10 changed files with 861 additions and 245 deletions

View file

@ -7,9 +7,9 @@ This guide covers setting up a complete Continuous Integration/Continuous Deploy
``` ```
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Forgejo Host │ │ CI/CD Linode │ │ Production Linode│ │ Forgejo Host │ │ CI/CD Linode │ │ Production Linode│
│ (Repository) │ │ (Actions Runner)│ │ (Docker Deploy) │ │ (Repository) │ │ (Actions Runner)│ │ (Podman Deploy) │
│ │ │ + Docker Registry│ │ │ │ │ │ + Podman Registry│ │ │
│ │ │ + DinD Container│ │ │ │ │ │ + PiP Container │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │ │ │ │
│ │ │ │ │ │
@ -27,18 +27,18 @@ This guide covers setting up a complete Continuous Integration/Continuous Deploy
5. **Production Deployment**: Production Linode pulls images and deploys 5. **Production Deployment**: Production Linode pulls images and deploys
6. **Health Check**: Application is verified and accessible 6. **Health Check**: Application is verified and accessible
## Key Benefits of DinD Approach ## Key Benefits of PiP (Podman-in-Podman) Approach
### **For Rust Testing:** ### **For Rust Testing:**
- ✅ **Fresh environment** every test run - ✅ **Fresh environment** every test run
- ✅ **Parallel execution** capability - ✅ **Parallel execution** capability
- ✅ **Isolated dependencies** - no test pollution - ✅ **Isolated dependencies** - no test pollution
- ✅ **Fast cleanup** - just restart DinD container - ✅ **Fast cleanup** - just restart PiP container
### **For CI/CD Operations:** ### **For CI/CD Operations:**
- ✅ **Zero resource contention** with Docker Registry - ✅ **Zero resource contention** with Podman Registry
- ✅ **Simple cleanup** - one-line container restart - ✅ **Simple cleanup** - one-line container restart
- ✅ **Perfect isolation** - CI/CD can't affect Docker Registry - ✅ **Perfect isolation** - CI/CD can't affect Podman Registry
- ✅ **Consistent environment** - same setup every time - ✅ **Consistent environment** - same setup every time
### **For Maintenance:** ### **For Maintenance:**
@ -64,16 +64,16 @@ This guide covers setting up a complete Continuous Integration/Continuous Deploy
### CI/CD Linode Features ### CI/CD Linode Features
- Forgejo Actions runner for automated builds - Forgejo Actions runner for automated builds
- **Docker-in-Docker (DinD) container** for isolated CI operations - **Podman-in-Podman (PiP) container** for isolated CI operations
- Docker Registry with nginx reverse proxy for image storage - Docker Registry v2 with nginx reverse proxy for image storage
- **FHS-compliant directory structure** for data, certificates, and logs - **FHS-compliant directory structure** for data, certificates, and logs
- Unauthenticated pulls, authenticated pushes - Unauthenticated pulls, authenticated pushes
- Automatic HTTPS with nginx - Automatic HTTPS with nginx
- Secure SSH communication with production - Secure SSH communication with production
- **Simplified cleanup** - just restart DinD container - **Simplified cleanup** - just restart PiP container
### Production Linode Features ### Production Linode Features
- Docker-based application deployment - Podman-based application deployment
- **Optional SSL/TLS certificate management** (if domain is provided) - **Optional SSL/TLS certificate management** (if domain is provided)
- Nginx reverse proxy with security headers - Nginx reverse proxy with security headers
- Automated backups and monitoring - Automated backups and monitoring
@ -81,11 +81,11 @@ This guide covers setting up a complete Continuous Integration/Continuous Deploy
### Pipeline Features ### Pipeline Features
- **Automated testing** on every code push in isolated environment - **Automated testing** on every code push in isolated environment
- **Automated image building** and registry push from DinD - **Automated image building** and registry push from PiP
- **Automated deployment** to production - **Automated deployment** to production
- **Rollback capability** with image versioning - **Rollback capability** with image versioning
- **Health monitoring** and logging - **Health monitoring** and logging
- **Zero resource contention** between CI/CD and Docker Registry - **Zero resource contention** between CI/CD and Docker Registry v2
## Security Model and User Separation ## Security Model and User Separation
@ -577,7 +577,26 @@ sudo apt install -y \
apache2-utils apache2-utils
``` ```
**What this does**: Installs development tools, SSL libraries, and utilities needed for Docker and application building. #### 1.5 Install Podman
```bash
# Install Podman and related tools
sudo apt install -y podman podman-compose
# Verify installation
podman --version
podman-compose --version
# Configure Podman for rootless operation (optional but recommended)
echo 'kernel.unprivileged_userns_clone=1' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
# Configure subuid/subgid for CI_SERVICE_USER
sudo usermod --add-subuids 100000-165535 CI_SERVICE_USER
sudo usermod --add-subgids 100000-165535 CI_SERVICE_USER
```
**What this does**: Installs development tools, SSL libraries, and utilities needed for Podman and application building.
### Step 2: Create Users ### Step 2: Create Users
@ -632,31 +651,27 @@ ls -la /opt/APP_NAME/registry/
- CI_SERVICE_USER owns all the files for security - CI_SERVICE_USER owns all the files for security
- Registry configuration files are now available at `/opt/APP_NAME/registry/` - Registry configuration files are now available at `/opt/APP_NAME/registry/`
### Step 4: Install Docker ### Step 4: Configure Podman for CI Service Account
#### 4.1 Add Docker Repository #### 4.1 Verify Podman Installation
```bash ```bash
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg # Podman should already be installed from Step 1.5
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null podman --version
sudo apt update podman-compose --version
``` ```
#### 4.2 Install Docker Packages #### 4.2 Configure Podman for CI Service Account
```bash ```bash
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin # Podman can run rootless, so no group membership needed
# But we'll configure it for the CI service account
sudo usermod -aG podman CI_SERVICE_USER
``` ```
#### 4.3 Configure Docker for CI Service Account ### Step 5: Set Up Docker Registry v2 with nginx
```bash We'll set up a basic Docker Registry v2 with nginx as a reverse proxy, configured to allow unauthenticated pulls but require authentication for pushes.
sudo usermod -aG docker CI_SERVICE_USER
```
### Step 5: Set Up Docker Registry with nginx
We'll set up a basic Docker Registry with nginx as a reverse proxy, configured to allow unauthenticated pulls but require authentication for pushes.
#### 5.1 Configure FHS-Compliant Registry Directories #### 5.1 Configure FHS-Compliant Registry Directories
@ -673,7 +688,7 @@ sudo chmod 755 /etc/registry/certs
sudo chmod 755 /var/log/registry sudo chmod 755 /var/log/registry
``` ```
#### 5.2 Create Docker Compose Setup #### 5.2 Create Docker Registry v2 Pod Setup
```bash ```bash
# Navigate to the cloned application directory # Navigate to the cloned application directory
@ -685,6 +700,45 @@ sudo sed -i "s/YOUR_CI_CD_IP/YOUR_ACTUAL_IP_ADDRESS/g" /opt/APP_NAME/registry/ng
# Update openssl.conf with your actual IP address and registry name # Update openssl.conf with your actual IP address and registry name
sudo sed -i "s/YOUR_CI_CD_IP/YOUR_ACTUAL_IP_ADDRESS/g" /opt/APP_NAME/registry/openssl.conf sudo sed -i "s/YOUR_CI_CD_IP/YOUR_ACTUAL_IP_ADDRESS/g" /opt/APP_NAME/registry/openssl.conf
sudo sed -i "s/YOUR_REGISTRY_NAME/APP_NAME-Registry/g" /opt/APP_NAME/registry/openssl.conf sudo sed -i "s/YOUR_REGISTRY_NAME/APP_NAME-Registry/g" /opt/APP_NAME/registry/openssl.conf
```
#### 5.2.1 Alternative: Use Podman Compose
```bash
# Use podman-compose (Docker Compose compatible)
podman-compose -f docker-registry.yml up -d
# Check status
podman-compose -f docker-registry.yml ps
```
#### 5.2.2 Security Features Applied
The Docker Registry v2 setup includes comprehensive security hardening:
**Container Security:**
- ✅ Rootless containers (runAsUser=1000, runAsGroup=1000)
- ✅ All Linux capabilities dropped
- ✅ Privilege escalation disabled
- ✅ Read-only root filesystem with tmpfs for /tmp
- ✅ Image deletion disabled (REGISTRY_STORAGE_DELETE_ENABLED=false)
**Network Security:**
- ✅ TLS 1.2/1.3 only with modern ciphers
- ✅ HSTS headers enabled
- ✅ Rate limiting (10r/s for reads, 5r/s for writes)
- ✅ Client max body size limits (2GB)
- ✅ Registry listens only internally (no host-published port)
**Resource Limits:**
- ✅ CPU limits: 1000m for registry, 500m for nginx
- ✅ Memory limits: 1Gi for registry, 512Mi for nginx
- ✅ File descriptor limits via ulimits
**Authentication & Authorization:**
- ✅ Basic auth with htpasswd for write operations
- ✅ Container policy enforcement via containers-policy.json
- ✅ Volume mounts with read-only where possible
# Create FHS-compliant authentication directory # Create FHS-compliant authentication directory
sudo mkdir -p /etc/registry/auth sudo mkdir -p /etc/registry/auth
@ -705,10 +759,14 @@ sudo chmod 600 /etc/registry/auth/.htpasswd
# Set proper permissions for configuration files # Set proper permissions for configuration files
sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/nginx.conf sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/nginx.conf
sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/openssl.conf sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/openssl.conf
sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/docker-compose.registry.yml sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/docker-registry.yml
sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/registry-pod.yaml
sudo chown CI_SERVICE_USER:CI_SERVICE_USER /opt/APP_NAME/registry/containers-policy.json
sudo chmod 644 /opt/APP_NAME/registry/nginx.conf sudo chmod 644 /opt/APP_NAME/registry/nginx.conf
sudo chmod 644 /opt/APP_NAME/registry/openssl.conf sudo chmod 644 /opt/APP_NAME/registry/openssl.conf
sudo chmod 644 /opt/APP_NAME/registry/docker-compose.registry.yml sudo chmod 644 /opt/APP_NAME/registry/docker-registry.yml
sudo chmod 644 /opt/APP_NAME/registry/registry-pod.yaml
sudo chmod 644 /opt/APP_NAME/registry/containers-policy.json
``` ```
#### 5.3 Create FHS-Compliant Directory Structure #### 5.3 Create FHS-Compliant Directory Structure
@ -720,9 +778,9 @@ sudo mkdir -p /etc/registry/certs/requests
sudo mkdir -p /etc/registry/certs/ca sudo mkdir -p /etc/registry/certs/ca
sudo mkdir -p /var/lib/registry/data sudo mkdir -p /var/lib/registry/data
# Set proper ownership for certificate and environment directories # Set proper ownership for certificate and auth directories
sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER /etc/registry/certs sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER /etc/registry/certs
sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER /etc/registry/env sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER /etc/registry/auth
sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER /var/lib/registry/data sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER /var/lib/registry/data
# Set proper permissions for certificate directories # Set proper permissions for certificate directories
@ -736,7 +794,7 @@ sudo chmod 755 /var/lib/registry/data # Registry data
sudo ln -sf /var/lib/registry/data /opt/APP_NAME/registry/registry sudo ln -sf /var/lib/registry/data /opt/APP_NAME/registry/registry
``` ```
#### 5.4 Generate TLS Certificate and Install in Docker Trust Store #### 5.4 Generate TLS Certificate and Install for Podman
**Choose one of the following options based on whether you have a domain name:** **Choose one of the following options based on whether you have a domain name:**
@ -787,12 +845,11 @@ sudo chmod 644 requests/registry.csr requests/openssl.conf # Requests - world r
# Verify certificate creation # Verify certificate creation
sudo -u CI_SERVICE_USER openssl x509 -in /etc/registry/certs/registry.crt -text -noout | grep -E "(Subject:|DNS:|IP Address:)" sudo -u CI_SERVICE_USER openssl x509 -in /etc/registry/certs/registry.crt -text -noout | grep -E "(Subject:|DNS:|IP Address:)"
# 2. Install CA certificate into Docker trust store # 2. Install CA certificate for Podman
sudo mkdir -p /etc/docker/certs.d/YOUR_ACTUAL_IP_ADDRESS sudo mkdir -p /etc/containers/certs.d/YOUR_ACTUAL_IP_ADDRESS
sudo cp /etc/registry/certs/ca/ca.crt /etc/docker/certs.d/YOUR_ACTUAL_IP_ADDRESS/ca.crt sudo cp /etc/registry/certs/ca/ca.crt /etc/containers/certs.d/YOUR_ACTUAL_IP_ADDRESS/ca.crt
sudo cp /etc/registry/certs/ca/ca.crt /usr/local/share/ca-certificates/registry-ca.crt sudo cp /etc/registry/certs/ca/ca.crt /usr/local/share/ca-certificates/registry-ca.crt
sudo update-ca-certificates sudo update-ca-certificates
sudo systemctl restart docker
``` ```
--- ---
@ -818,10 +875,9 @@ sudo chown CI_SERVICE_USER:CI_SERVICE_USER /etc/registry/certs/registry.key
sudo chmod 644 /etc/registry/certs/registry.crt sudo chmod 644 /etc/registry/certs/registry.crt
sudo chmod 600 /etc/registry/certs/registry.key sudo chmod 600 /etc/registry/certs/registry.key
# 2. Install certificate into Docker trust store # 2. Install certificate for Podman
sudo mkdir -p /etc/docker/certs.d/YOUR_DOMAIN_NAME sudo mkdir -p /etc/containers/certs.d/YOUR_DOMAIN_NAME
sudo cp /etc/registry/certs/registry.crt /etc/docker/certs.d/YOUR_DOMAIN_NAME/ca.crt sudo cp /etc/registry/certs/registry.crt /etc/containers/certs.d/YOUR_DOMAIN_NAME/ca.crt
sudo systemctl restart docker
``` ```
--- ---
@ -834,30 +890,27 @@ sudo systemctl restart docker
**After completing the steps for your chosen option, continue with Step 5.7 (Start Docker Registry with Docker Compose).** **After completing the steps for your chosen option, continue with Step 5.7 (Start Docker Registry with Docker Compose).**
#### 5.5 Install Certificate into Docker Trust Store (Option B Only) #### 5.5 Install Certificate for Podman (Option B Only)
**Important**: This step adds the Let's Encrypt certificate to Docker's trust store. Since Let's Encrypt is a trusted CA, Docker will automatically trust this certificate. **Important**: This step adds the Let's Encrypt certificate for Podman. Since Let's Encrypt is a trusted CA, Podman will automatically trust this certificate.
```bash ```bash
# Create Docker certificates directory for your domain # Create Podman certificates directory for your domain
sudo mkdir -p /etc/docker/certs.d/YOUR_DOMAIN_NAME sudo mkdir -p /etc/containers/certs.d/YOUR_DOMAIN_NAME
# Copy certificate to Docker trust store # Copy certificate for Podman
sudo cp /opt/registry/certs/registry.crt /etc/docker/certs.d/YOUR_DOMAIN_NAME/ca.crt sudo cp /opt/registry/certs/registry.crt /etc/containers/certs.d/YOUR_DOMAIN_NAME/ca.crt
# Restart Docker daemon to pick up the new certificate
sudo systemctl restart docker
# Verify certificate installation # Verify certificate installation
if [ -f "/etc/docker/certs.d/YOUR_DOMAIN_NAME/ca.crt" ]; then if [ -f "/etc/containers/certs.d/YOUR_DOMAIN_NAME/ca.crt" ]; then
echo "✅ Let's Encrypt certificate installed in Docker trust store" echo "✅ Let's Encrypt certificate installed for Podman"
else else
echo "❌ Failed to install certificate in Docker trust store" echo "❌ Failed to install certificate for Podman"
exit 1 exit 1
fi fi
echo "Certificate installation completed successfully!" echo "Certificate installation completed successfully!"
echo "Docker can now connect to the registry securely using your domain name" echo "Podman can now connect to the registry securely using your domain name"
``` ```
#### 5.6 Set Up Automatic Certificate Renewal (Option B Only) #### 5.6 Set Up Automatic Certificate Renewal (Option B Only)
@ -877,25 +930,26 @@ echo "Automatic certificate renewal configured!"
echo "Certificates will be renewed automatically and the registry service will be restarted" echo "Certificates will be renewed automatically and the registry service will be restarted"
``` ```
#### 5.7 Set Up Systemd Service for Docker Registry #### 5.7 Set Up Systemd Service for Docker Registry v2
```bash ```bash
# Create system-wide Docker configuration to avoid permission issues # Create system-wide Podman configuration to avoid permission issues
sudo mkdir -p /etc/docker sudo mkdir -p /etc/containers
sudo tee /etc/docker/config.json > /dev/null << 'EOF' sudo tee /etc/containers/registries.conf > /dev/null << 'EOF'
{ unqualified-search-registries = ["docker.io"]
"auths": {},
"HttpHeaders": { [[registry]]
"User-Agent": "Docker-Client/20.10.0 (linux)" prefix = "localhost"
} location = "localhost"
} insecure = true
EOF EOF
# Set proper permissions for system-wide Docker config # Set proper permissions for system-wide Podman config
sudo chmod 644 /etc/docker/config.json sudo chmod 644 /etc/containers/registries.conf
# Install systemd service from repository # Install systemd service and configuration files from repository
sudo cp /opt/APP_NAME/registry/docker-registry.service /etc/systemd/system/docker-registry.service sudo cp /opt/APP_NAME/registry/docker-registry.service /etc/systemd/system/docker-registry.service
sudo cp /opt/APP_NAME/registry/containers-policy.json /etc/containers/policy.json
# Replace APP_NAME placeholder with actual application name # Replace APP_NAME placeholder with actual application name
sudo sed -i "s/APP_NAME/YOUR_ACTUAL_APP_NAME/g" /etc/systemd/system/docker-registry.service sudo sed -i "s/APP_NAME/YOUR_ACTUAL_APP_NAME/g" /etc/systemd/system/docker-registry.service
@ -903,7 +957,7 @@ sudo sed -i "s/APP_NAME/YOUR_ACTUAL_APP_NAME/g" /etc/systemd/system/docker-regis
# Replace CI_SERVICE_USER placeholder with actual CI service user name # Replace CI_SERVICE_USER placeholder with actual CI service user name
sudo sed -i "s/CI_SERVICE_USER/YOUR_ACTUAL_CI_SERVICE_USER/g" /etc/systemd/system/docker-registry.service sudo sed -i "s/CI_SERVICE_USER/YOUR_ACTUAL_CI_SERVICE_USER/g" /etc/systemd/system/docker-registry.service
# Enable and start Docker Registry service # Enable and start Docker Registry v2 service
sudo systemctl daemon-reload sudo systemctl daemon-reload
sudo systemctl enable docker-registry.service sudo systemctl enable docker-registry.service
sudo systemctl start docker-registry.service sudo systemctl start docker-registry.service
@ -915,20 +969,20 @@ sudo systemctl status docker-registry.service
sudo journalctl -u docker-registry.service -f --no-pager -n 50 sudo journalctl -u docker-registry.service -f --no-pager -n 50
``` ```
#### 5.8 Verify Docker Registry Service #### 5.8 Verify Docker Registry v2 Service
```bash ```bash
# Check that the service is running properly # Check that the service is running properly
sudo systemctl status docker-registry.service sudo systemctl status docker-registry.service
# Check that containers are running # Check that pods are running
sudo su - CI_SERVICE_USER -c "cd /opt/APP_NAME/registry && docker compose -f docker-compose.registry.yml ps" sudo su - CI_SERVICE_USER -c "podman pod ps"
# Check nginx logs # Check nginx logs
sudo su - CI_SERVICE_USER -c "cd /opt/APP_NAME/registry && docker compose -f docker-compose.registry.yml logs nginx" sudo su - CI_SERVICE_USER -c "podman logs registry-pod-nginx"
# Check Registry logs # Check Registry logs
sudo su - CI_SERVICE_USER -c "cd /opt/APP_NAME/registry && docker compose -f docker-compose.registry.yml logs registry" sudo su - CI_SERVICE_USER -c "podman logs registry-pod-registry"
``` ```
#### 5.9 Test Registry Setup #### 5.9 Test Registry Setup
@ -936,34 +990,34 @@ sudo su - CI_SERVICE_USER -c "cd /opt/APP_NAME/registry && docker compose -f doc
**For Option A (Self-signed certificates):** **For Option A (Self-signed certificates):**
```bash ```bash
# Switch to CI_SERVICE_USER for testing (CI_SERVICE_USER runs CI pipeline and Docker operations) # Switch to CI_SERVICE_USER for testing (CI_SERVICE_USER runs CI pipeline and Podman operations)
sudo su - CI_SERVICE_USER sudo su - CI_SERVICE_USER
# Navigate to the application directory # Navigate to the application directory
cd /opt/APP_NAME cd /opt/APP_NAME
# Test authenticated push using the project's registry configuration (port 4443) # Test authenticated push using the project's registry configuration (port 4443)
echo "your-secure-registry-password" | docker login YOUR_ACTUAL_IP_ADDRESS:4443 -u registry-user --password-stdin echo "your-secure-registry-password" | podman login YOUR_ACTUAL_IP_ADDRESS:4443 -u registry-user --password-stdin
# Create and push test image to authenticated endpoint # Create and push test image to authenticated endpoint
echo "FROM alpine:latest" > /tmp/test.Dockerfile echo "FROM alpine:latest" > /tmp/test.Dockerfile
docker build -f /tmp/test.Dockerfile -t YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/test:latest /tmp podman build -f /tmp/test.Dockerfile -t YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/test:latest /tmp
docker push YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/test:latest podman push YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/test:latest
# Test unauthenticated pull from standard HTTPS endpoint (port 443) # Test unauthenticated pull from standard HTTPS endpoint (port 443)
docker logout YOUR_ACTUAL_IP_ADDRESS:4443 podman logout YOUR_ACTUAL_IP_ADDRESS:4443
docker pull YOUR_ACTUAL_IP_ADDRESS/APP_NAME/test:latest podman pull YOUR_ACTUAL_IP_ADDRESS/APP_NAME/test:latest
# Test that unauthorized push to authenticated endpoint is blocked # Test that unauthorized push to authenticated endpoint is blocked
echo "FROM alpine:latest" > /tmp/unauthorized.Dockerfile echo "FROM alpine:latest" > /tmp/unauthorized.Dockerfile
docker build -f /tmp/unauthorized.Dockerfile -t YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/unauthorized:latest /tmp podman build -f /tmp/unauthorized.Dockerfile -t YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/unauthorized:latest /tmp
docker push YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/unauthorized:latest podman push YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/unauthorized:latest
# Expected: This should fail with authentication error # Expected: This should fail with authentication error
# Clean up # Clean up
docker rmi YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/test:latest 2>/dev/null || true podman rmi YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/test:latest 2>/dev/null || true
docker rmi YOUR_ACTUAL_IP_ADDRESS/APP_NAME/test:latest 2>/dev/null || true podman rmi YOUR_ACTUAL_IP_ADDRESS/APP_NAME/test:latest 2>/dev/null || true
docker rmi YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/unauthorized:latest 2>/dev/null || true podman rmi YOUR_ACTUAL_IP_ADDRESS:4443/APP_NAME/unauthorized:latest 2>/dev/null || true
exit exit
``` ```
@ -976,13 +1030,13 @@ sudo su - CI_SERVICE_USER
# Navigate to the application directory # Navigate to the application directory
cd /opt/APP_NAME cd /opt/APP_NAME
# Test Docker login and push (now using Let's Encrypt certificate with domain) # Test Podman login and push (now using Let's Encrypt certificate with domain)
echo "your-secure-registry-password" | docker login YOUR_DOMAIN_NAME -u registry-user --password-stdin echo "your-secure-registry-password" | podman login YOUR_DOMAIN_NAME -u registry-user --password-stdin
# Create and push test image # Create and push test image
echo "FROM alpine:latest" > /tmp/test.Dockerfile echo "FROM alpine:latest" > /tmp/test.Dockerfile
docker build -f /tmp/test.Dockerfile -t YOUR_DOMAIN_NAME/APP_NAME/test:latest /tmp podman build -f /tmp/test.Dockerfile -t YOUR_DOMAIN_NAME/APP_NAME/test:latest /tmp
docker push YOUR_DOMAIN_NAME/APP_NAME/test:latest podman push YOUR_DOMAIN_NAME/APP_NAME/test:latest
# Test public pull (no authentication) # Test public pull (no authentication)
docker logout YOUR_DOMAIN_NAME docker logout YOUR_DOMAIN_NAME
@ -1017,8 +1071,8 @@ exit
If you get a TLS error like `remote error: tls: internal error` when using self-signed certificates, verify the certificate installation and Docker configuration: If you get a TLS error like `remote error: tls: internal error` when using self-signed certificates, verify the certificate installation and Docker configuration:
```bash ```bash
# Verify the certificate was installed correctly in Docker trust store # Verify the certificate was installed correctly for Podman
ls -la /etc/docker/certs.d/YOUR_ACTUAL_IP_ADDRESS/ ls -la /etc/containers/certs.d/YOUR_ACTUAL_IP_ADDRESS/
ls -la /usr/local/share/ca-certificates/registry-ca.crt ls -la /usr/local/share/ca-certificates/registry-ca.crt
# Verify certificate chain is valid # Verify certificate chain is valid
@ -1036,10 +1090,10 @@ sudo systemctl restart docker
# Wait for Docker Registry to restart, then test again # Wait for Docker Registry to restart, then test again
sleep 10 sleep 10
cd /opt/APP_NAME/registry cd /opt/APP_NAME/registry
docker compose -f docker-compose.registry.yml restart podman play kube registry-pod.yaml
# Test Docker login to authenticated endpoint # Test Podman login to authenticated endpoint
echo "your-secure-registry-password" | docker login YOUR_ACTUAL_IP_ADDRESS:4443 -u registry-user --password-stdin echo "your-secure-registry-password" | podman login YOUR_ACTUAL_IP_ADDRESS:4443 -u registry-user --password-stdin
``` ```
**Certificate Structure Summary:** **Certificate Structure Summary:**
@ -1053,7 +1107,7 @@ The project uses a two-port configuration:
- **CA Certificates**: `/etc/registry/certs/ca/` (mode 644) - **CA Certificates**: `/etc/registry/certs/ca/` (mode 644)
- **Certificate Requests**: `/etc/registry/certs/requests/` (mode 644) - **Certificate Requests**: `/etc/registry/certs/requests/` (mode 644)
- **Server Certificates**: `/etc/registry/certs/` (mode 644) - **Server Certificates**: `/etc/registry/certs/` (mode 644)
- **Docker Trust Store**: `/etc/docker/certs.d/YOUR_ACTUAL_IP_ADDRESS/` - **Podman Certificates**: `/etc/containers/certs.d/YOUR_ACTUAL_IP_ADDRESS/`
### Step 6: Install Forgejo Actions Runner ### Step 6: Install Forgejo Actions Runner
@ -1288,60 +1342,60 @@ sudo journalctl -u forgejo-runner.service -f --no-pager
- Check network: Ensure the runner can reach your Forgejo instance - Check network: Ensure the runner can reach your Forgejo instance
- Restart service: `sudo systemctl restart forgejo-runner.service` - Restart service: `sudo systemctl restart forgejo-runner.service`
### Step 7: Set Up Docker-in-Docker (DinD) for CI Operations ### Step 7: Set Up Podman-in-Podman (PiP) for CI Operations
**Important**: This step sets up a Docker-in-Docker container that provides an isolated environment for CI/CD operations, eliminating resource contention with Docker Registry and simplifying cleanup. **Important**: This step sets up a Podman-in-Podman container that provides an isolated environment for CI/CD operations, eliminating resource contention with Podman Registry and simplifying cleanup.
#### 7.1 Create Containerized CI/CD Environment #### 7.1 Create Containerized CI/CD Environment
```bash ```bash
# Switch to CI_SERVICE_USER (who has Docker group access) # Switch to CI_SERVICE_USER (who has Podman access)
sudo su - CI_SERVICE_USER sudo su - CI_SERVICE_USER
# Navigate to the application directory # Navigate to the application directory
cd /opt/APP_NAME cd /opt/APP_NAME
# Start DinD container for isolated Docker operations # Start PiP container for isolated Podman operations
docker run -d \ podman run -d \
--name ci-dind \ --name ci-pip \
--privileged \ --privileged \
-p 2375:2375 \ -p 2375:2375 \
-e DOCKER_TLS_CERTDIR="" \ -e DOCKER_TLS_CERTDIR="" \
docker:dind quay.io/podman/stable:latest
# Wait for a minute or two for DinD to be ready (wait for Docker daemon inside DinD) # Wait for a minute or two for PiP to be ready (wait for Podman daemon inside PiP)
# Test DinD connectivity # Test PiP connectivity
docker exec ci-dind docker version podman exec ci-pip podman version
``` ```
**What this does**: **What this does**:
- **Creates isolated DinD environment**: Provides isolated Docker environment for all CI/CD operations - **Creates isolated PiP environment**: Provides isolated Podman environment for all CI/CD operations
- **Health checks**: Ensures DinD is fully ready before proceeding - **Health checks**: Ensures PiP is fully ready before proceeding
- **Simple setup**: Direct Docker commands for maximum flexibility - **Simple setup**: Direct Podman commands for maximum flexibility
**Why CI_SERVICE_USER**: The CI_SERVICE_USER is in the docker group and runs the CI pipeline, so it needs direct access to the DinD container for seamless CI/CD operations. **Why CI_SERVICE_USER**: The CI_SERVICE_USER has Podman access and runs the CI pipeline, so it needs direct access to the PiP container for seamless CI/CD operations.
#### 7.2 Configure DinD for Docker Registry #### 7.2 Configure PiP for Docker Registry v2
```bash ```bash
# Navigate to the application directory # Navigate to the application directory
cd /opt/APP_NAME cd /opt/APP_NAME
# Login to Docker Registry from within DinD (using authenticated port 4443) # Login to Docker Registry v2 from within PiP (using authenticated port 4443)
echo "your-registry-password" | docker exec -i ci-dind docker login YOUR_CI_CD_IP:4443 -u registry-user --password-stdin echo "your-registry-password" | podman exec -i ci-pip podman login YOUR_CI_CD_IP:4443 -u registry-user --password-stdin
# Test Docker Registry connectivity from DinD # Test Docker Registry v2 connectivity from PiP
docker exec ci-dind docker pull alpine:latest podman exec ci-pip podman pull alpine:latest
docker exec ci-dind docker tag alpine:latest YOUR_CI_CD_IP:4443/APP_NAME/test:latest podman exec ci-pip podman tag alpine:latest YOUR_CI_CD_IP:4443/APP_NAME/test:latest
docker exec ci-dind docker push YOUR_CI_CD_IP:4443/APP_NAME/test:latest podman exec ci-pip podman push YOUR_CI_CD_IP:4443/APP_NAME/test:latest
# Test unauthenticated pull from standard port 443 # Test unauthenticated pull from standard port 443
docker exec ci-dind docker pull YOUR_CI_CD_IP/APP_NAME/test:latest podman exec ci-pip podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
# Clean up test images # Clean up test images
docker exec ci-dind docker rmi YOUR_CI_CD_IP:4443/APP_NAME/test:latest podman exec ci-pip podman rmi YOUR_CI_CD_IP:4443/APP_NAME/test:latest
docker exec ci-dind docker rmi YOUR_CI_CD_IP/APP_NAME/test:latest podman exec ci-pip podman rmi YOUR_CI_CD_IP/APP_NAME/test:latest
#### 7.3 Set Up Workspace Directory #### 7.3 Set Up Workspace Directory
@ -1378,10 +1432,12 @@ ls -la /tmp/ci-workspace
The Docker Registry setup now follows the Filesystem Hierarchy Standard (FHS) for better organization and security: The Docker Registry setup now follows the Filesystem Hierarchy Standard (FHS) for better organization and security:
**Application Files** (in `/opt/APP_NAME/registry/`): **Application Files** (in `/opt/APP_NAME/registry/`):
- `docker-compose.registry.yml` - Docker Compose configuration from project repository - `docker-registry.yml` - Podman Compose configuration for Docker Registry v2 and nginx
- `registry-pod.yaml` - Kubernetes Pod manifest for Docker Registry v2 and nginx
- `nginx.conf` - nginx reverse proxy configuration from project repository - `nginx.conf` - nginx reverse proxy configuration from project repository
- `openssl.conf` - OpenSSL configuration for certificate generation from project repository - `openssl.conf` - OpenSSL configuration for certificate generation from project repository
- `docker-registry.service` - Systemd service file for Docker Registry - `containers-policy.json` - Container policy for image signature verification
- `docker-registry.service` - Systemd service file for Docker Registry v2
**System Files** (FHS-compliant locations): **System Files** (FHS-compliant locations):
- `/var/lib/registry/data/` - Registry data storage - `/var/lib/registry/data/` - Registry data storage
@ -1397,11 +1453,12 @@ The Docker Registry setup now follows the Filesystem Hierarchy Standard (FHS) fo
**Benefits of FHS Compliance**: **Benefits of FHS Compliance**:
- **Data persistence**: Registry data stored in `/var/lib/registry/data/` survives container restarts - **Data persistence**: Registry data stored in `/var/lib/registry/data/` survives container restarts
- **Certificate security**: Hierarchical certificate structure with proper permissions - **Certificate security**: Hierarchical certificate structure with proper permissions
- **Environment security**: Secrets stored in `/etc/registry/env/` with restrictive permissions (600) - **Authentication security**: htpasswd file stored in `/etc/registry/auth/` with restrictive permissions (600)
- **Service management**: Systemd service for proper startup, shutdown, and monitoring - **Service management**: Systemd service for proper startup, shutdown, and monitoring
- **Separation of concerns**: Private keys isolated from public certificates, secrets isolated from configs - **Separation of concerns**: Private keys isolated from public certificates, auth isolated from configs
- **Log management**: Logs in `/var/log/registry/` for centralized logging - **Log management**: Logs in `/var/log/nginx/` for centralized logging
- **Configuration separation**: App configs in app directory, system data in system directories - **Configuration separation**: App configs in app directory, system data in system directories
- **Policy enforcement**: Container policies for image signature verification
``` ```
**What this does**: **What this does**:
@ -1423,7 +1480,7 @@ The CI/CD pipeline uses a three-stage approach with dedicated environments for e
- Rust toolchain for backend testing and migrations - Rust toolchain for backend testing and migrations
- Node.js toolchain for frontend testing - Node.js toolchain for frontend testing
- **Network**: All containers communicate through `ci-cd-test-network` - **Network**: All containers communicate through `ci-cd-test-network`
- **Setup**: DinD container created, Docker Registry login performed, code cloned into DinD from Forgejo - **Setup**: PiP container created, Docker Registry v2 login performed, code cloned into PiP from Forgejo
- **Cleanup**: Testing containers removed, DinD container kept running - **Cleanup**: Testing containers removed, DinD container kept running
**Job 2 (Building) - Direct Docker Commands:** **Job 2 (Building) - Direct Docker Commands:**
@ -1543,28 +1600,28 @@ sudo ufw allow 443/tcp # Docker Registry via nginx (public read access)
### Step 9: Test CI/CD Setup ### Step 9: Test CI/CD Setup
#### 9.1 Test Docker Installation #### 9.1 Test Podman Installation
```bash ```bash
docker --version podman --version
docker compose version podman-compose --version
``` ```
#### 9.2 Check Docker Registry Status #### 9.2 Check Docker Registry v2 Status
```bash ```bash
cd /opt/APP_NAME/registry cd /opt/APP_NAME/registry
docker compose -f docker-compose.registry.yml ps podman pod ps
``` ```
#### 9.3 Test Docker Registry Access #### 9.3 Test Docker Registry v2 Access
```bash ```bash
# Test Docker Registry API # Test Docker Registry v2 API
curl -k https://localhost/v2/_catalog curl -k https://localhost:443/v2/_catalog
# Test Docker Registry UI # Test Docker Registry v2 UI
curl -k -I https://localhost curl -k -I https://localhost:443
``` ```
--- ---
@ -1806,26 +1863,25 @@ pwd
exit exit
``` ```
### Step 12: Install Docker ### Step 12: Install Podman
#### 12.1 Add Docker Repository #### 12.1 Install Podman
```bash ```bash
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg # Install Podman and related tools
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt install -y podman podman-compose
sudo apt update
# Verify installation
podman --version
podman-compose --version
``` ```
#### 12.2 Install Docker Packages #### 12.2 Configure Podman for Production Service Account
```bash ```bash
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin # Podman can run rootless, so no group membership needed
``` # But we'll configure it for the production service account
sudo usermod -aG podman PROD_SERVICE_USER
#### 12.3 Configure Docker for Production Service Account
```bash
sudo usermod -aG docker PROD_SERVICE_USER
``` ```
#### 12.4 Create Application Directory #### 12.4 Create Application Directory
@ -1845,18 +1901,18 @@ ls -la /opt/APP_NAME
- Sets proper ownership for the PROD_SERVICE_USER - Sets proper ownership for the PROD_SERVICE_USER
- Ensures the directory exists before the CI workflow runs - Ensures the directory exists before the CI workflow runs
### Step 13: Configure Docker for Docker Registry Access ### Step 13: Configure Podman for Docker Registry v2 Access
**Important**: The Production Linode needs to be able to pull Docker images from the Docker Registry on the CI/CD Linode. Since we're using nginx with automatic HTTPS, no additional certificate configuration is needed. **Important**: The Production Linode needs to be able to pull images from the Docker Registry v2 on the CI/CD Linode. Since we're using nginx with automatic HTTPS, no additional certificate configuration is needed.
```bash ```bash
# Change to the PROD_SERVICE_USER # Change to the PROD_SERVICE_USER
sudo su - PROD_SERVICE_USER sudo su - PROD_SERVICE_USER
# Test that Docker can pull images from the Docker Registry (unauthenticated port 443) # Test that Podman can pull images from the Docker Registry v2 (unauthenticated port 443)
docker pull YOUR_CI_CD_IP/APP_NAME/test:latest podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
# If the pull succeeds, the Docker Registry is accessible for production deployments # If the pull succeeds, the Docker Registry v2 is accessible for production deployments
# Change back to PROD_DEPLOY_USER # Change back to PROD_DEPLOY_USER
exit exit
@ -1867,7 +1923,7 @@ exit
**Note**: Production deployments use unauthenticated pulls from port 443, while CI/CD operations use authenticated pushes to port 4443. **Note**: Production deployments use unauthenticated pulls from port 443, while CI/CD operations use authenticated pushes to port 4443.
**What this does**: **What this does**:
- **Tests Docker Registry access**: Verifies that Docker can successfully pull images from the Docker Registry - **Tests Docker Registry v2 access**: Verifies that Podman can successfully pull images from the Docker Registry v2
- **No certificate configuration needed**: nginx handles HTTPS automatically - **No certificate configuration needed**: nginx handles HTTPS automatically
- **Simple setup**: No complex certificate management required - **Simple setup**: No complex certificate management required
@ -2154,18 +2210,18 @@ sudo fail2ban-client status
### Step 16: Test Production Setup ### Step 16: Test Production Setup
#### 16.1 Test Docker Installation #### 16.1 Test Podman Installation
```bash ```bash
docker --version podman --version
docker compose --version podman-compose --version
``` ```
#### 16.2 Test Docker Registry Access #### 16.2 Test Docker Registry v2 Access
```bash ```bash
# Test pulling an image from the CI/CD Docker Registry (unauthenticated port 443) # Test pulling an image from the CI/CD Docker Registry v2 (unauthenticated port 443)
docker pull YOUR_CI_CD_IP/APP_NAME/test:latest podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
``` ```
**Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address. **Important**: Replace `YOUR_CI_CD_IP` with your actual CI/CD Linode IP address.
@ -2189,10 +2245,10 @@ Go to your Forgejo repository and add these secrets in **Settings → Secrets an
- `PROD_SERVICE_USER`: The production service user name (e.g., `prod-service`) - `PROD_SERVICE_USER`: The production service user name (e.g., `prod-service`)
- `APP_NAME`: Your application name (e.g., `sharenet`) - `APP_NAME`: Your application name (e.g., `sharenet`)
- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database - `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database
- `REGISTRY_USER`: Docker Registry username for CI operations (e.g., `registry-user`) - `REGISTRY_USER`: Docker Registry v2 username for CI operations (e.g., `registry-user`)
- `REGISTRY_PASSWORD`: Docker Registry password for CI operations (the password you set in the nginx configuration, default: `your-secure-registry-password`) - `REGISTRY_PASSWORD`: Docker Registry v2 password for CI operations (the password you set in the nginx configuration, default: `your-secure-registry-password`)
- `REGISTRY_PUSH_URL`: Docker Registry URL for authenticated pushes (e.g., `YOUR_CI_CD_IP:4443`) - `REGISTRY_PUSH_URL`: Docker Registry v2 URL for authenticated pushes (e.g., `YOUR_CI_CD_IP:4443`)
- `REGISTRY_PULL_URL`: Docker Registry URL for unauthenticated pulls (e.g., `YOUR_CI_CD_IP`) - `REGISTRY_PULL_URL`: Docker Registry v2 URL for unauthenticated pulls (e.g., `YOUR_CI_CD_IP`)
**Optional Secrets (for domain users):** **Optional Secrets (for domain users):**
- `DOMAIN`: Your domain name (e.g., `example.com`) - `DOMAIN`: Your domain name (e.g., `example.com`)
@ -2221,7 +2277,7 @@ The pipeline should execute these steps in order:
7. **Push to Registry**: Push images to Docker Registry from DinD 7. **Push to Registry**: Push images to Docker Registry from DinD
8. **Deploy to Production**: Deploy to production server 8. **Deploy to Production**: Deploy to production server
#### 18.3 Check Docker Registry #### 18.3 Check Docker Registry v2
```bash ```bash
# On CI/CD Linode # On CI/CD Linode
@ -2248,16 +2304,16 @@ curl -k https://YOUR_CI_CD_IP:4443/v2/_catalog
# On Production Linode # On Production Linode
cd /opt/APP_NAME cd /opt/APP_NAME
# Check if containers are running with new images # Check if pods are running with new images
docker compose -f docker-compose.prod.yml ps podman pod ps
# Check application health # Check application health
curl http://localhost:3000 curl http://localhost:3000
curl http://localhost:3001/health curl http://localhost:3001/health
# Check container logs for any errors # Check container logs for any errors
docker compose -f docker-compose.prod.yml logs backend podman logs sharenet-production-pod-backend
docker compose -f docker-compose.prod.yml logs frontend podman logs sharenet-production-pod-frontend
``` ```
#### 18.5 Test Application Functionality #### 18.5 Test Application Functionality
@ -2396,8 +2452,10 @@ sudo rm -rf /opt/APP_NAME/registry/openssl.conf
sudo rm -rf /opt/APP_NAME/registry/certs/requests/openssl.conf sudo rm -rf /opt/APP_NAME/registry/certs/requests/openssl.conf
# Note: DO NOT remove these files as they are needed for operation: # Note: DO NOT remove these files as they are needed for operation:
# - /opt/APP_NAME/registry/docker-compose.registry.yml # - /opt/APP_NAME/registry/docker-registry.yml
# - /opt/APP_NAME/registry/nginx.conf # - /opt/APP_NAME/registry/registry-pod.yaml
# - /opt/APP_NAME/registry/nginx.conf
# - /opt/APP_NAME/registry/containers-policy.json
# - /opt/APP_NAME/registry/docker-registry.service # - /opt/APP_NAME/registry/docker-registry.service
# - /etc/registry/auth/.htpasswd (contains the actual secrets) # - /etc/registry/auth/.htpasswd (contains the actual secrets)
# - /etc/systemd/system/docker-registry.service # - /etc/systemd/system/docker-registry.service

121
production-pod.yaml Normal file
View file

@ -0,0 +1,121 @@
apiVersion: v1
kind: Pod
metadata:
name: sharenet-production-pod
labels:
app: sharenet-production
spec:
containers:
- name: postgres
image: postgres:15-alpine
env:
- name: POSTGRES_DB
value: "sharenet"
- name: POSTGRES_USER
value: "sharenet"
- name: POSTGRES_PASSWORD
value: "changeme"
ports:
- containerPort: 5432
protocol: TCP
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
livenessProbe:
exec:
command:
- pg_isready
- -U
- sharenet
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 10
failureThreshold: 3
readinessProbe:
exec:
command:
- pg_isready
- -U
- sharenet
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
- name: backend
image: localhost:4443/sharenet/backend:latest
env:
- name: DATABASE_URL
value: "postgresql://sharenet:changeme@localhost:5432/sharenet"
- name: RUST_LOG
value: "info"
- name: RUST_BACKTRACE
value: "1"
ports:
- containerPort: 3001
protocol: TCP
livenessProbe:
httpGet:
path: /health
port: 3001
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 3001
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
- name: frontend
image: localhost:4443/sharenet/frontend:latest
env:
- name: NEXT_PUBLIC_API_HOST
value: "localhost"
- name: NEXT_PUBLIC_API_PORT
value: "3001"
- name: NODE_ENV
value: "production"
ports:
- containerPort: 3000
protocol: TCP
dependsOn:
- name: backend
condition: Ready
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
protocol: TCP
- containerPort: 443
protocol: TCP
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
readOnly: true
- name: nginx-ssl
mountPath: /etc/nginx/ssl
readOnly: true
dependsOn:
- name: frontend
condition: Ready
- name: backend
condition: Ready
volumes:
- name: postgres-data
hostPath:
path: /var/lib/postgresql/data
type: Directory
- name: nginx-config
hostPath:
path: /opt/sharenet/nginx/nginx.conf
type: File
- name: nginx-ssl
hostPath:
path: /opt/sharenet/nginx/ssl
type: Directory

View file

@ -4,9 +4,9 @@ This folder contains the configuration files for the Docker Registry setup used
## Files ## Files
- **`docker-compose.registry.yml`**: Docker Compose configuration for the registry and nginx reverse proxy - **`docker-registry.yml`**: Podman Compose configuration for Docker Registry v2 and nginx reverse proxy
- **`nginx.conf`**: nginx configuration for HTTPS and authentication - **`nginx.conf`**: nginx configuration for HTTPS and authentication
- **`docker-registry.service`**: Systemd service file for Docker Registry - **`docker-registry.service`**: Systemd service file for Docker Registry v2
- **`README.md`**: This documentation file - **`README.md`**: This documentation file
## Architecture ## Architecture
@ -49,10 +49,33 @@ The setup is configured through:
The registry is automatically set up during the CI/CD pipeline setup process. The configuration files are copied from this folder to the registry server and customized with the appropriate IP address and credentials. All files and running services should be owned by `CI_SERVICE_USER`. The registry is automatically set up during the CI/CD pipeline setup process. The configuration files are copied from this folder to the registry server and customized with the appropriate IP address and credentials. All files and running services should be owned by `CI_SERVICE_USER`.
## Security ## Security Features
- Authentication is handled by nginx using htpasswd file ### Container Security
- HTTPS is automatically managed by nginx - **Rootless operation**: Containers run as non-root user (UID 1000)
- Registry data is persisted in Docker volumes - **Capability dropping**: All Linux capabilities are dropped
- Environment file contains sensitive credentials and should be properly secured - **Privilege escalation**: Disabled via allowPrivilegeEscalation=false
- All files and services are owned by `CI_SERVICE_USER` (not a separate registry user) - **Read-only filesystem**: Root filesystem is read-only with tmpfs for /tmp
- **Image deletion disabled**: REGISTRY_STORAGE_DELETE_ENABLED=false
### Network Security
- **TLS 1.2/1.3 only**: Modern cipher suites with HSTS headers
- **Rate limiting**: 10 requests/second for reads, 5 requests/second for writes
- **Client size limits**: 2GB max body size for large image uploads
- **Internal registry**: Registry listens only internally, proxied via nginx
- **Port restrictions**: Only ports 443 and 4443 exposed
### Resource Management
- **CPU limits**: 1000m for registry, 500m for nginx
- **Memory limits**: 1Gi for registry, 512Mi for nginx
- **File descriptors**: Proper ulimits configuration
### Authentication & Authorization
- **Basic auth**: htpasswd-based authentication for write operations
- **Policy enforcement**: containers-policy.json for image signature verification
- **Volume security**: Read-only mounts where possible with nosuid,nodev,noexec
### Data Protection
- **FHS compliance**: Proper directory structure and permissions
- **Credential isolation**: htpasswd file stored separately with 600 permissions
- **Log management**: Structured logging with proper volume mounts

View file

@ -0,0 +1,33 @@
{
"default": [
{
"type": "reject"
}
],
"transports": {
"docker": {
"localhost:443": [
{
"type": "insecureAcceptAnything"
}
],
"localhost:4443": [
{
"type": "insecureAcceptAnything"
}
],
"docker.io": [
{
"type": "insecureAcceptAnything"
}
]
},
"docker-daemon": {
"": [
{
"type": "insecureAcceptAnything"
}
]
}
}
}

View file

@ -1,32 +0,0 @@
services:
registry:
image: registry:2
container_name: registry
restart: unless-stopped
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /var/lib/registry
# Optional but recommended if you want to be able to delete images:
REGISTRY_STORAGE_DELETE_ENABLED: "true"
# Listen only inside the compose network
REGISTRY_HTTP_ADDR: 0.0.0.0:5000
volumes:
- ./registry:/var/lib/registry
expose:
- "5000" # internal only, not published
nginx:
image: nginx:alpine
container_name: nginx
restart: unless-stopped
depends_on:
- registry
ports:
- "443:443" # HTTPS only
- "4443:4443"
# deliberately no "80:80" no HTTP
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- /etc/registry/certs:/etc/registry/certs:ro
- /etc/registry/auth/.htpasswd:/etc/nginx/.htpasswd:ro
- /var/log/nginx:/var/log/nginx

View file

@ -1,7 +1,6 @@
[Unit] [Unit]
Description=Docker Registry with Caddy Reverse Proxy Description=Docker Registry v2 with nginx Reverse Proxy
After=docker.service After=network.target
Requires=docker.service
[Service] [Service]
Type=oneshot Type=oneshot
@ -9,11 +8,9 @@ RemainAfterExit=yes
User=CI_SERVICE_USER User=CI_SERVICE_USER
Group=CI_SERVICE_USER Group=CI_SERVICE_USER
WorkingDirectory=/opt/APP_NAME/registry WorkingDirectory=/opt/APP_NAME/registry
EnvironmentFile=/etc/registry/env/.env ExecStart=/usr/bin/podman play kube registry-pod.yaml
Environment=DOCKER_CONFIG=/etc/docker ExecStop=/usr/bin/podman pod stop registry-pod
ExecStart=/usr/bin/docker compose -f docker-compose.registry.yml up -d ExecReload=/usr/bin/podman pod restart registry-pod
ExecStop=/usr/bin/docker compose -f docker-compose.registry.yml down
ExecReload=/usr/bin/docker compose -f docker-compose.registry.yml restart
TimeoutStartSec=0 TimeoutStartSec=0
# Security settings # Security settings

View file

@ -0,0 +1,66 @@
services:
registry:
image: registry@sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d
container_name: registry
restart: unless-stopped
user: "1000:1000"
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /var/lib/registry
# Disable image deletion for security
REGISTRY_STORAGE_DELETE_ENABLED: "false"
# Listen only inside the compose network
REGISTRY_HTTP_ADDR: 0.0.0.0:5000
# TLS configuration
REGISTRY_HTTP_TLS_CERTIFICATE: /etc/registry/certs/registry.crt
REGISTRY_HTTP_TLS_KEY: /etc/registry/certs/private/registry.key
volumes:
- ./registry:/var/lib/registry
- /etc/registry/certs:/etc/registry/certs:ro
expose:
- "5000" # internal only, not published
deploy:
resources:
limits:
cpus: '1.0'
memory: 1G
reservations:
cpus: '0.5'
memory: 512M
security_opt:
- no-new-privileges:true
- seccomp:unconfined
read_only: true
tmpfs:
- /tmp:noexec,nosuid,size=100m
nginx:
image: nginx@sha256:6650513efd1d27c1f8a5351cbd33edf85cc7e3b73dc4d4d4e8f8c0b3d0b3d0b3d
container_name: nginx
restart: unless-stopped
user: "1000:1000"
depends_on:
- registry
ports:
- "443:443" # HTTPS only
- "4443:4443"
# deliberately no "80:80" no HTTP
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- /etc/registry/certs:/etc/registry/certs:ro
- /etc/registry/auth/.htpasswd:/etc/nginx/.htpasswd:ro
- /var/log/nginx:/var/log/nginx
- ./containers-policy.json:/etc/containers/policy.json:ro
deploy:
resources:
limits:
cpus: '0.5'
memory: 512M
reservations:
cpus: '0.25'
memory: 256M
security_opt:
- no-new-privileges:true
- seccomp:unconfined
read_only: true
tmpfs:
- /tmp:noexec,nosuid,size=100m

View file

@ -1,10 +1,22 @@
# Docker Registry Nginx Configuration # Docker Registry v2 Nginx Configuration with Enhanced Security
# Port 443: Unauthenticated pulls (GET requests only) # Port 443: Unauthenticated pulls (GET requests only)
# Port 4443: Authenticated operations (login, logout, push, delete, etc.) # Port 4443: Authenticated operations (login, logout, push, delete, etc.)
# Rate limiting with different zones for different operations
limit_req_zone $binary_remote_addr zone=registry:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=push:10m rate=5r/s;
limit_req_zone $binary_remote_addr zone=login:10m rate=2r/s;
# Connection limiting
limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;
# Upstream Docker Registry # Upstream Docker Registry
upstream registry { upstream registry {
server registry:5000; server registry:5000;
keepalive 32;
# Health check
keepalive_requests 100;
keepalive_timeout 60s;
} }
# HTTP server for unauthenticated pulls on port 443 # HTTP server for unauthenticated pulls on port 443
@ -12,26 +24,48 @@ server {
listen 443 ssl http2; listen 443 ssl http2;
server_name _; server_name _;
# SSL Configuration # Connection limits
limit_conn conn_limit_per_ip 10;
# SSL Configuration - TLS 1.2/1.3 only with modern ciphers
ssl_certificate /etc/registry/certs/registry.crt; ssl_certificate /etc/registry/certs/registry.crt;
ssl_certificate_key /etc/registry/certs/private/registry.key; ssl_certificate_key /etc/registry/certs/private/registry.key;
ssl_protocols TLSv1.2 TLSv1.3; ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384; ssl_ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers off; ssl_prefer_server_ciphers off;
ssl_session_cache shared:SSL:10m; ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m; ssl_session_timeout 10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
# Security headers # Security headers with HSTS
add_header X-Frame-Options DENY; add_header X-Frame-Options DENY always;
add_header X-Content-Type-Options nosniff; add_header X-Content-Type-Options nosniff always;
add_header X-XSS-Protection "1; mode=block"; add_header X-XSS-Protection "1; mode=block" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header Content-Security-Policy "default-src 'none'; frame-ancestors 'none';" always;
add_header Permissions-Policy "geolocation=(), microphone=(), camera=()" always;
# Client max body size for large image uploads
client_max_body_size 2G;
client_body_timeout 60s;
client_header_timeout 60s;
# Rate limiting for read operations
limit_req zone=registry burst=20 nodelay;
# Block all write operations explicitly # Block all write operations explicitly
if ($request_method !~ ^(GET|HEAD)$) { if ($request_method !~ ^(GET|HEAD)$) {
return 405 "Method Not Allowed"; return 405 "Method Not Allowed";
} }
# Block suspicious user agents
if ($http_user_agent ~* (bot|crawler|spider|scraper)) {
return 403 "Forbidden";
}
# Allow all GET requests to v2 API (Docker Registry itself will handle security) # Allow all GET requests to v2 API (Docker Registry itself will handle security)
location /v2/ { location /v2/ {
proxy_pass http://registry; proxy_pass http://registry;
@ -42,21 +76,39 @@ server {
proxy_read_timeout 900; proxy_read_timeout 900;
proxy_connect_timeout 60; proxy_connect_timeout 60;
proxy_send_timeout 60; proxy_send_timeout 60;
proxy_buffering off;
proxy_request_buffering off;
# Additional security headers for registry
proxy_hide_header X-Powered-By;
proxy_hide_header Server;
} }
# Health check endpoint
location /health { location /health {
access_log off; access_log off;
return 200 "healthy\n"; return 200 "healthy\n";
add_header Content-Type text/plain; add_header Content-Type text/plain;
add_header Cache-Control "no-cache, no-store, must-revalidate";
} }
# Default location - deny all # Block access to hidden files
location / { location ~ /\. {
return 404; deny all;
access_log off;
log_not_found off;
} }
# Logging # Block access to sensitive files
location ~* \.(htaccess|htpasswd|ini|log|sh|sql|conf)$ {
deny all;
access_log off;
log_not_found off;
}
location / {
return 404;
}
access_log /var/log/nginx/registry_access.log; access_log /var/log/nginx/registry_access.log;
error_log /var/log/nginx/registry_error.log; error_log /var/log/nginx/registry_error.log;
} }
@ -66,25 +118,61 @@ server {
listen 4443 ssl http2; listen 4443 ssl http2;
server_name _; server_name _;
# SSL Configuration # Connection limits
limit_conn conn_limit_per_ip 5;
# SSL Configuration - TLS 1.2/1.3 only with modern ciphers
ssl_certificate /etc/registry/certs/registry.crt; ssl_certificate /etc/registry/certs/registry.crt;
ssl_certificate_key /etc/registry/certs/private/registry.key; ssl_certificate_key /etc/registry/certs/private/registry.key;
ssl_protocols TLSv1.2 TLSv1.3; ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384; ssl_ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers off; ssl_prefer_server_ciphers off;
ssl_session_cache shared:SSL:10m; ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m; ssl_session_timeout 10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
# Security headers # Security headers with HSTS
add_header X-Frame-Options DENY; add_header X-Frame-Options DENY always;
add_header X-Content-Type-Options nosniff; add_header X-Content-Type-Options nosniff always;
add_header X-XSS-Protection "1; mode=block"; add_header X-XSS-Protection "1; mode=block" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header Content-Security-Policy "default-src 'none'; frame-ancestors 'none';" always;
add_header Permissions-Policy "geolocation=(), microphone=(), camera=()" always;
# Client max body size for large image uploads
client_max_body_size 2G;
client_body_timeout 60s;
client_header_timeout 60s;
# Rate limiting for write operations
limit_req zone=push burst=10 nodelay;
# Rate limiting for login attempts
location ~ ^/v2/.*/blobs/uploads/ {
limit_req zone=login burst=3 nodelay;
auth_basic "Docker Registry v2";
auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://registry;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;
proxy_connect_timeout 60;
proxy_send_timeout 60;
proxy_buffering off;
proxy_request_buffering off;
proxy_hide_header X-Powered-By;
proxy_hide_header Server;
}
# Basic authentication for write operations # Basic authentication for write operations
location ~ ^/v2/.*$ { location ~ ^/v2/.*$ {
# Require auth for all v2 API operations # Require auth for all v2 API operations
auth_basic "Docker Registry"; auth_basic "Docker Registry v2";
auth_basic_user_file /etc/nginx/.htpasswd; auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://registry; proxy_pass http://registry;
@ -95,14 +183,30 @@ server {
proxy_read_timeout 900; proxy_read_timeout 900;
proxy_connect_timeout 60; proxy_connect_timeout 60;
proxy_send_timeout 60; proxy_send_timeout 60;
proxy_buffering off;
proxy_request_buffering off;
proxy_hide_header X-Powered-By;
proxy_hide_header Server;
} }
# Default location - deny all # Block access to hidden files
location / { location ~ /\. {
return 404; deny all;
access_log off;
log_not_found off;
} }
# Logging # Block access to sensitive files
location ~* \.(htaccess|htpasswd|ini|log|sh|sql|conf)$ {
deny all;
access_log off;
log_not_found off;
}
location / {
return 404;
}
access_log /var/log/nginx/registry_auth_access.log; access_log /var/log/nginx/registry_auth_access.log;
error_log /var/log/nginx/registry_auth_error.log; error_log /var/log/nginx/registry_auth_error.log;
} }

169
registry/registry-pod.yaml Normal file
View file

@ -0,0 +1,169 @@
apiVersion: v1
kind: Pod
metadata:
name: registry-pod
labels:
app: registry
security: hardened
spec:
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
seccompProfile:
type: RuntimeDefault
# Additional security hardening
sysctls:
- name: net.ipv4.ip_forward
value: "0"
- name: net.ipv4.conf.all.forwarding
value: "0"
- name: net.ipv4.conf.default.forwarding
value: "0"
containers:
- name: registry
image: registry@sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
env:
- name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
value: "/var/lib/registry"
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "false"
- name: REGISTRY_HTTP_ADDR
value: "0.0.0.0:5000"
- name: REGISTRY_HTTP_TLS_CERTIFICATE
value: "/etc/registry/certs/registry.crt"
- name: REGISTRY_HTTP_TLS_KEY
value: "/etc/registry/certs/private/registry.key"
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "1000m"
memory: "1Gi"
volumeMounts:
- name: registry-data
mountPath: /var/lib/registry
readOnly: false
- name: registry-certs
mountPath: /etc/registry/certs
readOnly: true
- name: tmp-volume
mountPath: /tmp
readOnly: false
ports:
- containerPort: 5000
protocol: TCP
livenessProbe:
httpGet:
path: /v2/
port: 5000
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /v2/
port: 5000
initialDelaySeconds: 5
periodSeconds: 5
- name: nginx
image: nginx@sha256:6650513efd1d27c1f8a5351cbd33edf85cc7e3b73dc4d4d4e8f8c0b3d0b3d0b3d
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
# Additional security hardening
procMount: Default
seccompProfile:
type: RuntimeDefault
ports:
- containerPort: 443
protocol: TCP
- containerPort: 4443
protocol: TCP
resources:
requests:
cpu: "250m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
readOnly: true
- name: registry-certs
mountPath: /etc/registry/certs
readOnly: true
- name: registry-auth
mountPath: /etc/nginx/.htpasswd
subPath: .htpasswd
readOnly: true
- name: nginx-logs
mountPath: /var/log/nginx
readOnly: false
- name: tmp-volume
mountPath: /tmp
readOnly: false
livenessProbe:
httpGet:
path: /health
port: 443
scheme: HTTPS
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /health
port: 443
scheme: HTTPS
initialDelaySeconds: 5
periodSeconds: 5
volumes:
- name: registry-data
hostPath:
path: /var/lib/registry/data
type: Directory
- name: nginx-config
hostPath:
path: /opt/APP_NAME/registry/nginx.conf
type: File
- name: registry-certs
hostPath:
path: /etc/registry/certs
type: Directory
- name: registry-auth
hostPath:
path: /etc/registry/auth
type: Directory
- name: nginx-logs
hostPath:
path: /var/log/nginx
type: Directory
- name: tmp-volume
emptyDir:
medium: Memory
sizeLimit: "100Mi"

77
test-pod.yaml Normal file
View file

@ -0,0 +1,77 @@
apiVersion: v1
kind: Pod
metadata:
name: ci-cd-test-pod
labels:
app: ci-cd-test
spec:
containers:
- name: postgres
image: postgres:15-alpine
env:
- name: POSTGRES_DB
value: "sharenet_test"
- name: POSTGRES_USER
value: "postgres"
- name: POSTGRES_PASSWORD
value: "password"
ports:
- containerPort: 5432
protocol: TCP
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
command: ["postgres"]
args: ["-c", "log_statement=all"]
livenessProbe:
exec:
command:
- pg_isready
- -U
- postgres
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
exec:
command:
- pg_isready
- -U
- postgres
initialDelaySeconds: 5
periodSeconds: 5
- name: rust-toolchain
image: rust:1.75-slim
volumeMounts:
- name: workspace
mountPath: /workspace
workingDir: /workspace/backend
command: ["sleep"]
args: ["infinity"]
env:
- name: CI_HOST
value: "localhost"
- name: APP_NAME
value: "sharenet"
- name: node-toolchain
image: node:20-slim
volumeMounts:
- name: workspace
mountPath: /workspace
workingDir: /workspace/frontend
command: ["sleep"]
args: ["infinity"]
env:
- name: CI_HOST
value: "localhost"
- name: APP_NAME
value: "sharenet"
volumes:
- name: postgres-data
emptyDir: {}
- name: workspace
hostPath:
path: /tmp/ci-workspace
type: Directory