Further migration from docker to podman
Some checks failed
CI/CD Pipeline (Forgejo Container Registry) / Run Tests (DinD) (push) Failing after 0s
CI/CD Pipeline (Forgejo Container Registry) / Build and Push Docker Images (DinD) (push) Failing after 0s
CI/CD Pipeline (Forgejo Container Registry) / Deploy to Production (push) Has been skipped
Some checks failed
CI/CD Pipeline (Forgejo Container Registry) / Run Tests (DinD) (push) Failing after 0s
CI/CD Pipeline (Forgejo Container Registry) / Build and Push Docker Images (DinD) (push) Failing after 0s
CI/CD Pipeline (Forgejo Container Registry) / Deploy to Production (push) Has been skipped
This commit is contained in:
parent
eb6e373981
commit
0131412aaa
2 changed files with 62 additions and 308 deletions
|
@ -577,7 +577,7 @@ sudo apt install -y \
|
||||||
|
|
||||||
#### 1.5 Install Podman
|
#### 1.5 Install Podman
|
||||||
|
|
||||||
**Note**: For detailed Podman installation and configuration, see the [Docker Registry Install Guide](Docker_Registry_Install_Guide.md#step-1-install-podman-if-not-already-installed).
|
**Note**: Podman is required for container operations and will be installed in the following steps.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Install Podman and related tools
|
# Install Podman and related tools
|
||||||
|
@ -595,7 +595,7 @@ sudo usermod --add-subuids 100000-165535 CI_SERVICE_USER
|
||||||
sudo usermod --add-subgids 100000-165535 CI_SERVICE_USER
|
sudo usermod --add-subgids 100000-165535 CI_SERVICE_USER
|
||||||
```
|
```
|
||||||
|
|
||||||
**What this does**: Installs Podman and configures it for rootless operation, which is needed for the CI pipeline and Docker Registry operations.
|
**What this does**: Installs Podman and configures it for rootless operation, which is needed for the CI pipeline and Forgejo Container Registry operations.
|
||||||
|
|
||||||
### Step 2: Create Users
|
### Step 2: Create Users
|
||||||
|
|
||||||
|
@ -639,8 +639,8 @@ cd /opt
|
||||||
sudo git clone https://your-forgejo-instance/your-username/APP_NAME.git
|
sudo git clone https://your-forgejo-instance/your-username/APP_NAME.git
|
||||||
sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER APP_NAME/
|
sudo chown -R CI_SERVICE_USER:CI_SERVICE_USER APP_NAME/
|
||||||
|
|
||||||
# Verify the registry folder exists
|
# Verify the application directory structure
|
||||||
ls -la /opt/APP_NAME/registry/
|
ls -la /opt/APP_NAME/
|
||||||
```
|
```
|
||||||
|
|
||||||
**Important**: Replace `your-forgejo-instance`, `your-username`, and `APP_NAME` with your actual Forgejo instance URL, username, and application name.
|
**Important**: Replace `your-forgejo-instance`, `your-username`, and `APP_NAME` with your actual Forgejo instance URL, username, and application name.
|
||||||
|
@ -648,7 +648,7 @@ ls -la /opt/APP_NAME/registry/
|
||||||
**What this does**:
|
**What this does**:
|
||||||
- CI_DEPLOY_USER creates the directory structure and clones the repository
|
- CI_DEPLOY_USER creates the directory structure and clones the repository
|
||||||
- CI_SERVICE_USER owns all the files for security
|
- CI_SERVICE_USER owns all the files for security
|
||||||
- Registry configuration files are now available at `/opt/APP_NAME/registry/`
|
- Application configuration files are now available at `/opt/APP_NAME/`
|
||||||
|
|
||||||
### Step 4: Configure Forgejo Container Registry Access
|
### Step 4: Configure Forgejo Container Registry Access
|
||||||
|
|
||||||
|
@ -940,15 +940,11 @@ podman exec ci-pip mkdir -p /etc/containers
|
||||||
podman exec ci-pip tee /etc/containers/registries.conf > /dev/null << 'EOF'
|
podman exec ci-pip tee /etc/containers/registries.conf > /dev/null << 'EOF'
|
||||||
[[registry]]
|
[[registry]]
|
||||||
location = "YOUR_CI_CD_IP:4443"
|
location = "YOUR_CI_CD_IP:4443"
|
||||||
client_cert = "/etc/registry/certs/clients/client.crt"
|
# Forgejo Container Registry uses Personal Access Tokens for authentication
|
||||||
client_key = "/etc/registry/certs/private/client.key"
|
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
# Copy client certificates to PiP container
|
# Copy client certificates to PiP container
|
||||||
podman cp /etc/registry/certs/clients/client.crt ci-pip:/etc/registry/certs/clients/
|
# Forgejo Container Registry authentication is handled through environment variables
|
||||||
podman cp /etc/registry/certs/private/client.key ci-pip:/etc/registry/certs/private/
|
|
||||||
podman exec ci-pip chmod 600 /etc/registry/certs/private/client.key
|
|
||||||
podman exec ci-pip chmod 644 /etc/registry/certs/clients/client.crt
|
|
||||||
|
|
||||||
# Test Forgejo Container Registry connectivity from PiP
|
# Test Forgejo Container Registry connectivity from PiP
|
||||||
podman exec ci-pip podman pull alpine:latest
|
podman exec ci-pip podman pull alpine:latest
|
||||||
|
@ -997,29 +993,19 @@ ls -la /tmp/ci-workspace
|
||||||
|
|
||||||
The Forgejo Container Registry setup uses the built-in registry functionality, providing secure and integrated container image storage.
|
The Forgejo Container Registry setup uses the built-in registry functionality, providing secure and integrated container image storage.
|
||||||
|
|
||||||
**Application Files** (in `/opt/APP_NAME/registry/`):
|
**Application Files** (in `/opt/APP_NAME/`):
|
||||||
- `containers-policy.json` - Container policy for image signature verification
|
- Configuration files for the application
|
||||||
- `openssl.conf` - OpenSSL configuration for certificate generation from project repository
|
- Nginx configuration for reverse proxy
|
||||||
|
|
||||||
**System Files** (FHS-compliant locations):
|
**System Files** (FHS-compliant locations):
|
||||||
- `/var/lib/registry/` - Registry data storage
|
- `/var/lib/` - Application data storage
|
||||||
- `/etc/registry/certs/` - SSL/TLS certificate hierarchy:
|
- `/etc/nginx/` - Nginx configuration
|
||||||
- `/etc/registry/certs/private/` - Private keys (mode 600)
|
|
||||||
- `/etc/registry/certs/ca/` - CA certificates (mode 644)
|
|
||||||
- `/etc/registry/certs/clients/` - Client CA certificates (mode 640)
|
|
||||||
- `/etc/registry/certs/requests/` - Certificate requests and configs (mode 644)
|
|
||||||
- `/etc/registry/certs/registry.crt` - Server certificate (mode 644)
|
|
||||||
- `/etc/registry/nginx.conf` - nginx reverse proxy configuration (mode 644)
|
|
||||||
- `/etc/systemd/user/registry.service` - Systemd user service for rootless registry
|
|
||||||
- `/etc/systemd/system/registry-proxy.service` - Systemd system service for TLS proxy
|
|
||||||
- `/var/log/nginx/` - nginx proxy logs
|
- `/var/log/nginx/` - nginx proxy logs
|
||||||
|
|
||||||
**Benefits of FHS Compliance**:
|
**Benefits of FHS Compliance**:
|
||||||
- **Data persistence**: Registry data stored in `/var/lib/registry/` survives container restarts
|
- **Data persistence**: Application data stored in proper locations survives restarts
|
||||||
- **Certificate security**: Hierarchical certificate structure with proper permissions
|
- **Service management**: Proper separation of application components
|
||||||
- **mTLS authentication**: Client certificates for secure push operations
|
- **Log management**: Centralized logging for easier troubleshooting
|
||||||
- **Service management**: Systemd services for proper startup, shutdown, and monitoring
|
|
||||||
- **Separation of concerns**: Private keys isolated from public certificates
|
|
||||||
- **Log management**: Logs in `/var/log/nginx/` for centralized logging
|
- **Log management**: Logs in `/var/log/nginx/` for centralized logging
|
||||||
- **Configuration separation**: App configs in app directory, system data in system directories
|
- **Configuration separation**: App configs in app directory, system data in system directories
|
||||||
- **Policy enforcement**: Container policies for image signature verification
|
- **Policy enforcement**: Container policies for image signature verification
|
||||||
|
@ -1034,7 +1020,7 @@ The Forgejo Container Registry setup uses the built-in registry functionality, p
|
||||||
|
|
||||||
The CI/CD pipeline uses a three-stage approach with dedicated environments for each stage:
|
The CI/CD pipeline uses a three-stage approach with dedicated environments for each stage:
|
||||||
|
|
||||||
**Job 1 (Testing) - `docker-compose.test.yml`:**
|
**Job 1 (Testing) - `ci-pod.yaml`:**
|
||||||
- **Purpose**: Comprehensive testing with multiple containers
|
- **Purpose**: Comprehensive testing with multiple containers
|
||||||
- **Environment**: DinD with PostgreSQL, Rust, and Node.js containers
|
- **Environment**: DinD with PostgreSQL, Rust, and Node.js containers
|
||||||
- **Code Checkout**: Code is checked out directly into the DinD container at `/workspace` from the Forgejo repository that triggered the build
|
- **Code Checkout**: Code is checked out directly into the DinD container at `/workspace` from the Forgejo repository that triggered the build
|
||||||
|
@ -1057,7 +1043,7 @@ The CI/CD pipeline uses a three-stage approach with dedicated environments for e
|
||||||
- **Registry Access**: Reuses Forgejo Container Registry authentication from Job 1
|
- **Registry Access**: Reuses Forgejo Container Registry authentication from Job 1
|
||||||
- **Cleanup**: DinD container stopped and removed (clean slate for next run)
|
- **Cleanup**: DinD container stopped and removed (clean slate for next run)
|
||||||
|
|
||||||
**Job 3 (Deployment) - `docker-compose.prod.yml`:**
|
**Job 3 (Deployment) - `prod-pod.yaml`:**
|
||||||
- **Purpose**: Production deployment with pre-built images
|
- **Purpose**: Production deployment with pre-built images
|
||||||
- **Environment**: Production runner on Production Linode
|
- **Environment**: Production runner on Production Linode
|
||||||
- **Process**:
|
- **Process**:
|
||||||
|
@ -1070,7 +1056,7 @@ The CI/CD pipeline uses a three-stage approach with dedicated environments for e
|
||||||
- **🧹 Complete Isolation**: Each job has its own dedicated environment
|
- **🧹 Complete Isolation**: Each job has its own dedicated environment
|
||||||
- **🚫 No Resource Contention**: Testing and building don't interfere with Forgejo Container Registry
|
- **🚫 No Resource Contention**: Testing and building don't interfere with Forgejo Container Registry
|
||||||
- **⚡ Consistent Environment**: Same setup every time
|
- **⚡ Consistent Environment**: Same setup every time
|
||||||
- **🎯 Purpose-Specific**: Each Docker Compose file serves a specific purpose
|
- **🎯 Purpose-Specific**: Each pod configuration serves a specific purpose
|
||||||
- **🔄 Parallel Safety**: Jobs can run safely in parallel
|
- **🔄 Parallel Safety**: Jobs can run safely in parallel
|
||||||
|
|
||||||
**Testing DinD Setup:**
|
**Testing DinD Setup:**
|
||||||
|
@ -1099,7 +1085,7 @@ docker exec ci-dind docker rmi YOUR_CI_CD_IP/APP_NAME/dind-test:latest
|
||||||
|
|
||||||
#### 6.5 Production Deployment Architecture
|
#### 6.5 Production Deployment Architecture
|
||||||
|
|
||||||
The production deployment uses a separate Docker Compose file (`docker-compose.prod.yml`) that pulls built images from the Forgejo Container Registry and deploys the complete application stack.
|
The production deployment uses a separate pod configuration (`prod-pod.yaml`) that pulls built images from the Forgejo Container Registry and deploys the complete application stack.
|
||||||
|
|
||||||
**Production Stack Components:**
|
**Production Stack Components:**
|
||||||
- **PostgreSQL**: Production database with persistent storage
|
- **PostgreSQL**: Production database with persistent storage
|
||||||
|
@ -1110,11 +1096,11 @@ The production deployment uses a separate Docker Compose file (`docker-compose.p
|
||||||
**Deployment Flow:**
|
**Deployment Flow:**
|
||||||
1. **Production Runner**: Runs on Production Linode with `production` label
|
1. **Production Runner**: Runs on Production Linode with `production` label
|
||||||
2. **Image Pull**: Pulls latest images from Forgejo Container Registry
|
2. **Image Pull**: Pulls latest images from Forgejo Container Registry
|
||||||
3. **Stack Deployment**: Uses `docker-compose.prod.yml` to deploy complete stack
|
3. **Stack Deployment**: Uses `prod-pod.yaml` to deploy complete stack
|
||||||
4. **Health Verification**: Ensures all services are healthy before completion
|
4. **Health Verification**: Ensures all services are healthy before completion
|
||||||
|
|
||||||
**Key Benefits:**
|
**Key Benefits:**
|
||||||
- **🔄 Image Registry**: Centralized image storage in Docker Registry
|
- **🔄 Image Registry**: Centralized image storage in Forgejo Container Registry
|
||||||
- **📦 Consistent Deployment**: Same images tested in CI are deployed to production
|
- **📦 Consistent Deployment**: Same images tested in CI are deployed to production
|
||||||
- **⚡ Fast Deployment**: Only pulls changed images
|
- **⚡ Fast Deployment**: Only pulls changed images
|
||||||
- **🛡️ Rollback Capability**: Can easily rollback to previous image versions
|
- **🛡️ Rollback Capability**: Can easily rollback to previous image versions
|
||||||
|
@ -1153,11 +1139,11 @@ sudo ufw --force enable
|
||||||
sudo ufw default deny incoming
|
sudo ufw default deny incoming
|
||||||
sudo ufw default allow outgoing
|
sudo ufw default allow outgoing
|
||||||
sudo ufw allow ssh
|
sudo ufw allow ssh
|
||||||
sudo ufw allow 443/tcp # Docker Registry via nginx (public read access)
|
# Note: Forgejo Container Registry access is configured through Forgejo itself
|
||||||
```
|
```
|
||||||
|
|
||||||
**Security Model**:
|
**Security Model**:
|
||||||
- **Port 443 (Docker Registry)**: Public read access, authenticated write access
|
- **Forgejo Registry**: Integrated authentication and authorization
|
||||||
- **SSH**: Restricted to your IP addresses
|
- **SSH**: Restricted to your IP addresses
|
||||||
- **All other ports**: Blocked
|
- **All other ports**: Blocked
|
||||||
|
|
||||||
|
@ -1169,21 +1155,18 @@ sudo ufw allow 443/tcp # Docker Registry via nginx (public read access)
|
||||||
podman --version
|
podman --version
|
||||||
```
|
```
|
||||||
|
|
||||||
#### 8.2 Check Docker Registry v2 Status
|
#### 8.2 Check Forgejo Container Registry Status
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd /opt/APP_NAME/registry
|
cd /opt/APP_NAME
|
||||||
podman pod ps
|
podman pod ps
|
||||||
```
|
```
|
||||||
|
|
||||||
#### 8.3 Test Docker Registry v2 Access
|
#### 8.3 Test Forgejo Container Registry Access
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Test Docker Registry v2 API
|
# Test Forgejo Container Registry access
|
||||||
curl -k https://localhost:443/v2/_catalog
|
# Registry access is handled through your Forgejo instance's web interface
|
||||||
|
|
||||||
# Test Docker Registry v2 UI
|
|
||||||
curl -k -I https://localhost:443
|
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
@ -1461,9 +1444,9 @@ ls -la /opt/APP_NAME
|
||||||
- Sets proper ownership for the PROD_SERVICE_USER
|
- Sets proper ownership for the PROD_SERVICE_USER
|
||||||
- Ensures the directory exists before the CI workflow runs
|
- Ensures the directory exists before the CI workflow runs
|
||||||
|
|
||||||
### Step 13: Configure Podman for Docker Registry v2 Access
|
### Step 13: Configure Podman for Forgejo Container Registry Access
|
||||||
|
|
||||||
**Important**: The Production Linode needs to be able to pull images from the Docker Registry v2 on the CI/CD Linode. Since we're using nginx with automatic HTTPS, no additional certificate configuration is needed.
|
**Important**: The Production Linode needs to be able to pull images from the Forgejo Container Registry. Authentication is handled through the CI/CD pipeline configuration.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Change to the PROD_SERVICE_USER
|
# Change to the PROD_SERVICE_USER
|
||||||
|
@ -1485,7 +1468,7 @@ exit
|
||||||
|
|
||||||
### Step 14: Set Up Forgejo Runner for Production Deployment
|
### Step 14: Set Up Forgejo Runner for Production Deployment
|
||||||
|
|
||||||
**Important**: The Production Linode needs a Forgejo runner to execute the deployment job from the CI/CD workflow. This runner will pull images from Docker Registry and deploy using `docker-compose.prod.yml`.
|
**Important**: The Production Linode needs a Forgejo runner to execute the deployment job from the CI/CD workflow. This runner will pull images from Forgejo Container Registry and deploy using the production pod configuration.
|
||||||
|
|
||||||
#### 14.1 Download Runner
|
#### 14.1 Download Runner
|
||||||
|
|
||||||
|
@ -1636,19 +1619,19 @@ sudo journalctl -u forgejo-runner.service -f --no-pager
|
||||||
|
|
||||||
When the workflow runs, it will:
|
When the workflow runs, it will:
|
||||||
|
|
||||||
1. Pull the latest Docker images from Docker Registry
|
1. Pull the latest Docker images from Forgejo Container Registry
|
||||||
2. Use the `docker-compose.prod.yml` file to deploy the application stack
|
2. Use the `prod-pod.yaml` file to deploy the application stack
|
||||||
3. Create the necessary environment variables for production deployment
|
3. Create the necessary environment variables for production deployment
|
||||||
4. Verify that all services are healthy after deployment
|
4. Verify that all services are healthy after deployment
|
||||||
|
|
||||||
The production runner will automatically handle the deployment process when you push to the main branch.
|
The production runner will automatically handle the deployment process when you push to the main branch.
|
||||||
|
|
||||||
#### 14.6 Understanding the Production Docker Compose Setup
|
#### 14.6 Understanding the Production Pod Setup
|
||||||
|
|
||||||
The `docker-compose.prod.yml` file is specifically designed for production deployment and differs from development setups:
|
The `prod-pod.yaml` file is specifically designed for production deployment and uses Kubernetes pod specifications:
|
||||||
|
|
||||||
**Key Features**:
|
**Key Features**:
|
||||||
- **Image-based deployment**: Uses pre-built images from Docker Registry instead of building from source
|
- **Image-based deployment**: Uses pre-built images from Forgejo Container Registry instead of building from source
|
||||||
- **Production networking**: All services communicate through a dedicated `sharenet-network`
|
- **Production networking**: All services communicate through a dedicated `sharenet-network`
|
||||||
- **Health checks**: Each service includes health checks to ensure proper startup order
|
- **Health checks**: Each service includes health checks to ensure proper startup order
|
||||||
- **Nginx reverse proxy**: Includes Nginx for SSL termination, load balancing, and security headers
|
- **Nginx reverse proxy**: Includes Nginx for SSL termination, load balancing, and security headers
|
||||||
|
@ -1662,9 +1645,9 @@ The `docker-compose.prod.yml` file is specifically designed for production deplo
|
||||||
4. **Nginx**: Reverse proxy that serves the frontend and proxies API requests to backend
|
4. **Nginx**: Reverse proxy that serves the frontend and proxies API requests to backend
|
||||||
|
|
||||||
**Deployment Process**:
|
**Deployment Process**:
|
||||||
1. The production runner pulls the latest images from Docker Registry
|
1. The production runner pulls the latest images from Forgejo Container Registry
|
||||||
2. Creates environment variables for the deployment
|
2. Creates environment variables for the deployment
|
||||||
3. Runs `docker compose -f docker-compose.prod.yml up -d`
|
3. Runs `podman play kube prod-pod.yaml`
|
||||||
4. Waits for all services to be healthy
|
4. Waits for all services to be healthy
|
||||||
5. Verifies the deployment was successful
|
5. Verifies the deployment was successful
|
||||||
|
|
||||||
|
@ -1772,10 +1755,10 @@ sudo fail2ban-client status
|
||||||
podman --version
|
podman --version
|
||||||
```
|
```
|
||||||
|
|
||||||
#### 16.2 Test Docker Registry v2 Access
|
#### 16.2 Test Forgejo Container Registry Access
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Test pulling an image from the CI/CD Docker Registry v2 (unauthenticated port 443)
|
# Test pulling an image from Forgejo Container Registry
|
||||||
podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
|
podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -1794,20 +1777,20 @@ podman pull YOUR_CI_CD_IP/APP_NAME/test:latest
|
||||||
Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**:
|
Go to your Forgejo repository and add these secrets in **Settings → Secrets and Variables → Actions**:
|
||||||
|
|
||||||
**Required Secrets:**
|
**Required Secrets:**
|
||||||
- `CI_HOST`: Your CI/CD Linode IP address (used for Docker Registry access)
|
- `CI_HOST`: Your CI/CD Linode IP address (used for testing environment)
|
||||||
- `PRODUCTION_IP`: Your Production Linode IP address
|
- `PRODUCTION_IP`: Your Production Linode IP address
|
||||||
- `PROD_DEPLOY_USER`: The production deployment user name (e.g., `prod-deploy`)
|
- `PROD_DEPLOY_USER`: The production deployment user name (e.g., `prod-deploy`)
|
||||||
- `PROD_SERVICE_USER`: The production service user name (e.g., `prod-service`)
|
- `PROD_SERVICE_USER`: The production service user name (e.g., `prod-service`)
|
||||||
- `APP_NAME`: Your application name (e.g., `sharenet`)
|
- `APP_NAME`: Your application name (e.g., `sharenet`)
|
||||||
- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database
|
- `POSTGRES_PASSWORD`: A strong password for the PostgreSQL database
|
||||||
- `REGISTRY_CLIENT_CERT`: Path to client certificate for mTLS authentication (e.g., `/etc/registry/certs/clients/client.crt`)
|
- `REGISTRY_HOST`: Your Forgejo instance's registry URL
|
||||||
- `REGISTRY_CLIENT_KEY`: Path to client private key for mTLS authentication (e.g., `/etc/registry/certs/private/client.key`)
|
- `REGISTRY_TOKEN`: Personal Access Token with package write permissions
|
||||||
|
|
||||||
**Note**: The CI pipeline now uses mTLS authentication for pushes (port 4443) and Cosign for image signing. The registry policy enforces Sigstore signatures for all images consumed from the registry.
|
**Note**: The CI pipeline now uses mTLS authentication for pushes (port 4443) and Cosign for image signing. The registry policy enforces Sigstore signatures for all images consumed from the registry.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**Note**: This setup uses custom Dockerfiles for testing environments with base images stored in Docker Registry. The CI pipeline automatically checks if base images exist in Docker Registry and pulls them from Docker Hub only when needed, eliminating rate limiting issues and providing better control over the testing environment.
|
**Note**: This setup uses custom Dockerfiles for testing environments with base images. The CI pipeline automatically checks if base images exist in Forgejo Container Registry and pulls them from Docker Hub only when needed, eliminating rate limiting issues and providing better control over the testing environment.
|
||||||
|
|
||||||
### Step 18: Test Complete Pipeline
|
### Step 18: Test Complete Pipeline
|
||||||
|
|
||||||
|
@ -1827,10 +1810,10 @@ The pipeline should execute these steps in order:
|
||||||
4. **Test Frontend**: Run frontend tests in isolated environment
|
4. **Test Frontend**: Run frontend tests in isolated environment
|
||||||
5. **Build Backend**: Build backend Docker image in DinD
|
5. **Build Backend**: Build backend Docker image in DinD
|
||||||
6. **Build Frontend**: Build frontend Docker image in DinD
|
6. **Build Frontend**: Build frontend Docker image in DinD
|
||||||
7. **Push to Registry**: Push images to Docker Registry from DinD
|
7. **Push to Registry**: Push images to Forgejo Container Registry from DinD
|
||||||
8. **Deploy to Production**: Deploy to production server
|
8. **Deploy to Production**: Deploy to production server
|
||||||
|
|
||||||
#### 18.3 Check Docker Registry v2
|
#### 18.3 Check Forgejo Container Registry
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# On CI/CD Linode
|
# On CI/CD Linode
|
||||||
|
@ -1954,7 +1937,7 @@ tail -f /tmp/monitor.log
|
||||||
|
|
||||||
## Forgejo Container Registry Setup
|
## Forgejo Container Registry Setup
|
||||||
|
|
||||||
This repository has been configured to use **Forgejo's built-in container registry** instead of a custom Docker Registry. This provides a simpler, more integrated solution while maintaining the same Podman Pods CI approach.
|
This repository uses **Forgejo's built-in container registry** which provides a simpler, more integrated solution while maintaining the same Podman Pods CI approach.
|
||||||
|
|
||||||
### Prerequisites
|
### Prerequisites
|
||||||
|
|
||||||
|
@ -2011,23 +1994,6 @@ For example:
|
||||||
- `forgejo.example.com/devteam/sharenet/backend:abc123`
|
- `forgejo.example.com/devteam/sharenet/backend:abc123`
|
||||||
- `forgejo.example.com/devteam/sharenet/frontend:abc123`
|
- `forgejo.example.com/devteam/sharenet/frontend:abc123`
|
||||||
|
|
||||||
### Migration from Custom Registry
|
|
||||||
|
|
||||||
If you were previously using the custom Docker Registry setup:
|
|
||||||
|
|
||||||
1. **Remove old registry artifacts**:
|
|
||||||
- Delete `Docker_Registry_Install_Guide.md`
|
|
||||||
- Remove `registry/` folder
|
|
||||||
- Update any references to old registry paths
|
|
||||||
|
|
||||||
2. **Update secrets**:
|
|
||||||
- Replace `CI_HOST`, `REGISTRY_USER`, `REGISTRY_PASSWORD` with new Forgejo registry secrets
|
|
||||||
- Add `REGISTRY_HOST`, `REGISTRY_USERNAME`, `REGISTRY_TOKEN`
|
|
||||||
|
|
||||||
3. **Update deployment**:
|
|
||||||
- Production environment will now pull from Forgejo registry
|
|
||||||
- No changes needed to Dockerfiles or application code
|
|
||||||
|
|
||||||
### Benefits of Forgejo Container Registry
|
### Benefits of Forgejo Container Registry
|
||||||
|
|
||||||
- ✅ **Simplified setup** - No custom registry installation required
|
- ✅ **Simplified setup** - No custom registry installation required
|
||||||
|
@ -2074,13 +2040,13 @@ podman image inspect REGISTRY_HOST/OWNER_REPO/backend:TAG
|
||||||
You have successfully set up a complete CI/CD pipeline with:
|
You have successfully set up a complete CI/CD pipeline with:
|
||||||
|
|
||||||
- ✅ **Automated testing** on every code push in isolated DinD environment
|
- ✅ **Automated testing** on every code push in isolated DinD environment
|
||||||
- ✅ **Docker image building** and Docker Registry storage
|
- ✅ **Docker image building** and Forgejo Container Registry storage
|
||||||
- ✅ **Automated deployment** to production
|
- ✅ **Automated deployment** to production
|
||||||
- ✅ **Health monitoring** and logging
|
- ✅ **Health monitoring** and logging
|
||||||
- ✅ **Backup and cleanup** automation
|
- ✅ **Backup and cleanup** automation
|
||||||
- ✅ **Security hardening** with proper user separation
|
- ✅ **Security hardening** with proper user separation
|
||||||
- ✅ **SSL/TLS support** with self-signed certificates and mTLS authentication
|
- ✅ **SSL/TLS support** with self-signed certificates and mTLS authentication
|
||||||
- ✅ **Zero resource contention** between CI/CD and Docker Registry
|
- ✅ **Zero resource contention** between CI/CD and Forgejo Container Registry
|
||||||
- ✅ **FHS-compliant directory structure** for better organization and security
|
- ✅ **FHS-compliant directory structure** for better organization and security
|
||||||
- ✅ **Robust rootless services** via systemd user manager
|
- ✅ **Robust rootless services** via systemd user manager
|
||||||
- ✅ **Host TLS reverse proxy** with rootless registry isolation
|
- ✅ **Host TLS reverse proxy** with rootless registry isolation
|
||||||
|
@ -2091,44 +2057,29 @@ Your application is now ready for continuous deployment with proper security, mo
|
||||||
|
|
||||||
After successful setup, you can clean up the installation files to remove sensitive information:
|
After successful setup, you can clean up the installation files to remove sensitive information:
|
||||||
|
|
||||||
```bash
|
**Security Note**: Forgejo Container Registry uses built-in authentication. Ensure your Personal Access Tokens are stored securely and never committed to version control.
|
||||||
# Remove installation files (optional - for security)
|
- **Rotate tokens regularly** by generating new Personal Access Tokens
|
||||||
sudo rm -rf /opt/APP_NAME/registry/openssl.conf
|
- **Use minimal permissions** - only grant `write:packages` scope for CI/CD operations
|
||||||
sudo rm -rf /opt/APP_NAME/registry/certs/requests/openssl.conf
|
|
||||||
|
|
||||||
# Note: DO NOT remove these files as they are needed for operation:
|
|
||||||
# - /opt/APP_NAME/registry/containers-policy.json
|
|
||||||
# - /etc/registry/certs/ (contains all certificates and keys)
|
|
||||||
# - /etc/systemd/user/registry.service
|
|
||||||
# - /etc/systemd/system/registry-proxy.service
|
|
||||||
# - /etc/registry/nginx.conf
|
|
||||||
```
|
|
||||||
|
|
||||||
**Security Note**: The certificate files in `/etc/registry/certs/` contain sensitive authentication data and should be:
|
|
||||||
- **Backed up securely** if needed for disaster recovery
|
|
||||||
- **Never committed to version control**
|
|
||||||
- **Protected with proper permissions** (600 for private keys, 640 for client CA)
|
|
||||||
- **Rotated regularly** by regenerating certificates and updating client configurations
|
|
||||||
|
|
||||||
### Step 7.4 CI/CD Workflow Summary Table
|
### Step 7.4 CI/CD Workflow Summary Table
|
||||||
|
|
||||||
| Stage | What Runs | How/Where |
|
| Stage | What Runs | How/Where |
|
||||||
|---------|--------------------------|--------------------------|
|
|---------|--------------------------|--------------------------|
|
||||||
| Test | All integration/unit tests| `docker-compose.test.yml`|
|
| Test | All integration/unit tests| `ci-pod.yaml` |
|
||||||
| Build | Build & push images | Direct Docker commands |
|
| Build | Build & push images | Podman build & push |
|
||||||
| Deploy | Deploy to production | `docker-compose.prod.yml`|
|
| Deploy | Deploy to production | `prod-pod.yaml` |
|
||||||
|
|
||||||
**How it works:**
|
**How it works:**
|
||||||
- **Test:** The workflow spins up a full test environment using `docker-compose.test.yml` (Postgres, backend, frontend, etc.) and runs all tests inside containers.
|
- **Test:** The workflow spins up a full test environment using `ci-pod.yaml` (Postgres, backend, frontend, etc.) and runs all tests inside containers.
|
||||||
- **Build:** If tests pass, the workflow uses direct Docker commands (no compose file) to build backend and frontend images and push them to Docker Registry.
|
- **Build:** If tests pass, the workflow uses Podman to build backend and frontend images and push them to Forgejo Container Registry.
|
||||||
- **Deploy:** The production runner pulls images from Docker Registry and deploys the stack using `docker-compose.prod.yml`.
|
- **Deploy:** The production runner pulls images from Forgejo Container Registry and deploys the stack using `prod-pod.yaml`.
|
||||||
|
|
||||||
**Expected Output:**
|
**Expected Output:**
|
||||||
- Each stage runs in its own isolated environment.
|
- Each stage runs in its own isolated environment.
|
||||||
- Test failures stop the pipeline before any images are built or deployed.
|
- Test failures stop the pipeline before any images are built or deployed.
|
||||||
- Only tested images are deployed to production.
|
- Only tested images are deployed to production.
|
||||||
|
|
||||||
### Manual Testing with docker-compose.test.yml
|
### Manual Testing with Podman Pods
|
||||||
|
|
||||||
You can use the same test environment locally that the CI pipeline uses for integration testing. This is useful for debugging, development, or verifying your setup before pushing changes.
|
You can use the same test environment locally that the CI pipeline uses for integration testing. This is useful for debugging, development, or verifying your setup before pushing changes.
|
||||||
|
|
||||||
|
@ -2140,10 +2091,10 @@ For local development testing, you can run the test environment directly:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Start the test environment locally
|
# Start the test environment locally
|
||||||
docker compose -f docker-compose.test.yml up -d
|
podman play kube ci-pod.yaml
|
||||||
|
|
||||||
# Check service health
|
# Check service health
|
||||||
docker compose -f docker-compose.test.yml ps
|
podman pod ps
|
||||||
```
|
```
|
||||||
|
|
||||||
**Important**: This local setup is for development only. The CI pipeline uses a more isolated DinD environment.
|
**Important**: This local setup is for development only. The CI pipeline uses a more isolated DinD environment.
|
||||||
|
@ -2162,7 +2113,7 @@ docker exec ci-cd-test-node npm run test
|
||||||
#### Cleanup
|
#### Cleanup
|
||||||
When you're done, stop and remove all test containers:
|
When you're done, stop and remove all test containers:
|
||||||
```bash
|
```bash
|
||||||
docker compose -f docker-compose.test.yml down
|
podman pod stop ci-cd-test-pod && podman pod rm ci-cd-test-pod
|
||||||
```
|
```
|
||||||
|
|
||||||
**Tip:** The CI pipeline uses the same test containers but runs them inside a DinD environment for complete isolation.
|
**Tip:** The CI pipeline uses the same test containers but runs them inside a DinD environment for complete isolation.
|
|
@ -1,197 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
|
|
||||||
# Sharenet Local Deployment Script
|
|
||||||
# This script handles local development deployments
|
|
||||||
|
|
||||||
set -e
|
|
||||||
|
|
||||||
# Configuration
|
|
||||||
APP_NAME="sharenet"
|
|
||||||
DOCKER_COMPOSE_FILE="docker-compose.yml"
|
|
||||||
|
|
||||||
# Colors
|
|
||||||
RED='\033[0;31m'
|
|
||||||
GREEN='\033[0;32m'
|
|
||||||
YELLOW='\033[1;33m'
|
|
||||||
BLUE='\033[0;34m'
|
|
||||||
NC='\033[0m'
|
|
||||||
|
|
||||||
log_info() { echo -e "${BLUE}[INFO]${NC} $1"; }
|
|
||||||
log_success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; }
|
|
||||||
log_warning() { echo -e "${YELLOW}[WARNING]${NC} $1"; }
|
|
||||||
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
|
|
||||||
|
|
||||||
# Pre-deployment checks
|
|
||||||
pre_deployment_checks() {
|
|
||||||
log_info "Running pre-deployment checks..."
|
|
||||||
|
|
||||||
# Check if docker-compose.yml exists
|
|
||||||
if [ ! -f "$DOCKER_COMPOSE_FILE" ]; then
|
|
||||||
log_error "docker-compose.yml not found in current directory"
|
|
||||||
log_error "Current directory: $(pwd)"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Check if Docker is running
|
|
||||||
if ! docker info >/dev/null 2>&1; then
|
|
||||||
log_error "Docker is not running or not accessible"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Check if docker-compose is available
|
|
||||||
if ! command -v docker-compose >/dev/null 2>&1; then
|
|
||||||
log_error "docker-compose is not installed or not in PATH"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
log_success "Pre-deployment checks passed"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Pull latest code (if in git repository)
|
|
||||||
pull_latest_code() {
|
|
||||||
if [ -d ".git" ]; then
|
|
||||||
log_info "Pulling latest code from repository..."
|
|
||||||
if git pull origin main 2>/dev/null || git pull origin master 2>/dev/null; then
|
|
||||||
log_success "Code updated successfully"
|
|
||||||
else
|
|
||||||
log_warning "Could not pull latest code (not a git repository or no remote configured)"
|
|
||||||
fi
|
|
||||||
else
|
|
||||||
log_info "Not a git repository, skipping code pull"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# Pull latest images
|
|
||||||
pull_images() {
|
|
||||||
log_info "Pulling latest Docker images..."
|
|
||||||
if docker-compose pull; then
|
|
||||||
log_success "Images pulled successfully"
|
|
||||||
else
|
|
||||||
log_error "Failed to pull images"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# Stop existing containers
|
|
||||||
stop_containers() {
|
|
||||||
log_info "Stopping existing containers..."
|
|
||||||
if docker-compose down; then
|
|
||||||
log_success "Containers stopped successfully"
|
|
||||||
else
|
|
||||||
log_warning "Some containers may not have stopped cleanly"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# Start new containers
|
|
||||||
start_containers() {
|
|
||||||
log_info "Starting new containers..."
|
|
||||||
if docker-compose up -d; then
|
|
||||||
log_success "Containers started successfully"
|
|
||||||
else
|
|
||||||
log_error "Failed to start containers"
|
|
||||||
log_error "Recent logs:"
|
|
||||||
docker-compose logs --tail=20
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# Verify deployment
|
|
||||||
verify_deployment() {
|
|
||||||
log_info "Verifying deployment..."
|
|
||||||
|
|
||||||
# Wait for containers to start
|
|
||||||
sleep 10
|
|
||||||
|
|
||||||
# Check if containers are running
|
|
||||||
if docker-compose ps | grep -q "Up"; then
|
|
||||||
log_success "Deployment successful! All containers are running."
|
|
||||||
docker-compose ps
|
|
||||||
else
|
|
||||||
log_error "Deployment failed! Some containers are not running."
|
|
||||||
docker-compose ps
|
|
||||||
log_error "Recent logs:"
|
|
||||||
docker-compose logs --tail=20
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# Clean up old images
|
|
||||||
cleanup_images() {
|
|
||||||
log_info "Cleaning up old Docker images..."
|
|
||||||
if docker image prune -f; then
|
|
||||||
log_success "Old images cleaned up successfully"
|
|
||||||
else
|
|
||||||
log_warning "Image cleanup had some issues"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# Show service status
|
|
||||||
show_status() {
|
|
||||||
log_info "Current service status:"
|
|
||||||
docker-compose ps
|
|
||||||
echo
|
|
||||||
log_info "Recent logs:"
|
|
||||||
docker-compose logs --tail=10
|
|
||||||
}
|
|
||||||
|
|
||||||
# Main deployment process
|
|
||||||
main() {
|
|
||||||
local command="${1:-deploy}"
|
|
||||||
|
|
||||||
case "$command" in
|
|
||||||
deploy)
|
|
||||||
log_info "Starting local deployment for $APP_NAME..."
|
|
||||||
pre_deployment_checks
|
|
||||||
pull_latest_code
|
|
||||||
pull_images
|
|
||||||
stop_containers
|
|
||||||
start_containers
|
|
||||||
verify_deployment
|
|
||||||
cleanup_images
|
|
||||||
log_success "Local deployment completed successfully"
|
|
||||||
;;
|
|
||||||
status)
|
|
||||||
log_info "Checking service status..."
|
|
||||||
if [ -f "$DOCKER_COMPOSE_FILE" ]; then
|
|
||||||
show_status
|
|
||||||
else
|
|
||||||
log_error "docker-compose.yml not found in current directory"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
;;
|
|
||||||
restart)
|
|
||||||
log_info "Restarting services..."
|
|
||||||
pre_deployment_checks
|
|
||||||
stop_containers
|
|
||||||
start_containers
|
|
||||||
verify_deployment
|
|
||||||
log_success "Services restarted successfully"
|
|
||||||
;;
|
|
||||||
logs)
|
|
||||||
log_info "Showing recent logs..."
|
|
||||||
if [ -f "$DOCKER_COMPOSE_FILE" ]; then
|
|
||||||
docker-compose logs --tail=50 -f
|
|
||||||
else
|
|
||||||
log_error "docker-compose.yml not found in current directory"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
log_error "Unknown command: $command"
|
|
||||||
echo "Usage: $0 {deploy|status|restart|logs}"
|
|
||||||
echo ""
|
|
||||||
echo "Commands:"
|
|
||||||
echo " deploy - Deploy the application (default)"
|
|
||||||
echo " status - Show current service status"
|
|
||||||
echo " restart - Restart all services"
|
|
||||||
echo " logs - Show and follow recent logs"
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
# Handle interrupts gracefully
|
|
||||||
trap 'log_error "Deployment interrupted"; exit 1' INT TERM
|
|
||||||
|
|
||||||
# Run main function
|
|
||||||
main "$@"
|
|
Loading…
Add table
Reference in a new issue