Cloud Deployment

Deploy CLI Proxy API in cloud environments with dynamic configuration

CLI Proxy API supports cloud deployment mode with dynamic configuration loading, allowing you to deploy without an initial configuration file.

Cloud Deploy Mode

Set the DEPLOY environment variable to cloud to enable cloud deployment mode:

DEPLOY=cloud

In cloud mode, the server:

1. Starts without requiring a valid configuration file
2. Waits for configuration to be provided via mounted volumes or management API
3. Begins serving requests once configuration is detected

Docker Deployment

Using Docker Compose

1. Create docker-compose.yml

   docker-compose.yml

   services:
     cli-proxy-api:
       image: eceasy/cli-proxy-api:latest
       container_name: cli-proxy-api
       environment:
         DEPLOY: cloud
       ports:
         - "8317:8317"
       volumes:
         - config-volume:/CLIProxyAPI
       restart: unless-stopped
   
   volumes:
     config-volume:
       external: true
   
2. Create external volume

   docker volume create config-volume
   
3. Start the service

   docker compose up -d
   

   The service starts and waits for configuration:

   Cloud deploy mode: No configuration file detected; standing by for configuration
   
4. Mount configuration

   Copy your configuration file to the volume:

   docker cp config.yaml cli-proxy-api:/CLIProxyAPI/config.yaml
   docker compose restart
   

   The service detects the configuration and starts:

   Cloud deploy mode: Configuration file detected; starting service
   

Using Docker Run

docker run -d \
  --name cli-proxy-api \
  -e DEPLOY=cloud \
  -p 8317:8317 \
  -v config-volume:/CLIProxyAPI \
  eceasy/cli-proxy-api:latest

Configuration Detection

In cloud mode, the server checks for a valid configuration file at startup:

if isCloudDeploy {
    if info, errStat := os.Stat(configFilePath); errStat != nil {
        log.Info("Cloud deploy mode: No configuration file detected; standing by for configuration")
        configFileExists = false
    } else if info.IsDir() {
        log.Info("Cloud deploy mode: Config path is a directory; standing by for configuration")
        configFileExists = false
    } else if cfg.Port == 0 {
        log.Info("Cloud deploy mode: Configuration file is empty or invalid; standing by for valid configuration")
        configFileExists = false
    } else {
        log.Info("Cloud deploy mode: Configuration file detected; starting service")
        configFileExists = true
    }
}

The server validates:

1. Configuration file exists and is not a directory
2. Configuration file is not empty
3. Configuration contains a valid port setting

Environment Variables

Core Variables

Variable	Description	Default
DEPLOY	Deployment mode (cloud or empty)	(none)

Storage Backend Variables

See Storage Backends for remote storage configuration:

• PostgreSQL: PGSTORE_DSN, PGSTORE_SCHEMA, PGSTORE_LOCAL_PATH
• Git: GITSTORE_GIT_URL, GITSTORE_GIT_USERNAME, GITSTORE_GIT_TOKEN, GITSTORE_LOCAL_PATH
• Object Storage: OBJECTSTORE_ENDPOINT, OBJECTSTORE_BUCKET, OBJECTSTORE_ACCESS_KEY, OBJECTSTORE_SECRET_KEY, OBJECTSTORE_LOCAL_PATH

Kubernetes Deployment

ConfigMap Approach

1. Create ConfigMap

   configmap.yaml

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: cli-proxy-config
     namespace: default
   data:
     config.yaml: |
       port: 8317
       log_level: info
       # Your configuration here
   

   kubectl apply -f configmap.yaml
   
2. Create Deployment

   deployment.yaml

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: cli-proxy-api
     namespace: default
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: cli-proxy-api
     template:
       metadata:
         labels:
           app: cli-proxy-api
       spec:
         containers:
         - name: cli-proxy-api
           image: eceasy/cli-proxy-api:latest
           ports:
           - containerPort: 8317
           env:
           - name: DEPLOY
             value: "cloud"
           volumeMounts:
           - name: config
             mountPath: /CLIProxyAPI/config.yaml
             subPath: config.yaml
         volumes:
         - name: config
           configMap:
             name: cli-proxy-config
   

   kubectl apply -f deployment.yaml
   
3. Create Service

   service.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: cli-proxy-api
     namespace: default
   spec:
     selector:
       app: cli-proxy-api
     ports:
     - protocol: TCP
       port: 8317
       targetPort: 8317
     type: LoadBalancer
   

   kubectl apply -f service.yaml
   

Secret for Authentication

Store OAuth tokens in Kubernetes Secrets:

apiVersion: v1
kind: Secret
metadata:
  name: cli-proxy-auth
  namespace: default
type: Opaque
data:
  # Base64 encoded auth files
  gemini.json: <base64-encoded-content>

Mount the secret:

volumeMounts:
- name: auth
  mountPath: /root/.cli-proxy-api
volumes:
- name: auth
  secret:
    secretName: cli-proxy-auth

Configuration with Storage Backends

For production cloud deployments, use remote storage backends:

PostgreSQL Backend

docker-compose.yml

services:
  cli-proxy-api:
    image: eceasy/cli-proxy-api:latest
    environment:
      DEPLOY: cloud
      PGSTORE_DSN: postgresql://user:pass@postgres:5432/cliproxy
      PGSTORE_SCHEMA: public
    depends_on:
      - postgres

  postgres:
    image: postgres:15
    environment:
      POSTGRES_DB: cliproxy
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

Git Backend

services:
  cli-proxy-api:
    image: eceasy/cli-proxy-api:latest
    environment:
      DEPLOY: cloud
      GITSTORE_GIT_URL: https://github.com/your-org/cli-proxy-config.git
      GITSTORE_GIT_USERNAME: git-user
      GITSTORE_GIT_TOKEN: ghp_your_token
      GITSTORE_LOCAL_PATH: /data/cliproxy/gitstore
    volumes:
      - gitstore:/data/cliproxy/gitstore

volumes:
  gitstore:

Object Storage Backend

services:
  cli-proxy-api:
    image: eceasy/cli-proxy-api:latest
    environment:
      DEPLOY: cloud
      OBJECTSTORE_ENDPOINT: https://s3.amazonaws.com
      OBJECTSTORE_BUCKET: cli-proxy-config
      OBJECTSTORE_ACCESS_KEY: ${AWS_ACCESS_KEY}
      OBJECTSTORE_SECRET_KEY: ${AWS_SECRET_KEY}
      OBJECTSTORE_LOCAL_PATH: /data/cliproxy/objectstore
    volumes:
      - objectstore:/data/cliproxy/objectstore

volumes:
  objectstore:

Health Monitoring

Liveness Probe

livenessProbe:
  httpGet:
    path: /
    port: 8317
  initialDelaySeconds: 30
  periodSeconds: 10

Readiness Probe

readinessProbe:
  httpGet:
    path: /
    port: 8317
  initialDelaySeconds: 10
  periodSeconds: 5

Logging

Configure log output via configuration file:

config.yaml

log_level: info
log_output:
  - type: file
    path: /CLIProxyAPI/logs/app.log
    max_size_mb: 100
    max_backups: 3
  - type: stdout

In cloud environments, prefer stdout logging for integration with log aggregation systems:

log_output:
  - type: stdout

Scaling Considerations

Stateless Deployment

When using remote storage backends (PostgreSQL, Git, or Object Storage), the service becomes stateless and can be scaled horizontally:

spec:
  replicas: 3  # Multiple instances

Local File Storage

With local file storage, use single-instance deployments or shared volumes:

volumes:
- name: shared-config
  persistentVolumeClaim:
    claimName: cli-proxy-pvc

Troubleshooting

Service Won’t Start

Check logs for configuration issues:

docker compose logs cli-proxy-api

Common issues:

• Missing DEPLOY=cloud variable
• Invalid configuration file format
• Missing required configuration fields (e.g., port)

Configuration Not Detected

Verify file is mounted correctly:

docker exec cli-proxy-api ls -la /CLIProxyAPI/config.yaml
docker exec cli-proxy-api cat /CLIProxyAPI/config.yaml

Storage Backend Connection Failed

Check environment variables and network connectivity:

# PostgreSQL
docker exec cli-proxy-api env | grep PGSTORE

# Git
docker exec cli-proxy-api env | grep GITSTORE

# Object Storage
docker exec cli-proxy-api env | grep OBJECTSTORE

Next Steps

Storage Backends

Configure PostgreSQL, Git, or Object Storage backends

Configuration

Learn about available configuration options