Storage Backends - CLI Proxy API

CLI Proxy API supports multiple storage backends for configuration and authentication data, enabling distributed and cloud-native deployments.

Overview

By default, CLI Proxy API stores configuration and OAuth tokens in local files. For production deployments, you can use:

PostgreSQL - Centralized database storage
Git - Version-controlled configuration with remote repository sync
Object Storage - S3-compatible object storage

Storage backends are mutually exclusive. If multiple backends are configured, PostgreSQL takes precedence, followed by Object Storage, then Git.

PostgreSQL Store

Store configuration and authentication data in a PostgreSQL database.

Configuration

Set environment variables:

.env

PGSTORE_DSN=postgresql://user:password@localhost:5432/cliproxy
PGSTORE_SCHEMA=public
PGSTORE_LOCAL_PATH=/var/lib/cliproxy

Variable	Description	Required	Default
`PGSTORE_DSN`	PostgreSQL connection string	Yes	(none)
`PGSTORE_SCHEMA`	Database schema name	No	`public`
`PGSTORE_LOCAL_PATH`	Local mirror directory	No	`./pgstore`

Database Schema

The PostgreSQL store automatically creates two tables:

Config Table

CREATE TABLE IF NOT EXISTS config_store (
    id TEXT PRIMARY KEY,
    content TEXT NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Auth Table

CREATE TABLE IF NOT EXISTS auth_store (
    id TEXT PRIMARY KEY,
    content JSONB NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Docker Compose Example

docker-compose.yml

services:
  cli-proxy-api:
    image: eceasy/cli-proxy-api:latest
    environment:
      PGSTORE_DSN: postgresql://cliproxy:secure_password@postgres:5432/cliproxy
      PGSTORE_SCHEMA: public
      PGSTORE_LOCAL_PATH: /data/pgstore
    volumes:
      - pgstore:/data/pgstore
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped

  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: cliproxy
      POSTGRES_USER: cliproxy
      POSTGRES_PASSWORD: secure_password
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U cliproxy"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  pgstore:
  postgres_data:

Connection String Format

postgresql://[user[:password]@][host][:port][/dbname][?param1=value1&...]

Examples:

# Local PostgreSQL
PGSTORE_DSN=postgresql://cliproxy:password@localhost:5432/cliproxy

# With SSL
PGSTORE_DSN=postgresql://user:pass@db.example.com:5432/cliproxy?sslmode=require

# AWS RDS
PGSTORE_DSN=postgresql://admin:password@cliproxy.abc123.us-east-1.rds.amazonaws.com:5432/cliproxy

# Google Cloud SQL
PGSTORE_DSN=postgresql://user:pass@/cliproxy?host=/cloudsql/project:region:instance

How It Works

The PostgreSQL store:

Initialization (cmd/server/main.go:243-266):
- Connects to PostgreSQL using the provided DSN
- Creates schema and tables if they don’t exist
- Bootstraps configuration from example or database
Mirroring (internal/store/postgresstore.go:146-158):
- Syncs configuration from database to local workspace
- Mirrors auth files to local directory for file-based operations
- Maintains bidirectional sync between database and filesystem
Persistence (internal/store/postgresstore.go:188-260):
- Saves auth metadata to local files
- Upserts records to database with timestamps
- Handles updates and deletions

Git Store

Store configuration and authentication data in a Git repository with automatic sync.

Configuration

Set environment variables:

.env

GITSTORE_GIT_URL=https://github.com/your-org/cli-proxy-config.git
GITSTORE_GIT_USERNAME=git-user
GITSTORE_GIT_TOKEN=ghp_your_personal_access_token
GITSTORE_LOCAL_PATH=/data/cliproxy/gitstore

Variable	Description	Required	Default
`GITSTORE_GIT_URL`	Git repository URL	Yes	(none)
`GITSTORE_GIT_USERNAME`	Git username	No	`git`
`GITSTORE_GIT_TOKEN`	Personal access token or password	Yes	(none)
`GITSTORE_LOCAL_PATH`	Local clone directory	No	`./gitstore`

Repository Structure

your-config-repo/
├── config/
│   └── config.yaml
└── auths/
    ├── gemini.json
    ├── claude.json
    └── codex.json

Docker Compose Example

docker-compose.yml

services:
  cli-proxy-api:
    image: eceasy/cli-proxy-api:latest
    environment:
      GITSTORE_GIT_URL: https://github.com/your-org/cli-proxy-config.git
      GITSTORE_GIT_USERNAME: ${GIT_USERNAME}
      GITSTORE_GIT_TOKEN: ${GIT_TOKEN}
      GITSTORE_LOCAL_PATH: /data/gitstore
    volumes:
      - gitstore:/data/gitstore
    restart: unless-stopped

volumes:
  gitstore:

Git Authentication

GitHub Personal Access Token

Generate a token at https://github.com/settings/tokens
Required scopes: repo (full control of private repositories)
Use token as GITSTORE_GIT_TOKEN

GITSTORE_GIT_URL=https://github.com/your-org/cli-proxy-config.git
GITSTORE_GIT_USERNAME=your-github-username
GITSTORE_GIT_TOKEN=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

GitLab Personal Access Token

Generate at Settings → Access Tokens
Required scopes: read_repository, write_repository
Use oauth2 as username

GITSTORE_GIT_URL=https://gitlab.com/your-org/cli-proxy-config.git
GITSTORE_GIT_USERNAME=oauth2
GITSTORE_GIT_TOKEN=glpat-xxxxxxxxxxxxxxxxxxxx

Bitbucket App Password

Generate at Account Settings → App passwords
Required permissions: repository:write

GITSTORE_GIT_URL=https://bitbucket.org/your-workspace/cli-proxy-config.git
GITSTORE_GIT_USERNAME=your-username
GITSTORE_GIT_TOKEN=your-app-password

How It Works

The Git store:

Initialization (internal/store/gitstore.go:92-213):
- Clones the repository if not present
- Pulls latest changes from remote
- Creates directory structure if empty
Auto-commit (internal/store/gitstore.go:556-628):
- Commits changes to local repository
- Rewrites history as single commit (squash)
- Force pushes to remote
File Operations (internal/store/gitstore.go:216-296):
- Saves auth files locally
- Stages and commits changes
- Pushes to remote repository

The Git store uses force push (--force) to maintain a clean history. Do not manually edit the repository or use it for other purposes.

Object Store

Store configuration and authentication data in S3-compatible object storage.

Configuration

Set environment variables:

.env

OBJECTSTORE_ENDPOINT=https://s3.amazonaws.com
OBJECTSTORE_BUCKET=cli-proxy-config
OBJECTSTORE_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
OBJECTSTORE_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
OBJECTSTORE_LOCAL_PATH=/data/cliproxy/objectstore

Variable	Description	Required	Default
`OBJECTSTORE_ENDPOINT`	S3-compatible endpoint URL	Yes	(none)
`OBJECTSTORE_BUCKET`	Bucket name	Yes	(none)
`OBJECTSTORE_ACCESS_KEY`	Access key ID	Yes	(none)
`OBJECTSTORE_SECRET_KEY`	Secret access key	Yes	(none)
`OBJECTSTORE_LOCAL_PATH`	Local mirror directory	No	`./objectstore`

Endpoint Format

The endpoint supports both HTTP and HTTPS:

# HTTPS (default)
OBJECTSTORE_ENDPOINT=https://s3.amazonaws.com

# HTTP (for local development)
OBJECTSTORE_ENDPOINT=http://localhost:9000

# Without scheme (defaults to HTTPS)
OBJECTSTORE_ENDPOINT=s3.amazonaws.com

The server automatically:

Parses the scheme to determine SSL usage
Uses path-style bucket access for compatibility
Handles endpoint URL parsing (cmd/server/main.go:276-302)

Provider Examples

AWS S3

OBJECTSTORE_ENDPOINT=https://s3.amazonaws.com
OBJECTSTORE_BUCKET=my-cli-proxy-config
OBJECTSTORE_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
OBJECTSTORE_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Or use regional endpoint:

OBJECTSTORE_ENDPOINT=https://s3.us-west-2.amazonaws.com

MinIO

docker-compose.yml

services:
  cli-proxy-api:
    image: eceasy/cli-proxy-api:latest
    environment:
      OBJECTSTORE_ENDPOINT: http://minio:9000
      OBJECTSTORE_BUCKET: cli-proxy-config
      OBJECTSTORE_ACCESS_KEY: minioadmin
      OBJECTSTORE_SECRET_KEY: minioadmin
      OBJECTSTORE_LOCAL_PATH: /data/objectstore
    volumes:
      - objectstore:/data/objectstore
    depends_on:
      - minio

  minio:
    image: minio/minio:latest
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_data:/data

volumes:
  objectstore:
  minio_data:

Cloudflare R2

OBJECTSTORE_ENDPOINT=https://<account-id>.r2.cloudflarestorage.com
OBJECTSTORE_BUCKET=cli-proxy-config
OBJECTSTORE_ACCESS_KEY=<r2-access-key-id>
OBJECTSTORE_SECRET_KEY=<r2-secret-access-key>

DigitalOcean Spaces

OBJECTSTORE_ENDPOINT=https://nyc3.digitaloceanspaces.com
OBJECTSTORE_BUCKET=cli-proxy-config
OBJECTSTORE_ACCESS_KEY=<spaces-access-key>
OBJECTSTORE_SECRET_KEY=<spaces-secret-key>

Google Cloud Storage (S3 Compatible)

OBJECTSTORE_ENDPOINT=https://storage.googleapis.com
OBJECTSTORE_BUCKET=cli-proxy-config
OBJECTSTORE_ACCESS_KEY=<hmac-access-id>
OBJECTSTORE_SECRET_KEY=<hmac-secret>

Object Structure

Objects are stored with the following keys:

config/config.yaml          # Main configuration
auths/gemini.json           # Auth files
auths/claude.json
auths/codex.json

How It Works

The object store:

Initialization (internal/store/objectstore.go:54-119):
- Creates MinIO client with endpoint and credentials
- Ensures bucket exists (creates if needed)
- Syncs objects to local workspace
Mirroring (internal/store/objectstore.go:388-438):
- Downloads all objects from bucket
- Writes to local directory structure
- Maintains local cache for file operations
Upload (internal/store/objectstore.go:440-460):
- Reads local file changes
- Uploads to object storage with content type
- Handles deletions by removing objects

Storage Backend Priority

When multiple backends are configured, the following priority applies (cmd/server/main.go:237-455):

PostgreSQL - If PGSTORE_DSN is set
Object Storage - If OBJECTSTORE_ENDPOINT is set and PostgreSQL is not configured
Git - If GITSTORE_GIT_URL is set and neither PostgreSQL nor Object Storage are configured
Local Files - Default if no backend is configured

if value, ok := lookupEnv("PGSTORE_DSN", "pgstore_dsn"); ok {
    usePostgresStore = true
    pgStoreDSN = value
}
if usePostgresStore {
    useGitStore = false  // Disable Git if Postgres is enabled
}

Comparison

Feature	PostgreSQL	Git	Object Storage	Local Files
Distributed	✅	✅	✅	❌
Version Control	❌	✅	❌	❌
Transaction Support	✅	❌	❌	❌
Scalability	High	Medium	High	Low
Setup Complexity	Medium	Low	Low	None
Cost	Medium	Free (GitHub)	Low-Medium	Free
Best For	Production	Small teams	Cloud-native	Development

Migration

From Local Files to PostgreSQL

Start with local file configuration
Set up PostgreSQL database
Set PGSTORE_DSN environment variable
Restart service - configuration is automatically migrated

From Local Files to Git

Create a Git repository
Set Git environment variables
Restart service - files are committed and pushed

From Local Files to Object Storage

Create an S3 bucket
Set object storage environment variables
Restart service - files are uploaded to bucket

Troubleshooting

PostgreSQL Connection Failed

# Test connection
psql "postgresql://user:pass@host:5432/dbname"

# Check logs
docker compose logs cli-proxy-api | grep postgres

Common issues:

Wrong credentials in PGSTORE_DSN
PostgreSQL not accepting connections
Firewall blocking port 5432

Git Push Failed

# Check repository access
git ls-remote https://username:token@github.com/org/repo.git

# Verify token permissions
# GitHub: repo scope required
# GitLab: read_repository, write_repository

Common issues:

Invalid or expired token
Insufficient token permissions
Repository doesn’t exist
Force push disabled on branch

Object Storage Upload Failed

# Test credentials with AWS CLI
aws s3 ls s3://bucket-name --endpoint-url https://endpoint

# Check MinIO logs
docker compose logs minio

Common issues:

Wrong access key or secret key
Bucket doesn’t exist
Insufficient permissions
Endpoint URL incorrect

Next Steps

Docker Deployment

Deploy using Docker and Docker Compose

Cloud Deployment

Deploy in cloud environments with dynamic configuration

​Overview

​PostgreSQL Store

​Configuration

​Database Schema

​Config Table

​Auth Table

​Docker Compose Example

​Connection String Format

​How It Works

​Git Store

​Configuration

​Repository Structure

​Docker Compose Example

​Git Authentication

​GitHub Personal Access Token

​GitLab Personal Access Token

​Bitbucket App Password

​How It Works

​Object Store

​Configuration

​Endpoint Format

​Provider Examples

​AWS S3

​MinIO

​Cloudflare R2

​DigitalOcean Spaces

​Google Cloud Storage (S3 Compatible)

​Object Structure

​How It Works

​Storage Backend Priority

​Comparison

​Migration

​From Local Files to PostgreSQL

​From Local Files to Git

​From Local Files to Object Storage

​Troubleshooting

​PostgreSQL Connection Failed

​Git Push Failed

​Object Storage Upload Failed

​Next Steps

Docker Deployment

Cloud Deployment

Overview

PostgreSQL Store

Configuration

Database Schema

Config Table

Auth Table

Docker Compose Example

Connection String Format

How It Works

Git Store

Configuration

Repository Structure

Docker Compose Example

Git Authentication

GitHub Personal Access Token

GitLab Personal Access Token

Bitbucket App Password

How It Works

Object Store

Configuration

Endpoint Format

Provider Examples

AWS S3

MinIO

Cloudflare R2

DigitalOcean Spaces

Google Cloud Storage (S3 Compatible)

Object Structure

How It Works

Storage Backend Priority

Comparison

Migration

From Local Files to PostgreSQL

From Local Files to Git

From Local Files to Object Storage

Troubleshooting

PostgreSQL Connection Failed

Git Push Failed

Object Storage Upload Failed

Next Steps