LiteLLM on macOS (Docker) — setup, config, maintenance, and usage supervision

This guide walks through running the LiteLLM Proxy on a MacBook Pro with Docker Desktop. It covers first-time setup, config.yaml and environment configuration, day-to-day maintenance, and how to monitor spend and usage.

What you get: an OpenAI-compatible gateway on http://localhost:4000, optional Admin UI, PostgreSQL for keys/spend/logs, and optional Prometheus metrics.

Table of contents

1. Prerequisites
2. Recommended layout on your Mac
3. First-time setup
4. Configuration
5. Start, stop, and verify
6. Using the proxy and Admin UI
7. Maintaining the stack
8. Supervising usage and cost
9. Troubleshooting
10. Security checklist (local dev)
11. Official references

1. Prerequisites

Apple Silicon (M2 Pro): Official LiteLLM and postgres:16 / prom/prometheus images are multi-arch and pull native arm64 builds, so no special configuration is needed. (Only if a future image lacks an arm64 build would you enable Docker Desktop’s “Use Rosetta for x86/amd64 emulation” as a fallback.)

Do not commit .env, real config.yaml with secrets, or database dumps to git.

2. Recommended layout on your Mac

Keep LiteLLM outside application repos (optional but cleaner). Example:

~/litellm-local/
├── docker-compose.yml      # from upstream LiteLLM repo
├── prometheus.yml          # optional; bundled in official compose
├── .env                    # secrets — chmod 600, never commit
├── config.yaml             # model routing (mount into container)
└── backups/                # pg_dump output

You can copy the compose file from the LiteLLM GitHub docker-compose.yml. The upstream stack defines three services:

• litellm — proxy on port 4000
• db — PostgreSQL 16 with a named volume litellm_postgres_data
• prometheus — metrics UI on port 9090 (optional but included in the default compose)

3. First-time setup

3.1 Create the project directory

mkdir -p ~/litellm-local/backups
cd ~/litellm-local

3.2 Download Docker Compose and Prometheus config

curl -fsSLO https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml
curl -fsSLO https://raw.githubusercontent.com/BerriAI/litellm/main/prometheus.yml

3.3 Create .env (master key, salt key, UI login)

The proxy requires a master key (admin API bearer token). If you use the database (default in compose), you also need a salt key to encrypt stored provider credentials.

# Master key — used as Bearer token for admin API and UI (do not use the example value)
printf 'LITELLM_MASTER_KEY="sk-%s"\n' "$(openssl rand -hex 16)" > .env
 
# Salt key — REQUIRED for DB-backed credential storage.
# Treat like a vault key: if you lose it, encrypted provider keys cannot be recovered.
# If you change it after models/keys were stored, those credentials break.
printf 'LITELLM_SALT_KEY="sk-%s"\n' "$(openssl rand -hex 32)" >> .env
 
# Admin UI (optional but recommended)
echo 'UI_USERNAME=admin' >> .env
echo 'UI_PASSWORD=change-me-on-first-login' >> .env
 
chmod 600 .env

Replace the openssl substitutions with real values (or use a password manager generator). Official docs: UI credentials.

3.4 Add provider credentials to .env

Examples (add only what you use):

# OpenAI
echo 'OPENAI_API_KEY="sk-..."' >> .env
 
# Azure OpenAI
echo 'AZURE_API_BASE="https://your-resource.openai.azure.com/"' >> .env
echo 'AZURE_API_KEY="..."' >> .env
 
# Anthropic (direct)
echo 'ANTHROPIC_API_KEY="sk-ant-..."' >> .env

Reference env vars in config.yaml as os.environ/VAR_NAME so secrets stay out of the config file.

3.5 Enable config.yaml in Compose

In docker-compose.yml, under the litellm service, uncomment the volume and command blocks so the proxy starts with your config:

volumes:
  - ./config.yaml:/app/config.yaml
command:
  - "--config=/app/config.yaml"

Also delete the build: block from the litellm service. Upstream ships both an image: and a build: context: .:

litellm:
  build:            # <-- remove these lines
    context: .      # <-- remove
    args:           # <-- remove
      target: runtime  # <-- remove
  image: docker.litellm.ai/berriai/litellm:main-stable

You only downloaded docker-compose.yml (not the full repo), so there is no Dockerfile in ~/litellm-local/. As long as the image pulls, the build: block is ignored — but if a pull ever fails, Compose falls back to building from context: . and errors out. Removing the block makes the stack pull-only and avoids that trap.

Create a minimal config.yaml (see section 4).

3.6 Pull images and start

docker compose pull
docker compose up -d
docker compose ps

Wait until the litellm service is healthy (compose defines a healthcheck that polls http://localhost:4000/health/liveliness inside the container).

3.7 Load secrets into your shell

The keys in .env are loaded into the container (via env_file), not into your terminal. Many curl examples below reference ${LITELLM_MASTER_KEY}, so export it into your current shell first:

# Load .env into the current shell (strips the surrounding quotes from values)
set -a; source ./.env; set +a
 
# Verify it's set (should print your sk-... master key)
echo "$LITELLM_MASTER_KEY"

Re-run this in any new terminal before using the admin/spend APIs.

4. Configuration

LiteLLM configuration splits into environment (.env) and YAML (config.yaml). With STORE_MODEL_IN_DB=True (default in upstream compose), you can also add or adjust models via the Admin UI; YAML remains the best place for version-controlled defaults.

4.1 Minimal config.yaml

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/my-deployment-name
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
      api_version: "2025-01-01-preview"
 
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
 
litellm_settings:
  drop_params: true
  set_verbose: false

DATABASE_URL is already set in docker-compose.yml for the bundled Postgres service (postgresql://llmproxy:dbpassword9090@db:5432/litellm). For a custom DB, change both compose and general_settings.database_url.

Docs: proxy config, managed files / DB.

4.2 Multiple models and aliases

Add one model_list entry per public alias (model_name). Clients call model: <model_name>; LiteLLM maps to the provider model string.

model_list:
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY

4.3 Router, fallbacks, and budgets (optional)

router_settings:
  routing_strategy: simple-shuffle
  num_retries: 2
 
litellm_settings:
  callbacks: ["prometheus"]

For team/key budgets and rate limits, use the database + Admin UI or /key/generate API (see section 8).

4.4 Reloading config

After editing config.yaml:

docker compose restart litellm

For changes made only in the UI / DB (STORE_MODEL_IN_DB), a restart is often not required; YAML-only changes always need a restart unless you use hot-reload features documented for your LiteLLM version.

4.5 Environment variables cheat sheet

5. Start, stop, and verify

5.1 Lifecycle commands

cd ~/litellm-local
 
# Start in background
docker compose up -d
 
# Follow logs
docker compose logs -f litellm
 
# Stop (keep data volumes)
docker compose stop
 
# Stop and remove containers (volumes persist)
docker compose down
 
# Destroy containers AND named volumes (deletes Postgres data)
docker compose down -v

5.2 Health checks

# Liveness (used by Docker healthcheck)
curl -sS http://localhost:4000/health/liveliness
 
# Readiness (when configured)
curl -sS http://localhost:4000/health/readiness

5.3 Smoke test (chat completion)

Use your master key or a generated virtual key:

export LITELLM_KEY="sk-your-master-or-virtual-key"
 
curl -sS http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer ${LITELLM_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Say hello in one word."}],
    "max_tokens": 10
  }'

5.4 Point tools at local LiteLLM

Match model names to your model_name aliases in config.yaml.

6. Using the proxy and Admin UI

6.1 Admin UI

Open http://localhost:4000/ui (path may vary slightly by version; check logs on first start).

Sign in with UI_USERNAME / UI_PASSWORD from .env. The UI supports:

• Models and credentials (when STORE_MODEL_IN_DB=True)
• Virtual keys, teams, users
• Spend dashboards and request logs (with DB enabled)

Docs: LiteLLM UI.

6.2 Virtual keys (recommended for apps)

Do not hand out the master key to every app. Generate a scoped key:

curl -sS -X POST 'http://localhost:4000/key/generate' \
  -H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
  -H 'Content-Type: application/json' \
  -d '{
    "key_alias": "my-laptop-cursor",
    "max_budget": 5.0,
    "rpm_limit": 60
  }'

Store the returned key in 1Password or your shell profile; rotate by generating a new key and revoking the old one in the UI.

7. Maintaining the stack

7.1 Upgrading LiteLLM

1. Back up Postgres (see below).
2. Pin a version in docker-compose.yml instead of floating main-stable when you need reproducibility, for example: image: docker.litellm.ai/berriai/litellm:1.86.2
3. Pull and recreate:

docker compose pull litellm
docker compose up -d litellm
docker compose logs -f litellm

Read release notes before major jumps; database migrations run on startup when the proxy connects to Postgres.

7.2 Backing up PostgreSQL

From the host (default compose credentials):

docker exec litellm_db pg_dump -U llmproxy -d litellm -F c \
  -f /tmp/litellm_backup.dump
 
docker cp litellm_db:/tmp/litellm_backup.dump \
  ~/litellm-local/backups/litellm_$(date +%Y%m%d).dump

Restore only into a fresh volume or after docker compose down with a clear recovery plan; test restores on a copy first.

Official guidance: backup before upgrade.

7.3 Upgrading Postgres

Prefer upgrading Postgres by dumping from the old container and restoring into a new major version container, rather than in-place major upgrades on the named volume.

7.4 Disk and Docker Desktop

• Named volume litellm_postgres_data grows with spend logs and key metadata.
• Prometheus retains 15 days by default. This is set in the prometheus service command: inside docker-compose.yml (--storage.tsdb.retention.time=15d), not in prometheus.yml (which only holds the scrape config).
• In Docker Desktop → Settings → Resources, monitor disk use; prune unused images periodically:

docker system df
docker image prune -f

7.5 Rotating secrets

7.6 Config and compose in git

Track sanitized config.yaml (no literal API keys) and a docker-compose.override.yml if you customize ports. Keep .env in .gitignore.

8. Supervising usage and cost

LiteLLM records spend in Postgres when DATABASE_URL is configured. Use the Admin UI for exploration and the HTTP API for automation.

8.1 Admin UI

• Dashboards: spend by key, team, model, time range.
• Logs: request-level metadata (hashed key, model, tokens, USD spend).
• Alerts: configure budget and rate-limit notifications per monitoring docs.

8.2 Spend API (master key)

# Info + spend for one virtual key
curl -sS "http://localhost:4000/key/info?key=<virtual-key>" \
  -H "Authorization: Bearer ${LITELLM_MASTER_KEY}"
 
# Spend log for a specific request id (from response headers / body)
curl -sS "http://localhost:4000/spend/logs?request_id=<id>" \
  -H "Authorization: Bearer ${LITELLM_MASTER_KEY}"

Endpoint names vary by LiteLLM version. For aggregated spend across keys/teams/models, prefer the Admin UI dashboards or the /global/spend/* family; confirm exact routes against your pinned version’s API docs (http://localhost:4000 → /docs).

Log entry fields include spend, total_tokens, model_group, team_id, end_user, and request_tags. Details: cost tracking.

8.3 Prometheus (optional service in compose)

1. Ensure litellm_settings.callbacks includes "prometheus" in config.yaml.
2. Open http://localhost:9090 and query metrics such as:

• litellm_requests_metric
• litellm_remaining_team_budget_metric
• Latency metrics (litellm_request_total_latency_metric, etc.)

Configure label cardinality in prometheus_metrics_config if you need fewer labels on high-traffic deployments. Docs: Prometheus.

8.4 Operational habits

• Set per-key max_budget and rpm_limit for every non-human integration.
• Use key_alias and request_tags in client calls to attribute spend to projects.
• Review weekly spend in the UI; export or scrape Prometheus for trends.
• Watch docker compose logs litellm during incidents; switch LITELLM_LOG=DEBUG only temporarily.

8.5 Example budget-aware key for CI

curl -sS -X POST 'http://localhost:4000/key/generate' \
  -H "Authorization: Bearer ${LITELLM_MASTER_KEY}" \
  -H 'Content-Type: application/json' \
  -d '{
    "key_alias": "ci-nightly",
    "max_budget": 0.50,
    "rpm_limit": 10,
    "duration": "30d"
  }'

Adjust duration and limits per virtual keys documentation.

9. Troubleshooting

Enable debug logging:

# In .env
LITELLM_LOG=DEBUG
docker compose up -d litellm

10. Security checklist (local dev)

• Bind to localhost only unless you explicitly need LAN access; avoid exposing port 4000 on public networks.
• Never commit .env or database dumps.
• Use virtual keys with budgets for IDE and script access.
• Rotate UI_PASSWORD and virtual keys if a laptop is lost or keys leak.
• Treat LITELLM_MASTER_KEY like a root password.
• On shared Macs, use separate virtual keys per person and disable UI password reuse.

11. Official references

• Docker quick start
• Deploy / production notes
• Config file
• Admin UI
• Virtual keys
• Cost tracking / spend logs
• Prometheus
• Upstream docker-compose.yml

Last updated: 2026-06-16 — align image tags and API paths with your pinned LiteLLM version when upgrading.