Self-Hosted Workers

Why Self-Hosted?

TestOps cloud workers can reach public URLs, but many organizations run staging and development environments on internal networks behind firewalls. A self-hosted worker solves this by running inside your network — it connects outboundto TestOps (no inbound firewall changes needed) and processes only your organization's jobs.

• Security — your code never leaves your network. Discovery, analysis, and test generation all run locally.
• Internal access — test against http://staging.internal:3000, VPN-only services, and private databases.
• Isolation— each org's jobs are routed to dedicated queues. Your self-hosted worker will never process another org's workload.
• Full coverage — all engines run on the self-hosted worker: discovery, gap analysis, test generation, unit/integration tests, and browser E2E.

How It Works

┌─────────────────────┐         outbound          ┌──────────────────┐
│  Your Network       │  ─────────────────────▶    │  TestOps Cloud   │
│                     │    Redis (BullMQ)           │                  │
│  ┌───────────────┐  │         ◀─── job queue ──── │  API + Dashboard │
│  │ Self-Hosted   │  │                             │                  │
│  │ Worker        │──┤── results + artifacts ───▶  │  PostgreSQL      │
│  │ (Docker)      │  │                             │  R2 Storage      │
│  └───────┬───────┘  │                             └──────────────────┘
│          │          │
│  ┌───────▼───────┐  │
│  │ internal-app  │  │
│  │ :3000         │  │
│  └───────────────┘  │
└─────────────────────┘

The worker connects outbound to the shared Redis instance. TestOps routes jobs to an org-scoped queue (test-runs__sh__{orgId}). The self-hosted worker picks up only its own org's jobs, executes them locally in Docker sandboxes, and pushes results back via the API.

Prerequisites

• A Linux server (Ubuntu 22.04+ recommended) with Docker installed
• Outbound network access to your TestOps Redis instance and API
• At least 2 CPU cores and 4 GB RAM (more for parallel E2E runs)
• Your Organization ID — find it in Settings

Step 1 — Download the Worker Config

Go to the Workers page in the dashboard. Click “Add Self-Hosted Worker” and copy the generated setup command — it includes your org ID and Redis connection string pre-filled.

Alternatively, create the config manually. Create a directory on your server and add an .env file:

# .env — self-hosted worker configuration
REDIS_URL=rediss://your-redis-host:6379
DATABASE_URL=postgresql://...

WORKER_NAME=my-org-worker-1
WORKER_TYPE=self_hosted
WORKER_ORG_ID=your-organization-uuid

WORKER_CONCURRENCY=2

# Sandbox configuration
TESTOPS_SANDBOX_RUNTIME=docker
TESTOPS_SANDBOX_DIR=/var/lib/testops-sandbox
TESTOPS_SANDBOX_MEMORY=2g
TESTOPS_SANDBOX_CPUS=2

# Playwright browser image (for E2E tests)
TESTOPS_IMAGE_PLAYWRIGHT=mcr.microsoft.com/playwright:v1.48.0-jammy
TESTOPS_IMAGE_NODE=node:20-bookworm-slim

Step 2 — Start the Worker

Create a docker-compose.yml in the same directory:

version: '3.8'
services:
  worker:
    image: ghcr.io/calibertechnologies/testops-worker:latest
    env_file: .env
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/testops-sandbox:/var/lib/testops-sandbox
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G

Then start it:

docker compose up -d

The worker will connect to Redis, register itself, and begin listening for your org's jobs. You should see it appear on the Workers page within 15 seconds.

Step 3 — Register the Worker

Register the worker in TestOps so the platform knows to route jobs to it. From the Workers page, click “Add Self-Hosted Worker” and fill in:

• Name — a unique identifier (e.g. staging-worker-1)
• Server Host — the hostname or IP where the worker runs
• Type — select “Self-Hosted”

Or register via API:

curl -X POST https://app.testops.calimatic.com/api/workers \
  -H "Content-Type: application/json" \
  -d '{
    "name": "staging-worker-1",
    "serverHost": "10.0.1.50",
    "type": "self_hosted",
    "organizationId": "your-org-uuid"
  }'

What Runs on the Self-Hosted Worker

Once registered, all engines route to your self-hosted worker automatically. No per-job configuration needed.

Monitoring & Troubleshooting

• Dashboard — the Workers page shows real-time CPU, memory, heartbeat, and job status for all workers. Self-hosted workers display a purple badge.
• Logs — view worker logs with docker compose logs -f worker
• Heartbeat timeout— if a worker misses heartbeats for >60 seconds, its status changes to offline. Jobs will queue until the worker recovers or a cloud fallback is available.
• Scaling — deploy multiple self-hosted workers for the same org. They share the same org-scoped queue and process jobs in parallel.

Security Model

• Test sandboxes run in Docker with read-only filesystems, capability drops, memory limits, and PID limits.
• The worker only connects outbound — no inbound ports need to be opened on your firewall.
• Org-scoped queues ensure your worker never receives jobs from another organization.
• Git credentials are encrypted at rest and decrypted server-side before transmission — the worker receives a short-lived token, never the stored credential.
• Source code cloned for discovery and test generation stays on your worker and is cleaned up after each job.