Test Execution

Overview

When you trigger a test run, TestOps connects to your configured server over SSH and orchestrates the entire lifecycle automatically. The worker pulls the latest code from your repository, installs any new or updated dependencies, seeds the database with sentinel-tagged test data, and then executes the selected test layers in sequence.

As tests complete, results stream back in real time via server-sent events (SSE). Once all layers finish, TestOps collects coverage reports, aggregates pass/fail/skip counts, groups errors by type, and performs a full cleanup of seeded data. The entire process is observable from the run detail page.

Every run is governed by environment-aware safety guards that prevent destructive operations from executing against production data without explicit confirmation. Layer restrictions, confirmation gates, and sentinel-based cleanup ensure your environments stay clean and safe.

Starting a Test Run

Navigate to Runs > New Run from the sidebar. Select the project you want to test. The project must already have a connected server and a valid repository configured under Project Settings.

Choose which test layers to execute. You can select any combination of the six available layers: unit, integration, system, e2e, smoke, and contracts. Running all layers provides the most comprehensive coverage, but you can also run a single layer for quick validation.

Optionally configure the number of parallel workers (1 to 10), the git branch to test against (defaults to the project's default branch), and the test type:

• functional — runs behavioral and feature tests only
• code — runs code quality, linting, and static analysis tests
• both — runs functional and code tests together

Click Run to start the execution. You will be redirected to the run detail page where you can monitor progress in real time.

The Six Test Layers

Parallel Execution

Tests can be split across multiple workers to reduce total execution time. When you set the worker count to a value greater than 1, TestOps divides the test suite into equal chunks and dispatches each chunk to a separate worker process.

Each worker receives its own isolated segment of the test files and runs them independently. The platform handles process coordination, ensuring that data seeding and cleanup occur correctly even when tests run in parallel. Workers use separate database transactions or distinct sentinel markers to avoid collisions.

Results from all workers are aggregated upon completion. The final report combines pass/fail/skip counts, coverage data, and error summaries from every worker into a single unified view. If any worker fails, the run is marked as failed and the errors from that worker are highlighted.

# Example: 4 parallel workers splitting 200 tests
Worker 1 → tests 1-50    ✓ 48 passed, 2 failed
Worker 2 → tests 51-100  ✓ 50 passed
Worker 3 → tests 101-150 ✓ 49 passed, 1 skipped
Worker 4 → tests 151-200 ✓ 50 passed

Aggregated: 197 passed, 2 failed, 1 skipped (42s total)

Test Data Lifecycle

TestOps uses a sentinel-based data seeding strategy to manage test data safely. Before tests run, the platform creates all necessary test records in the database, tagging each one with a unique sentinel marker that identifies it as test data.

The sentinel is a unique identifier tied to the specific run. Every record created during seeding — users, organizations, transactions, settings — carries this marker in a designated field. This makes test data trivially identifiable and completely separable from real data.

After all tests complete (whether they pass or fail), the cleanup phase removes every record tagged with the run's sentinel. This ensures tests never leave orphaned data behind. The cleanup runs in dependency order to respect foreign key constraints, deleting child records before parents.

Lifecycle:
  1. Seed   → Create test records with sentinel "run_a1b2c3"
  2. Run    → Execute tests against seeded data
  3. Clean  → DELETE FROM users WHERE sentinel = 'run_a1b2c3'
              DELETE FROM orgs  WHERE sentinel = 'run_a1b2c3'
              ...all sentinel-tagged records removed

Production Safety

TestOps enforces strict safety guards to prevent accidental damage to production environments. These guards operate at multiple levels and cannot be bypassed without explicit human confirmation.

Layer restrictions prevent certain test layers from running in production environments. By default, integration and system tests that perform write operations are blocked in production. Only smoke tests and contract tests (which are read-only) are permitted without additional approval.

Confirmation gates require explicit approval before executing any destructive operation. If a test run targets a production server with write-heavy layers enabled, the user must confirm the action through a modal dialog. The confirmation includes a summary of what will be executed and which data will be affected.

Environment-based guards use the server's configured environment label (development, staging, production) to automatically apply the correct safety policies. Each environment has its own set of allowed layers, concurrency limits, and cleanup behavior. These guards are configured at the project level and enforced at runtime.

Monitoring a Run

The run detail page provides real-time visibility into every phase of execution. As soon as a run starts, the page establishes a server-sent events (SSE) connection that streams updates directly from the worker.

The header displays the current phase: setup (pulling code and installing dependencies), seeding (creating test data), running (executing test suites), cleanup (removing sentinel data), or complete. A progress bar shows the percentage of tests completed.

The live log panel streams stdout and stderr output from the worker in real time. You can see dependency installation progress, test runner output, individual test results, and any errors as they occur. Logs are color-coded: green for passes, red for failures, yellow for skips.

Individual test results appear in a scrolling list as each test completes. Each entry shows the test name, duration, and pass/fail/skip status. Failed tests are expanded automatically to show the error message and stack trace. You can click any test to view its full output.

Viewing Results

After a run completes, the results page provides a comprehensive breakdown of everything that happened. The summary header shows total pass, fail, and skip counts alongside the overall duration and final status.

Coverage metrics are collected automatically when available. TestOps reports line coverage, branch coverage, and function coverage as percentages with trend indicators showing whether coverage increased or decreased compared to the previous run.

Error summaries group failures by error type and message. If 15 tests fail with the same TypeError, they appear as a single group with a count badge. This makes it easy to identify systemic issues rather than sifting through individual failures. Each group expands to show the affected tests.

Individual test details are available for every test in the run. Each test entry includes the file path, test name, duration, status, and (for failures) the full error message and stack trace. You can filter the list by status, layer, or search by test name.

Run #1042 — Completed
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Status:    passed
Duration:  3m 12s
Layers:    unit, integration, e2e

Results:   247 passed · 3 failed · 5 skipped
Coverage:  Lines 84.2% (+1.3%)  Branches 71.8% (+0.5%)  Functions 89.1%

Errors (2 groups):
  TypeError: Cannot read property 'id' of undefined  (2 tests)
  AssertionError: expected 200 to equal 201           (1 test)