Run tests

Running a visual test captures new screenshots and compares them pixel-by-pixel against the baselines.

Running from the dashboard

All scenarios

From the project header, click "Run all visual tests". Every scenario in the project will be tested across all configured viewports and browsers.

Selected scenarios

From the Tests tab, you can select specific scenarios to test rather than running all of them.

What happens during a test run

Capture

VisualQ's worker launches Playwright, navigates to each scenario's URL, executes click selectors, applies content rules, waits for the configured delay, and takes full-page screenshots at each viewport. Large pages with fixed headers may use automatic stitched capture mode. During stabilization the worker also auto-dismisses common cookie banners, waits for network idle, freezes animations, and scrolls to trigger lazy-loaded content.

Compare

Each test screenshot is compared pixel-by-pixel against its corresponding baseline. The mismatch percentage is calculated.

Analyze

If AI analysis is enabled and failures are detected, Smart Diff analyzes the visual differences to classify severity and provide explanations.

Report

Results are saved and notifications are sent through configured integrations (Slack, GitHub, webhooks, etc.).

Run status

Each run produces one of these statuses:

Status	Meaning
`running`	Test is in progress
`completed`	All comparisons finished, results available
`failed`	An error occurred during capture or comparison

Real-time notifications

When a test run starts and completes, you'll see toast notifications in the dashboard. If you're on a different page, you can navigate to the Tests tab to see results.

Full audit (every unlocked pillar)

From any scenario card or scenario detail page, Full audit fans out to every pillar unlocked by your plan: Visual, Accessibility, Performance, SEO, Security, and Tracking. Each pillar runs as a child of a single audit batch so the activity feed shows one row per click.

Tracking is best-effort inside the batch:

If you haven't configured a tracking plan yet, the tracking pillar is silently skipped (with a warning) — the rest still runs.
If the audited scenario has no linked tracking pages or events, tracking is also skipped for that batch.
Set up the tracking plan to opt the scenario back in.

The same logic powers the Run full quality audit button on the project home — the only difference is whether you scope the run to one scenario (scenarios: [...]) or to the whole project (no filter).

You can trigger a per-scenario audit programmatically:

POST /api/tests/full-suite
Content-Type: application/json

{
  "project": "<projectId>",
  "scenarios": ["My scenario label"],
  "environment": "<optional environment id>"
}

Or the project-wide audit:

POST /api/quality-score/full-audit
Content-Type: application/json

{
  "project": "<projectId>",
  "environment": "<optional environment id>"
}

The response includes runId, the resolved audits[], and childRunIds per pillar. Results appear in the Tests tab as a Full audit row; expand it to drill into each pillar's child run.

Composite Quality Score

When a Full audit completes, VisualQ recomputes a 0–100 composite Quality Score from the latest run of every pillar your plan unlocks (including Tracking when it ran), and stamps it on the parent audit batch:

The score is rendered directly on the Full audit row in the activity feed (e.g. Quality 87/100).
Locked pillars (those gated by your plan) are excluded from the average — your plan never penalises you.
Pillars that have no run yet are simply skipped; the composite is the average of pillars that did run.
Tracking participates only when the audit actually executed it. A scenario without a tracking plan or without linked events drops tracking from the batch (with a trackingSkipReason) — the composite is then averaged across the remaining pillars.

The same number drives the project-home Quality Score widget (GET /api/quality-score/{project}) and the quality_score.dropped integration event when the score regresses by 5 points or more.

Running from CI/CD

You can trigger test runs programmatically:

GitHub Action: type: test (default)
REST API: POST /api/ci/run

See the CI/CD section for details.

Capture

Compare

Analyze

Report

On this page