Generation Pipeline

How VigilQA converts Knowledge Base entries into reviewed, executable test scripts using five sequential AI agents.

Pipeline overview

The generation pipeline transforms your Knowledge Base into test scripts in five stages. Each stage is handled by a dedicated AI agent. Stages run sequentially; if any stage produces output that fails the review criteria, it is either automatically retried or queued for human review before the next stage begins.

Stage 1 — AppExplorer

AppExplorer crawls your live application using a headless Chromium browser. For each page or API endpoint referenced in the KB, it:

• Discovers and captures stable DOM selectors for all interactive elements
• Records page structure and element hierarchy
• Captures API endpoint signatures and response shapes (for API modules)
• Stores selector data as structured JSON indexed by domain and page

AppExplorer output is used by ScriptGen to generate tests with real, working selectors rather than placeholder values.

Inputs: Project URL, KB domain base URLs, authenticated session credentials (if configured)
Output: locators/<domain>/*.json — selector maps per page

Stage 2 — DocGen

DocGen takes each KB scenario and generates a structured test case document in Markdown format. These documents are the intermediate specification that ScriptGen uses to write code. Each document includes:

• Scenario ID, title, and priority
• Preconditions derived from the KB steps
• Step-by-step test actions with inferred interaction types
• Expected assertions derived from the expected field
• Data requirements (test credentials, input values)

Inputs: KB scenarios, AppExplorer selector maps
Output: docs/test_cases/<domain>/*.md — one file per scenario

Stage 3 — DocReview

DocReview acts as a QA gate on the generated documents before code is written. It checks each document for:

• Completeness — all required fields present and non-empty
• Consistency — steps and expected outcomes are internally coherent
• Actionability — steps are concrete enough for ScriptGen to produce valid code
• Selector availability — referenced elements exist in the AppExplorer output

Documents that pass DocReview proceed to ScriptGen. Documents that fail are either automatically corrected with a retry or forwarded to the Approvals queue with a failure reason for human review.

Inputs: DocGen output documents
Output: Approved documents → ScriptGen; rejected documents → Approvals queue

Stage 4 — ScriptGen

ScriptGen converts approved test case documents into executable pytest test scripts. Generated scripts use:

• Playwright (for Web UI modules) — real browser automation using selectors from AppExplorer
• HTTPX (for API modules) — HTTP client with schema-aware assertions
• pytest fixtures for session setup, environment-specific configuration, and credential injection

Scripts follow a consistent structure: one test function per scenario ID, with setup, execution, and assertion phases clearly separated.

Inputs: Approved test case documents, locator maps
Output: tests/<domain>/test_<scenario_id>.py — one file per scenario

Stage 5 — ScriptReview

ScriptReview performs static validation of generated scripts before they are committed to the test suite:

• Syntax validation — parses the script and reports syntax errors
• Selector validation — verifies referenced selectors exist in the AppExplorer locator maps
• Assertion quality — checks that each test has at least one meaningful assertion
• Import checks — verifies all referenced fixtures and utilities are available

Scripts that pass proceed to the Approvals queue for final human sign-off. Failed scripts are automatically retried with ScriptGen once; if they fail again, they enter the Approvals queue as rejected items with detailed failure reasons.

Inputs: ScriptGen output scripts, locator maps
Output: Approved scripts → Approvals queue; failed scripts → retry or Approvals queue (rejected)

Re-running the pipeline

The generation pipeline can be re-run at any time. You can choose to regenerate:

• Entire project — re-runs all stages for all domains
• Specific domains — re-runs all stages for selected domains only
• Specific scenarios — re-runs ScriptGen and ScriptReview for selected scenario IDs
• AppExplorer only — refreshes selector maps without regenerating scripts (useful after UI changes)