QA-Pilot0.4

Project

What it enforces

The technology this plugin requires of a host project, how tightly each choice is bound in, and what it would cost to change.

Last verified 2026-10-01, against the commands in the final section.

Enforced #

Requirement Depth Reason
Claude Code Absolute It is a plugin. There is no version of this that runs elsewhere.
Playwright Hard, deliberate The deepest coupling in the system, and the one worth having. See below.
Node 20 or newer Trivial Scripts are zero-dependency ESM. Node 18 is end-of-life; the plugin's own CI runs 20, 22 and 24.
ESLint with eslint-plugin-playwright, typescript-eslint and typescript Light, one lint step Since 0.4.1 /qa-pilot:run-tests and the CI template lint specs with the committed eslint.qa-pilot.config.mjs, so a spec with no assertion, a fixed sleep or a skipped test fails before it can earn a verdict. The rules come from eslint-plugin-playwright, which the host adds as a dev dependency. Swapping the linter means rewriting one template and two lint commands.
Git Trivial Nothing is required to be committed. Committing the profile, the specs and the per-feature ledgers is what makes the pipeline work for a team rather than one machine, and CI needs them, but every skill warns and continues instead of refusing.
ClickUp Optional, surface only tracker: none runs the whole pipeline on local files. With a tracker, ClickUp is the one supported, and it is much shallower than the name suggests. See below.
YAML for config Cosmetic One file, one vendored parser. Chosen over JSON because the profile is hand-edited by QA and needs comments.
Deployed environments with a readable build SHA Hard, but it is process rather than technology Applies whatever tracker or runner you use. A verdict that cannot be pinned to a build is not a verdict.

Deliberately not enforced #

Worth stating, because it is the longer list and people assume otherwise.

No frontend framework assumption: nothing in the plugin knows or cares about Vue, React, or anything else. No test-id attribute name. No assertion style. No mock-mode mechanism. No status vocabulary. No spec directory layout. No environment naming.

No CI system either. ci-gate.mjs is plain Node and knows nothing about any runner; it answers "what did QA approve" and "what does this run mean for the build", and the runner invokes the tests. Only templates/qa-pilot-ci.yml, a file hosts copy and edit, is GitHub-shaped, and porting it means translating its steps, not changing the plugin.

All of those live in the committed host profile, which is the portability premise: onboarding a second project needs no plugin code changes. Its profile, including the write signatures that say what counts as a write on that host, is the work.

ClickUp is shallower than it looks #

No script in this plugin talks to ClickUp. Not one network call. Every ClickUp mention inside scripts/ is a comment, a doc-string, a warning message, or a read of profile.clickup.statuses.

That is a happy accident rather than foresight. Scripts cannot call MCP tools, so the writes had to live in skill instructions. That constraint left the deterministic core tracker-agnostic:

  • case-status.mjs consumes a plain {"CASE-ID": "status"} JSON produced by the model. It has no idea where those statuses came from.
  • validate-report.mjs takes two plain JSON objects: --map for key membership, and --statuses for the status each case carried when it ran. It resolves those status names through clickup.statuses in the profile, which is a host's own vocabulary rather than anything ClickUp-shaped, and it never learns where the names came from.
  • report.json contains no tracker-specific field at all.

Real coupling is confined to three places: the clickup-guard.mjs hook and its matcher in hooks.json, the skill prose that names ClickUp MCP tools, and five profile fields (plan_tier, space, folder, bug_list, statuses). Of those five, only plan_tier is genuinely ClickUp-shaped; the rest name a place and a vocabulary that any tracker has.

No tracker at all is already supported. tracker: none drops the clickup block from the normalized profile, so scripts fall back to the canonical status names, with approvals, evidence and bugs kept in local files (see SETUP-CLICKUP.md). It cost little because the scripts never knew where statuses came from.

Cost to support Jira, Linear, or GitHub Issues: roughly a day by estimate, almost entirely prose. Add a value to the profile's tracker key, rename clickup-map.json to tracker-map.json, generalise the guard matcher, and rewrite the skill sections that name MCP tools. publish-payload.mjs already emits the field set as plain data, so the scripts barely change.

Playwright is deep, and that is the point #

This is the real lock-in:

  • parse-report.mjs parses Playwright's specific JSON reporter shape, including its status vocabulary (expected, unexpected, timedOut, interrupted) and its per-project tests[] nesting.
  • save-storage-state.mjs uses the Playwright API directly, and storageState({ indexedDB: true }) is a Playwright feature.
  • The evidence model is the trace. One file carrying the video, console output, DOM snapshots, network log, and a scrubable film-strip, openable client-side at trace.playwright.dev with no upload and no account.

Cost to support another runner: one adapter, but you would give up the trace. parse-report.mjs is the seam, since report.json downstream is generic, so an adapter is roughly 180 lines (an estimate, not measured). What would not carry over is the trace as one file with video, console and network log; evidence would go back to separate files.

Recommendation: treat Playwright as a fixed premise. If a host cannot use it, this is the wrong tool for that host.

The position #

For a team already on ClickUp, enforcing it is fine, and the plugin now speaks the host's own status vocabulary rather than imposing one.

For wider distribution, ClickUp is the weakest constraint, since it lives in skill prose and a hook matcher, not in business logic.

Do not build a tracker abstraction speculatively. With one team on one tracker it costs clarity in every skill file. tracker: none is not that abstraction, just the absence of a tracker. If a second project adopts a different tracker, do the renames then.

Re-verifying these claims #

The claims above can go stale. Re-run these before trusting them:

bash
# 1. No script may call the ClickUp API. Expected output: nothing.
grep -rn 'api\.clickup\|clickup\.com' qa-pilot/scripts/ | grep -v yaml.mjs

# 2. Only the guard may know ClickUp tool names.
#    Expected output: qa-pilot/scripts/clickup-guard.mjs, and nothing else.
grep -rl 'mcp__' qa-pilot/scripts/*.mjs qa-pilot/scripts/lib/*.mjs | grep -v yaml.mjs

# 3. Where Playwright coupling lives. Expected exactly these six:
#      lib/profile.mjs          version floor check only
#      parse-report.mjs         the real one: parses the JSON reporter format
#      save-storage-state.mjs   the other real one: uses the Playwright API
#      validate-report.mjs      re-parses results.json through parse-report, plus messages
#      bug-report.mjs           mentions it in the bug text only
#      publish-payload.mjs      mentions it in the reviewer line only
#    The write guard's Playwright wiring lives in templates/, outside scripts/, on purpose:
#    hosts copy it, and nothing in scripts/ imports it.
#    ci-gate.mjs must NOT appear: it reasons about approval and verdicts, and
#    the runner is what knows how to invoke a test.
grep -rl 'playwright' qa-pilot/scripts/*.mjs qa-pilot/scripts/lib/*.mjs \
  | grep -v yaml.mjs | sort

If command 1 prints anything, a script has started calling ClickUp and the tracker-agnostic core is gone. If command 2 lists a second file, tracker knowledge has leaked out of the guard. Either way this document is out of date and the swap estimate no longer holds.