Proof Docs

Healing and results

A failed Playwright replay is evidence that automation broke—not yet proof that the product broke. Proof rechecks the business intent before deciding.

When a deterministic test fails

Proof preserves the failed replay, logs, screenshots, and trace. It then starts a fresh browser exploration from the existing scenario intent instead of asking AI to repair selectors directly.

Semantic adjudication

The new attempt receives the original customer goal and success criteria. An independent verifier evaluates its evidence. Proof then classifies the failure as:

  • stale automation The business outcome still works.
  • confirmed regression The outcome cannot be accomplished or verified.
  • inconclusive The evidence cannot support either claim.

Safe healing

A stale-automation result can produce a candidate test version. Proof compares semantic contracts, rejects removed or weakened assertions, and requires three clean deterministic validations. The active version remains immutable until the candidate passes policy.

Confirmed regressions

A confirmed regression means both the old deterministic path and a new semantic attempt failed the customer outcome. Proof prepares a report with the affected boundary, reasoning, deterministic evidence, screenshots, and presentation video, then returns it to the originating pull request or Slack thread.

When a human must review

Protected, financial, destructive, permission, and security scenarios cannot silently promote a heal. Their validated candidate enters review_required. An owner or administrator can inspect the old and new semantic behavior, validation evidence, and assertion diff before activation.