System Case Study

J5 AI Test

J5 AI Test lets anyone write end-to-end browser tests in plain English — an AI agent drives a real browser via Playwright to execute them, removing the need for selectors or framework knowledge. Tests are markdown with YAML frontmatter (version-controllable and diff-able), and failures produce structured, screenshot-rich briefs you can paste straight into an AI coding agent to fix.

AI AgentsEnd-to-End TestingPlaywrightTypeScriptBunSQLiteMulti-ProviderAgentic QABrowser AutomationVitest

J5 AI Test

Plain-English end-to-end testing, powered by the AI agent you already use.

Today, end-to-end testing is gated by engineering: you need JavaScript, selector knowledge, and an understanding of how the test framework wires up. The people who understand the product best often can't write a Playwright test. J5 AI Test removes that gate — you describe the test in plain English, and an AI agent drives a real browser to execute it.

What a Test Looks Like

No imports, no selectors, no setup — just intent:

---
name: Submit feedback from dashboard
tags: [smoke, feedback]
---

Navigate to /dashboard.
Click the "New feedback" button.
Fill in the title with "Login button is too small on mobile".
Click Submit.
Verify a success toast appears.

Tests are plain markdown with YAML frontmatter — version-controllable, diff-able, and readable by anyone. The test is the prompt; updating one is a single line of English.

Why an Agent Instead of Scripts

A renamed CSS class breaks a Playwright selector and fails the test; an agent sees a "Submit" button and clicks it. The flakiness budget shifts from "selectors that drift" to "prompts that are ambiguous" — and ambiguous prompts can be fixed by the same AI that runs the tests. When a run passes, the agent can even suggest how to make the English more precise to prevent future flakiness.

Multi-Provider

Bring the agent you already have. CLI providers use your installed tools' credentials — Claude Code, Cursor Agent, Codex CLI, Gemini CLI — or use direct API providers (Anthropic, OpenAI, Google). Set a default per project or override per test with a provider: frontmatter key.

Failure Reports Built for AI

When something breaks, you get a step-by-step report with screenshots and a one-click brief — intent, what was attempted, where it broke, and suggested next steps — designed to paste straight into Claude, Cursor, or any coding agent to diagnose and fix.

Stack

TypeScript · Bun · Playwright (Chromium/Firefox/WebKit) · SQLite · Vitest · npm workspaces · MIT-licensed