A customer does not experience a unit test, an application programming interface (API) response, and a browser page as separate systems. They see a signup form, an invitation, a dashboard, and an email or status message that tells them whether the job finished.
AI E2E testing targets that complete journey. E2E means end-to-end: the test begins at a user-facing entry point, crosses the boundaries the user crosses, and checks the final state the user can see. An AI E2E test adds a natural-language goal and page interpretation to browser controls, while preserving enough evidence for a failure to be fixed. The DOM, or Document Object Model, is the page’s structured tree of elements and attributes.
Tangle Browser Agent is Tangle’s browser driver for these goal-based sessions with screenshots and DOM evidence.
Its bad command is the command-line interface (CLI) for starting a run from a terminal.
This article shows where the approach belongs in a test stack, how to write one release-worthy case, and why a green model summary cannot serve as the result by itself.
For prompt design, read Natural Language Test Automation That Leaves Proof.
A full journey has more than one assertion
Consider a workspace invitation. The product promise is that an owner can create a workspace, invite a teammate, and see the invitation in the workspace settings.
An incomplete browser case clicks “Invite” and stops when a toast appears. A useful E2E case follows the user-visible journey:
- Open the application’s sign-in page with a disposable account.
- Create a workspace named “Smoke”.
- Open the members screen.
- Invite a test address.
- Confirm the members screen shows the pending invitation.
- Stop before sending mail to a real person.
The browser has to see the route changes, form validation, loading states, and final list. The backend has to accept the request. The final page has to represent the result accurately. The test should capture the boundary where each of those claims becomes visible.
The goal can stay in ordinary language:
bad run \
--url https://app.example.com \
--goal "Sign in with the disposable account, create a project named Smoke, invite the test address, and verify the invitation appears in the members list"This command follows the public bad run shape documented in the Browser Agent driver repository. The web address, account, and test address are illustrative. Replace them with a disposable environment that your team controls.
The E2E bar
The test case has to make five things visible.
| Requirement | Weak case | Reviewable case |
|---|---|---|
| input | “Test the app” | One named journey and start URL (web address) |
| starting state | Hidden account and mutable data | Disposable account, known data, and documented permissions |
| browser | Mocked page or a single isolated component | Real browser session crossing the user path |
| final condition | Model says the flow looks complete | Visible list, URL, state, or response that satisfies the goal |
| evidence | One summary sentence | Screenshots, page observations, actions, and stop reason |
The final condition must be observable. “The invite worked” is weaker than “the members screen shows [email protected] with a Pending status.” The latter gives the test a string, a location, and a state to check.
An evaluation is the step that compares this expected condition with the observed result. The evaluation can use a selector, an API response, a screenshot, or several pieces of evidence. The agent’s explanation can help a reviewer, but it should not replace the condition being checked.
Put AI E2E testing in the right layer
A healthy test stack gives each layer a narrow job.
| Layer | What it establishes | Typical owner |
|---|---|---|
| unit test | A function or component obeys a local invariant | Developer |
| API or contract test | A request, permission, and response shape agree | Service team |
| deterministic browser test | A stable UI path behaves under known selectors | QA or developer |
| AI E2E test | A changing user journey reaches a visible outcome | Product and QA |
| manual review | A new or high-risk experience deserves human judgment | Release owner |
Playwright is a good fit for deterministic browser cases. Its current documentation describes isolation, assertions, parallelization, and reports for modern web apps. The WebDriver standard defines the broader remote-control interface used by browser automation tools.
An AI case earns its place when page structure, copy, extensions, or conditional state make a fixed selector path expensive to maintain. A selector is a page query that identifies an element by its text, role, attributes, or position. It should still hand its result to deterministic checks where a known invariant matters. For example, an agent can navigate a billing page while a typed assertion checks that the displayed plan identifier matches the expected test plan.
Write a case a reviewer can rerun
A case needs more than a goal sentence. It needs enough context to explain whether a failure came from the application, the fixture, the browser, the network, or the agent.
{
"name": "workspace-invite-smoke",
"startUrl": "https://app.example.com",
"startingState": {
"account": "disposable owner account",
"workspace": "new workspace",
"invitedAddress": "[email protected]"
},
"goal": "Create Smoke, invite the test address, and verify the pending invitation appears",
"allowedActions": [
"navigate",
"fill forms",
"submit the workspace and invitation forms"
],
"stopBefore": [
"sending mail to a real address",
"changing a paid plan",
"deleting a workspace"
],
"evidence": [
"workspace form",
"members list",
"invitation status",
"first failure or stop reason"
]
}This object is an illustrative case definition. The public driver accepts a cases file through bad run —cases, but the exact case schema belongs to the installed version and should be checked with its documentation.
The case should also record the browser version, viewport, observation mode, model or agent profile, and fixture reset method. An agent profile is a named bundle of model choice, page-observation mode, permissions, time limits, and recovery rules. Two runs with different profiles may take different paths even when the goal is identical.
Keep fixtures from becoming the bug
End-to-end tests cross enough services that setup failures can dominate the result. Make the starting state deliberate.
| Fixture | Control to establish before the browser opens |
|---|---|
| account | disposable identity with the required role and no sensitive data |
| data | stable names, known empty states, and an explicit reset |
| browser | fixed browser family, viewport, locale, and extension list |
| network | reachable application and dependency health |
| time | deterministic clock or a tolerance for asynchronous states |
| credentials | least privilege, short-lived secrets, and no production access |
A health check can show that the application is reachable. It cannot prove the journey will succeed. A seeded database can make the run repeatable. It cannot prove the product handles a real empty state. Keep both the fixture description and the observed setup in the trace.
The runtime is the place where the test process, browser, files, and model calls execute. For a local run it may be a developer machine or continuous-integration worker. Continuous integration, or CI, is the automated process that runs checks for a change. Tangle Sandbox is Tangle’s isolated workspace for files, processes, and artifacts around an agent session. The runtime boundary affects network access, browser version, secret exposure, and reproducibility.
Turn a run into a release decision
A release rule blocks or permits a change based on a test result. An AI E2E rule should be small enough to inspect and important enough to matter.
| Rule | Final condition | Safe failure response |
|---|---|---|
| signup | new account reaches the dashboard | keep the account disposable and attach the final screenshot |
| activation | first project or integration is visible | record the first missing state and the preceding click |
| billing | test plan and billing state agree | stop before real payment and record the review page |
| wallet | selected account and chain appear in the application | stop at signing unless test-wallet permission is explicit |
| support path | the reported customer journey reaches the expected page | preserve the input class and redact sensitive values |
A few narrow cases are easier to own than many goals whose final conditions all mean “looks okay.” Choose the smallest set that covers the release risk and review every case’s evidence.
A passing rule means the specified condition was observed under the specified fixture and runtime. It does not mean every path through the product is healthy. It does not mean the model will follow the same route after a copy change.
Trace the first wrong step
A trace is the ordered record of goal, observation, action, result, recovery, final check, and stop reason. The first wrong step matters more than the final error because later actions may be consequences of the earlier mistake.
| Turn | Observation | Action | Result | Classification |
|---|---|---|---|---|
| 1 | Sign-in form is visible | Fill disposable credentials | Dashboard loads | expected |
| 2 | Empty workspace list is visible | Click “Create project” | Creation form loads | expected |
| 3 | Name field accepts “Smoke” | Submit form | Spinner remains for 30 seconds in this sample | environment or product |
| 4 | Error banner says service unavailable | Retry once within policy | Same error remains | failed |
| 5 | Final condition absent | Stop and save screenshot | fail with service error | actionable |
The last row is the evaluation result. The earlier rows explain whether engineering should inspect the page, the backend, the fixture, or the test environment. If the run only returns “project creation failed,” someone has to rerun it to recover that context.
Useful terminal states are:
| Status | Meaning |
|---|---|
| pass | The stated final condition was observed. |
| fail | The application or integration produced an observed result that violates the condition. |
| blocked | A required permission, login, human-verification challenge, dependency, or human approval prevented the case. |
| inconclusive | The evidence is insufficient to decide safely. |
Do not turn blocked or inconclusive into pass because the agent used all available steps. Do not turn a fixture outage into an application defect. The status vocabulary should make those distinctions visible to the release owner.
Parallel cases need separate worlds
Running several browser cases at once can shorten a suite, but concurrency does not make shared state safe. Each case needs its own account, browser context, wallet fixture, and data namespace unless the application explicitly supports isolation.
The public driver README documents a cases-file entry point and a concurrency option:
bad run --cases ./browser-cases.json --concurrency 4The command is useful when the four cases cannot affect one another. It is dangerous when two cases share a wallet nonce, mutate the same workspace, or rely on one email inbox. A parallel run should record the case identifier and fixture identifier with every artifact. Otherwise a screenshot from one account can be mistaken for a result from another.
Use serial execution for stateful wallet flows, shared test accounts, and cases that intentionally verify ordering. Use parallel execution for independent read-only pages or isolated disposable accounts. The right choice follows the state boundary, not the number of available browser slots.
Keep the execution owner visible
Browser Agent can run as a package or CLI without the rest of Tangle. If a team turns these cases into a service, a Blueprint is the reusable service definition and an operator is the provider that runs it. That packaging can describe who executes a job, which inputs it accepts, and which artifacts it returns. It cannot validate a browser result on its own.
Keep the final condition and its evidence in the test record, even when another service schedules or pays for the run.
Failure modes worth designing first
The hardest failures are often ordinary product states.
An invitation can be created twice because the page retried after the request succeeded. A wallet can show one account while the application keeps another account in memory. A dashboard can render cached data that looks complete while the API request failed. A login screen can send the agent into a loop that consumes steps without testing the product.
A test policy can still classify a challenge as blocked when the required human or account boundary was not safely verified. Do not treat a recovery attempt or a model summary as proof of identity.
The case should name the recovery policy. One bounded retry can be useful for a transient page load. Repeated clicks, changing user intent, bypassing a human-verification challenge, and submitting an unapproved payment are outside a safe recovery policy. When the boundary is reached, save the current evidence and stop.
What AI E2E testing cannot establish
One end-to-end run establishes one goal under one fixture, browser, model profile, and runtime. It does not prove every edge case, device, locale, account role, or network condition. It does not replace unit tests, API contracts, transaction simulations, or deterministic browser assertions. It does not make a production credential appropriate for an agent.
The strongest use is a product-boundary check that complements lower-level tests. The weakest use is a vague release ritual whose only output is a model saying “looks good.”
What is AI E2E testing?
AI E2E testing is end-to-end browser testing in which an agent follows a stated user goal, handles visible intermediate states, and checks the final user-visible condition.
Is AI E2E testing the same as Playwright?
No. Playwright is a browser automation framework. AI E2E testing adds goal interpretation and bounded recovery while still depending on browser controls and explicit assertions.
Is AI E2E testing reliable enough for CI?
It can be used in CI when fixtures are disposable, the final condition is explicit, the runtime is recorded, and every terminal result includes inspectable artifacts.
What should a first AI E2E case cover?
Choose one signup, activation, billing, wallet, or support journey whose failure would change a release decision. Keep the first case read-only or disposable until its stop policy is proven.
Make the release rule concrete
Use AI E2E testing when the user’s path crosses changing pages, conditional state, extensions, or visual decisions. Keep deterministic tests for stable invariants. Require a final condition, a saved trace, and a terminal status before the result can block or approve a release.