self_test

Automated regression testing for Flutter, driven from outside the app with real pointer events. Widgets are found by what is on screen (their text, key, tooltip or semantics label), so an app does not have to be modified, wrapped or annotated before it can be tested.

🤖 AI-Powered Testing: Pair with self_test_mcp to enable Claude and other AI agents to test your Flutter apps with Playwright feature parity - works on iOS, Android, Web, and Desktop!

Start here: drive your app from a test with no change to it · wire it into your app to record and replay on the device · the steps a script can hold · the report page, where a run is read and a script is edited.

The app serves that page itself, on the port the bridge already holds. Every picture in this README is of the example/ app in this repository, which is also the host the quickstart wires up.

Features

Core Testing

• Universal locators - Address a widget by the text it paints, its key, its tooltip, its semantics label or its type. No wrapper, no annotation, no change to the app
• Real pointer events - Taps, drags, scrolls and long presses go through GestureBinding, so hit testing runs and a button behind a dialog is not reachable
• Real text entry - Typing goes through the same method the soft keyboard calls, so input formatters run and a form validates what was actually typed
• Runtime Testing - Run tests in live app environments without external test frameworks
• Text Assertions - Built-in text validation for input fields
• Memory Safe - Automatic registration/unregistration prevents memory leaks
• Hot Restart Compatible - Works seamlessly with Flutter's hot restart
• Test Scenarios - Define and run multi-step test scenarios with TestScenario
• Record on the device - Tap through a flow in the running app and replay it. Taps, typing and scrolls are watched at the root, so an app that wraps nothing is still recordable
• Self-healing steps - A step keeps the point it was recorded from, so a name that stops resolving is read again rather than abandoned. A repair is reported, never folded into a pass
• Run reports in a browser - Every replay leaves a record: one entry per step with its outcome, timing, the locator it ran with and a screenshot. Served by the app on the port the bridge already holds
• Edit a script in the browser - Add, change, reorder and delete steps, with the action list read from the replayer itself and the targets read off the screen in front of you
• Import and export - A script travels as one JSON file; a run travels as one self-contained HTML file with its screenshots inlined
• Answer a platform channel - A mockChannel step answers a picker, a permission dialog or a plugin before the platform hears it, so a workflow that needs a photo finishes on a machine with no camera

AI-Powered Testing (with MCP)

• 60+ Playwright-Equivalent Tools - Full feature parity with Playwright browser testing
• State Management Inspection - Inspect and modify Riverpod, Bloc, and Provider state
• Network Mocking - Mock HTTP responses, block requests, monitor traffic
• Visual Regression - Golden file comparison for UI testing
• Platform Mocking - Mock GPS, permissions, platform channels, sensors
• Cross-Platform - Works on iOS, Android, Web (with or without bridge), Desktop
• Bridgeless Web Testing - Test any Flutter web app via Playwright + semantics tree

Installation

Core Package Only

Add to your pubspec.yaml:

dependencies:
  self_test: ^2.0.0

Requires Flutter 3.35.0 or newer (Dart 3.9.0). That is the oldest version the test suite runs against in CI, not a guess.

Then run:

flutter pub get

The core package has no dependencies beyond Flutter and meta. The build_runner code generator is a separate, optional package (self_test_gen), so the analyzer and formatter never end up in your app's dependency tree. See Code Generation.

With AI-Powered Testing (Optional)

For AI agent integration with Claude Code:

1. Install MCP server:

   git clone https://github.com/loonix/self_test
   cd self_test/packages/self_test_mcp
   npm install && npm run build
   ./scripts/setup-claude.sh
   

2. Add bridge to your app:

   dependencies:
     self_test_bridge: ^2.0.0
   

See Architecture section below for details.

Quick Start

Drive an app that has never heard of this package

flutter pub add dev:self_test

test/login_test.dart, complete and copy-pasteable:

import 'package:flutter_test/flutter_test.dart';
import 'package:self_test/self_test.dart';

import 'package:your_app/main.dart';

void main() {
  final app = SelfTestManager();

  testWidgets('a user can sign in', (tester) async {
    await tester.pumpWidget(const MyApp());
    await tester.pumpAndSettle();

    // "Username" is the label beside the field, which is how a person names
    // it. self_test resolves from the label to the field it belongs to.
    await app.typeInto(const SelfTestLocator.text('Username'), 'ada');
    await app.typeInto(const SelfTestLocator.text('Password'), 'correct horse');
    await app.tap(const SelfTestLocator.text('Sign in'));
    await tester.pumpAndSettle();

    expect(app.exists(const SelfTestLocator.text('Welcome, ada')), isTrue);
  });
}

flutter test test/login_test.dart

That is the whole quickstart. MyApp is your app, unchanged: no SelfTestRoot, no SelfTestableWidget, no annotations, no generated code. This exact flow runs in CI on every push as test/no_wrapper_test.dart, against an app that imports nothing from this package.

Nothing above needs a change to the app. The locator is resolved against the element tree at the moment it is used, and the tap is a real PointerDownEvent and PointerUpEvent through GestureBinding, so hit testing runs: a button under a dialog is not reachable, a disabled button swallows the tap, and typing goes through the field's input formatters.

Locators, in the order you will reach for them:

Locator	Finds
SelfTestLocator.text('Sign in')	the text a widget paints (exact: false for a substring)
SelfTestLocator.key('submit')	a ValueKey<String>
SelfTestLocator.tooltip('Delete')	an icon-only button
SelfTestLocator.semantics('Avatar')	what a screen reader would read
SelfTestLocator.type('Switch')	a widget type by name
SelfTestLocator.id('login_button')	a SelfTestableWidget id

Add .at(2) to any of them to pick between duplicates, in tree order.

Add .within(scope) when that index would be counting the data rather than the screen. Measured on a real list of sites, the row titles sat at Text index 1, 7, 13, 18 and 24, because each row carries badges that are sometimes absent: the stride belonged to that afternoon's records, not to the layout.

// The third row's title, whatever the rows happen to hold.
SelfTestLocator.type('Text').within(SelfTestLocator.type('ListTile').at(2))

Scopes nest, and a scope that matches nothing resolves to nothing rather than quietly widening the search back out to the whole screen.

describeScreen() answers "what can I do here?" without a locator at all: it returns every actionable widget on screen with its type, text, tooltip, rect and enabled state. That is what the MCP server hands an agent.

Over the bridge there are also getByText and getByRole, for callers who arrive with Playwright's vocabulary. getByRole('button', name: 'Sign in') maps the role onto the Flutter types that answer to it (anything ending in Button, plus InkWell and the Cupertino set) and matches the name against the text the button paints, which for Flutter lives in a descendant rather than on the button itself. Both read the element tree, so neither needs an id.

Two questions that look the same and are not: exists asks whether a widget is in the tree, isVisible asks whether the user can see it. A list keeps items built for a while after they scroll away, so a driven tap on one refuses rather than landing on whatever is drawn at those coordinates now.

Under integration_test, add one line to main() before your tests:

final binding = IntegrationTestWidgetsFlutterBinding.ensureInitialized();
binding.shouldPropagateDevicePointerEvents = true;

That binding drops every pointer event that did not come from a WidgetTester. Without the line the taps do nothing and say nothing; with self_test you get an error that tells you this instead.

Set up in your app

Everything above needed no change to your app: a widget test drives it as it is. Recording on the device, replaying from the device and reading a report in a browser do need a change, and this is all of it. Three levels, each one a superset of the one before, so stop at the one you need.

Level	What it costs	What it buys
1. nothing	no change to the app	drive the app from a widget or integration test
2. the root	one wrapper and one binding in main	record a session on the device, replay it, edit it
3. the bridge	a debug-only block in main	run and edit scripts in a browser, read run reports, answer platform channels

Level 2: record on the device

import 'package:path_provider/path_provider.dart';
import 'package:self_test/self_test.dart';

Future<void> main() async {
  // First, before anything else in main.
  SelfTestWidgetsFlutterBinding.ensureInitialized();

  final dir = await getApplicationSupportDirectory();
  SelfTestManager().useRecordingStore(FileRecordingStore(dir.path));
  // Where a file a recording was given is kept. A picker answers with a path
  // into a directory the app empties.
  SelfTestManager().keepRecordedFilesIn('${dir.path}/self_test_files');
  await SelfTestManager().initializeRecordingStore();

  runApp(SelfTestRoot(child: MyApp()));
}

In a debug build SelfTestRoot also draws the recording controls over your app. They are hidden automatically under flutter_test, because they are a live overlay and pumpAndSettle on a tree containing one never returns.

SelfTestRoot(showControls: false, child: MyApp())  // never draw them
SelfTestRoot(showControls: true, child: MyApp())   // always, tests included
SelfTestRoot(child: MyApp())                       // debug app yes, test no

A draggable button opens the two controls, Record and Scripts.

Scripts opens the panel below: what is on the device, how each one last went, and what can be done with it without leaving the app.

That panel is drawn from Material icons, so your app's pubspec.yaml needs

flutter:
  uses-material-design: true

Without it the font is not bundled and every control in the panel renders as an empty box. Declaring it in this package's pubspec instead changes nothing: the app's manifest is the one the tool reads.

If your app owns a GlobalKey<NavigatorState>, hand it over: SelfTestRoot(navigatorKey: myKey, child: MyApp()).

Level 3: the bridge and the report page

import 'package:flutter/foundation.dart';
import 'package:self_test_bridge/self_test_bridge.dart';

SelfTestBridge? bridge;
NavigatorStateBridgeNavigator? bridgeNavigator;

Future<void> main() async {
  SelfTestWidgetsFlutterBinding.ensureInitialized();
  // ... the level 2 block ...

  if (kDebugMode) {
    bridgeNavigator = NavigatorStateBridgeNavigator(myNavigatorKey);
    bridge = SelfTestBridge(
      navigator: bridgeNavigator,
      port: 9999,
      token: 'a token you choose',
    );
    SelfTestManager().useRunReportStore(
      FileRunReportStore('${dir.path}/self_test_reports'),
    );
    await bridge!.start();
  }

  runApp(SelfTestRoot(navigatorKey: myNavigatorKey, child: MyApp()));
}

And in your MaterialApp:

MaterialApp(
  navigatorKey: myNavigatorKey,
  navigatorObservers: [
    if (bridgeNavigator != null) bridgeNavigator!.observer,
    if (bridge != null) bridge!.navigationObserver,
  ],
  // ...
)

Then open http://127.0.0.1:9999/report?token=a token you choose.

Every line, and why it is there

Each of these was added after the thing it prevents had already happened.

Line	Why
SelfTestWidgetsFlutterBinding.ensureInitialized() first	A plugin takes the default messenger from whichever binding exists the first time it is used, and only this binding can answer a channel in place of the platform. Put anywhere later, mockChannel steps install and never fire, so a script walks into the real camera.
SelfTestRoot	Watches pointers and scrolls at the root. Recording without it captures nothing on an app that wraps nothing.
FileRecordingStore	The default store is in memory, so every recording is gone on exit.
keepRecordedFilesIn	A picker answers with a path into a directory the app empties, so a recording that kept the path replays against a file that no longer exists. The copy is kept by name, because iOS renames the data container on every install.
FileRunReportStore in the support directory	A temporary directory is emptied whenever the platform likes, and a report that lost its screenshots without saying so is worse than no report.
an explicit token	A generated token is only announced through debugPrint, which under simctl launch reaches neither the console nor the device log. A token nobody can read means every connection gets a 403.
both navigator observers	navigationStack reads the bridge's, not the navigator's. Without the second, the bridge answers depth 0 for an app three routes deep.
kDebugMode	The bridge is a loopback socket into your app. It has no business in a release build.

The package guards the release build too: the manager answers "nothing here" unless test mode is on, so a locator in shipped code resolves to nothing rather than to a widget.

Optional: give a widget an id

Wrapping is not required. It is worth it when a widget has no text, no key and no label to find it by, or when a recorded script should keep working after the copy changes.

SelfTestableWidget(
  id: 'username_field',
  onTextChange: (value) => setState(() => username = value),
  child: TextField(
    decoration: InputDecoration(labelText: 'Username'),
    onChanged: (value) => setState(() => username = value),
  ),
),

await SelfTestManager().enterText('username_field', 'john_doe');
await SelfTestManager().trigger('login_button');

trigger and enterText take an id, send a real pointer event when the widget is on screen, and fall back to the registered callback only when it is not.

Security

self_test can read the entire widget tree, tap anything, type anything and photograph the screen. That is the product in a debug build and a remote control in a shipped one, so:

• It is inert in a release build. Every action and every query returns nothing or throws. Opt in explicitly with SelfTestManager.enableInReleaseBuilds() if a device farm needs to drive a signed build.
• The bridge listens on loopback only, and refuses to start in a release build. Pass host: InternetAddress.anyIPv4 to reach it from a real device and accept that the network can reach it too.
• The bridge requires a token, generated per instance and printed at startup, presented as ?token= or an x-self-test-token header. A wrong token gets a 403 before the WebSocket upgrade.

final bridge = SelfTestBridge();      // loopback, random token
await bridge.start();
debugPrint(bridge.url);               // ws://127.0.0.1:9999?token=...

Pass your own token when nobody is reading a console. The generated one is announced through debugPrint, which reaches you only while you are attached to flutter run. Drive the app the way CI does, flutter build then xcrun simctl launch or adb shell am start, and that line goes nowhere: not to the launch console, not to the device log. The token is then unknowable and every connection gets a 403.

final bridge = SelfTestBridge(
  token: const String.fromEnvironment('SELF_TEST_TOKEN'),
);

Give the same value to the MCP server and the two agree without anyone having to read it off a screen.

Writing a step

A stored script is a list of steps, and these are the steps there are. The report page offers the same list, read from the same place the replayer reads, so a step the page offers is a step that runs.

Step	Target	Value	What it does
trigger or tap	a widget	none	Tap it.
doubleTap	a widget	none	Tap it twice.
longPress	a widget	none	Press and hold it.
enterText or type	a widget	text, e.g. Hello	Type into a field. Name the field by the word beside it.
submit	a widget	none	Press the keyboard action on a field.
scroll	a widget	dx,dy, e.g. 0,150	Move a page. Name the Scrollable, not a row: the rows are what this moves.
drag or dragAt	none	x1,y1,x2,y2, e.g. 40,400,360,400	Drag a finger from one point to another. The only way to move a control that answers to distance rather than to a press: a slider pressed in the middle goes to the middle, whatever the case asks for.
goBack	none	none	Pop the route, the way the back button does.
wait	none	milliseconds, e.g. 500	Let real time pass. A step already waits for its own target, so this is for time nothing on screen announces.
mockChannel or mock	none	mock, e.g. {"channel": "plugins.flutter.io/path_provider", "codec": "method", "method": "getTemporaryDirectory", "response": "/tmp"}	Answer a platform channel before the platform hears it, so a picker or a permission dialog never opens.
capture or remember	a widget	fact name, e.g. siteName	Read what it says and keep it under a name. Later steps write it as ${name}, and whatever runs after this scenario is handed the same name.
assertText	a widget	expected, e.g. Saved	Fail unless it reads exactly this.
assertExists or exists	a widget	none	Fail unless it is there.
assertAbsent or absent	a widget	none	Fail if it is there. The only way to write a rule about what a screen must not show.
assertAbove or above	a widget	the row it must be above, e.g. Automation Kulas Bartoletti and Kozey Site	Fail unless the target is drawn higher up the screen than this text. The only way to write a rule about the order of a list.

Two of them are worth reading twice.

scroll names the page, not the row it brings into view: the rows are what it moves, so a row named there is somewhere else by the time the step runs. A recording writes one for you by watching the page rather than the finger, and the distance is how far the page went, which is not how far the finger went.

assertAbsent is the only way to write a rule about what a screen must not show. A form missing a mandatory field still answers every assertExists the complete one answers, so asserting the text that replaced the button passes just as happily on a screen showing both.

Mocking one plugin across platforms

mockChannel normally names one channel. When a plugin has different generated channels per platform, the same step can carry a channels list. Each entry is installed before the next real step runs, and only the channel the platform actually calls is used.

image_picker is the concrete case. iOS pickImage returns one path, while Android pickImages returns a list of paths:

{
  "channels": [
    {
      "channel": "dev.flutter.pigeon.image_picker_ios.ImagePickerApi.pickImage",
      "codec": "message",
      "response": "self_test:kept/self_test_seed_photo.jpg"
    },
    {
      "channel": "dev.flutter.pigeon.image_picker_android.ImagePickerApi.pickImages",
      "codec": "message",
      "response": ["self_test:kept/self_test_seed_photo.jpg"]
    }
  ]
}

The self_test:kept/ marker is resolved against the directory the app passed to keepRecordedFilesIn, so the same script survives reinstalling an iOS app and also runs on Android where the sandbox path is different.

Run reports

Every replay can leave a record behind: one entry per step with its outcome, its timing, the locator it was recorded with, the locator it actually ran with, and a screenshot. The app serves that record to a browser on the port the bridge already holds.

final dir = await getApplicationSupportDirectory();
SelfTestManager().useRunReportStore(
  FileRunReportStore('${dir.path}/self_test_reports'),
);

Then open http://127.0.0.1:9999/report?token=<your token>.

A run reads as one row per step, and the screen at that step is beside it. The capture box at the top decides how much of that is kept: every step, only the failures, or nothing at all.

A failure is the step, its message and the screen at that moment, side by side. The run above asserts a welcome the app never writes, and the picture says so faster than the message does.

The app serves it rather than writing a file to your machine because the screenshots live inside the app's sandbox. On a simulator they can be copied out; on a real device they cannot be reached at all, so a report written to the developer's machine only ever works on a simulator.

What a step records

Field	Why it is there
outcome	passed, repaired, failed or skipped. A repair is not folded into a pass: a script that rewrites itself quietly is worse than one that fails.
locatorAsRecorded / locatorAsUsed	They differ exactly when self-healing fired, which is the thing a green run otherwise hides.
durationMs	Where a slow suite spends its time.
screenshot	A name inside the run's folder, never a path: iOS renames the data container on every install, so a stored path expires while the file survives.
appErrors	What the app threw while that step ran, whatever the step's own verdict.

What the app threw

A step's error is the replay refusing. appErrors is the app breaking, and the two are independent.

A real one: a draft saved with no scaffolds opened straight into a build crash, and the scenario reported text: "Date and Time" is not there. True, and it sends the reader looking for a missing widget. The record now carries the app's own words on the step that caused them, two steps earlier:

step 13 trigger      -> passed   threw  MobXCaughtException x1
                                        _HandoverStore.siteName (handover_store.dart:215:16)
step 15 assertExists -> failed   refused  text: "Date and Time" is not there

The worse case is the one with no failure at all: the tap lands, every step passes, and the run is green over an app that threw on every frame. That run is marked in the list, so it is worth opening.

Three rules, because a naive log of this is unreadable:

• The hook is chained, never replaced. The bridge owns FlutterError.onError, and a report that captured it for itself would silence the console and the bridge's own errors command. It is put back when the run ends, or the next run chains onto it and counts every error twice.
• The same error raised on every frame is one entry with a count. A build that throws throws once per frame, so five seconds of settling is three hundred copies of one defect.
• Stacks are trimmed to twelve frames. Below that is the framework calling itself, identical in every report.

Unhandled asynchronous errors are collected too, through PlatformDispatcher.instance.onError, and the answer that hook returns is whatever the previous handler returned: this log writes things down, it does not handle them.

How much it costs

A capture is about 185 ms and 100 KB at a pixel ratio of 3, measured on an iPhone 16 Pro simulator. Over an 866-step suite that is roughly 2.7 minutes and 87 MB, so capture is a choice and the default is the cheap one:

Mode	Photographs
ReportCapture.never	Nothing. The record still holds every step, outcome and timing.
ReportCapture.failuresOnly (default)	The failure, plus the first and last step, so a report always opens on something to look at.
ReportCapture.everyStep	Every step that ran. The mode for chasing one flaky scenario.

SelfTestManager().setReportCapture(ReportCapture.everyStep, pixelRatio: 1.5);

A step that never ran is never photographed under any mode: the screen at that moment is the screen of the step that failed, and a report full of identical pictures of one failure hides which one it is.

Reports are capped (20 runs by default) and the cap deletes a run's pictures with it, because the pictures are where all the bytes are.

Driving and editing the app from the page

The page is also where a script is run and changed. Everything it offers is a command the bridge already carries, behind the same token on the same loopback socket: this is another way in, not more reach.

In the browser	What it does
Run, Stop	Starts a script and cancels the run under way. A run answers as soon as it starts, and the page follows it step by step, because a scenario takes minutes and a browser that waited would time out and read as a failure.
Rename, Delete	On a script, and on a run with its screenshots.
Export	A script as a .self_test.json file, or a run as one self-contained HTML file.
Import	A .self_test.json file, chosen or dropped anywhere on the page.
Edit	Add, change, reorder and delete the steps of a stored script.

Editing a step

A recording is a first draft. Fixing one step used to mean recording the whole session again, and the sessions being re-recorded were fifteen steps long.

The editor only offers what the app will accept:

• The actions come from the app, over /report/api/vocabulary, which reads the same list the replayer switches on. So a step the page offers is a step that runs. Written into the page instead, that list drifted the day an action was added and kept offering a step the replayer skipped with a warning printed to a console nobody was reading.
• The form asks for what the action needs and nothing else: no target field for goBack, a number for wait, a JSON box for mockChannel, each with the example from the table above as its placeholder.
• The targets come from the screen in front of you. The app is running and knows what is on it, so the picker lists what it found, each with the strongest name that finds it: a key before the words it paints, because a key survives a reword. Indexes are filled in, so the second Delete on a screen arrives as the second one.
• A name can be checked before a run depends on it. "Does it find it?" answers how many widgets that name matches, which one the index picks, and what that one is. Finding out at step 220 of 296 that a name matched nothing costs a full run. It reports rather than judges: a label inside a row is not a control and a tap on it still works, because the hit test lands on the row that handles it.
• A refusal is the app's own sentence. A step the replayer cannot run is refused while it is being written, quoting what is wrong with it.

Two things the editor does deliberately:

A step written or retargeted by hand loses its recorded point. The point is where the finger was, and a replay re-reads it to repair a name that has stopped resolving. Kept after a correction, the first repair reads the point again and puts back the very name that was just corrected. Steps without a point are marked no anchor in the list.

Deleting a step closes the gap behind it. Two steps sharing an order number run in whichever order the sort left them, which is the same script behaving differently on two devices.

Taking a report away

GET /report/export/<runId> answers one run as a single HTML file with its screenshots inlined as data URIs. It links to nothing, so it survives the app exiting and can be attached to a CI job.

Continuous integration

The suite runs from a terminal, and from any CI server, with no model in the loop and no tokens spent. Recording a scenario is the part a person or an agent does once; running it afterwards is the app executing its own steps and answering with a verdict.

dart run self_test:suite --dir test_scenarios --junit build/self-test.xml

It runs on the Dart VM that ships with the Flutter the job already installed, so a CI agent needs no Node, no Appium and no second toolchain.

What it needs

The command drives an app that is already running: it never builds, installs or launches anything, because every CI server already knows how to do those and none of them agree on how. So the job does this much first:

1. build and install the app, wired as in Set up in your app;
2. start it;
3. on an Android device or emulator, forward the bridge port: adb forward tcp:9999 tcp:9999. An iOS simulator shares the host's loopback, so it needs no forward.

Then the suite connects, and keeps trying until --connect-timeout runs out: a job that starts the app and the suite in the same breath is the normal case, not the failure.

Options

Option	What it does
--bridge <uri>	Where the app is listening. ws://127.0.0.1:9999 by default
--token <secret>	Or the SELF_TEST_TOKEN variable, which keeps it out of the build log and out of ps
--dir <path>	The scenario directory. test_scenarios by default
--only <a,b>	Named scenarios, same as naming them as arguments
--junit <file>	Write the verdict as JUnit XML
--report-dir <dir>	Export the app's own HTML report, one file per scenario, plus an index
--capture <mode>	never, failuresOnly or everyStep
--timeout <seconds>	Per scenario, 300 by default
--connect-timeout <seconds>	How long to wait for the app, 60 by default
--no-reset	Do not walk back to the first route between scenarios
--list	Print the scenario names and stop

Each scenario is installed into the app as a script and run there, which is what buys the record: screenshots, the errors the app threw while it ran, and the locators the replayer had to repair all belong to a script run and none of them exist for steps executed off the wire. A script of the same name is replaced, so a nightly job does not leave the app holding a hundred copies.

Exit codes

Code	Meaning
0	Every scenario ran and passed
1	At least one failed, or never ran
2	The suite could not run at all: no app answered, no scenarios, bad arguments

A scenario the suite never reached is skipped in the XML rather than absent. The difference matters: a run that died after five of forty-two otherwise reports five passes and looks like a green build.

What each server reads

Jenkins reads the XML through its junit step, which is where per-scenario history and blame come from. See doc/ci/Jenkinsfile.

GitLab reads it natively through artifacts:reports:junit, so the merge request shows which scenarios a change broke without opening a log. See doc/ci/gitlab-ci.yml.

GitHub Actions has no JUnit reader of its own, so the suite writes what GitHub does read: an annotation per failing scenario, which lands on the pull request, and a table in the job summary. It notices GITHUB_ACTIONS by itself, so there is no flag for it. See doc/ci/github-actions.yml.

A pass is not always the whole story

Two things travel in the XML that an exit code cannot express, both on scenarios that passed:

• repaired steps. A script whose recorded names no longer found anything, and which read the point the finger went down at instead. It passed, and it is the run to go and look at.
• errors the app threw. A screen can satisfy every assertion while throwing twice on the way in. That is exactly what the report is for.

Usage

Code Generation with Annotations

A typed controller removes the stringly-typed ids from your tests. It is generated by self_test_gen, which is a dev dependency: nothing it pulls in reaches your app.

dependencies:
  self_test: ^2.0.0

dev_dependencies:
  build_runner: ^2.4.9
  self_test_gen: ^2.0.0

Annotate the handlers you already wrote. The id is the same one the widget registers under:

// lib/login_form.dart
import 'package:flutter/material.dart';
import 'package:self_test/self_test.dart';

class _LoginFormState extends State<LoginForm> {
  @SelfTestButton('login_btn')
  void onLoginPressed() { /* ... */ }

  @SelfTestInput('username_field')
  void onUsernameChanged(String value) { /* ... */ }

  @SelfTestInput('password_field')
  void onPasswordChanged(String value) { /* ... */ }
}

Generate:

dart run build_runner build

That writes lib/login_form.self_test.g.dart, a standalone library. Do not add a part directive for it, and do not import it from login_form.dart. It is a library, not a part file, and importing a file that does not exist yet makes login_form.dart unresolvable, which leaves the generator with no annotations to find. Import it from your test:

// test/login_form_test.dart
import 'package:your_app/login_form.self_test.g.dart';

final controller = LoginFormStateTestController();

controller.enterUsernameField('john_doe');
controller.enterPasswordField('secret123');
controller.tapLoginBtn();
controller.expectUsernameFieldText('john_doe');
controller.expectLoginBtnExists();

The extension is .self_test.g.dart rather than plain .g.dart because source_gen already owns the latter: every part-based generator (json_serializable, freezed, mobx, hive, drift) writes through source_gen:combining_builder, which claims .dart -> .g.dart. Two builders claiming one output makes build_runner refuse to start in that package at all, taking your existing codegen down with it.

Naming rules, so you can predict the generated API without reading the output:

From	Generated
class _LoginFormState	LoginFormStateTestController (a leading _ is dropped so the controller is usable from a test)
@SelfTestButton('login_btn')	tapLoginBtn(), expectLoginBtnExists(), expectLoginBtnDoesNotExist()
@SelfTestInput('username_field')	enterUsernameField(String), expectUsernameFieldText(String), plus the two existence assertions

Ids are converted to camelCase for method names, so generated code passes the same lints as the rest of your project. The id itself is used verbatim in the calls, because that is the key the widget registered under.

A working end-to-end example, annotations through to a passing test, lives in example/: see lib/example.dart for the annotations and test/controller_test.dart for the generated controller in use.

Test Scenarios

Define multi-step test scenarios:

final scenario = TestScenario(
  name: 'Login flow',
  steps: [
    TestStep.enterText('username_field', 'user@example.com'),
    TestStep.enterText('password_field', 'password123'),
    TestStep.tap('login_button'),
    TestStep.wait(Duration(milliseconds: 500)),
    TestStep.screenshot('after_login'),
  ],
);

final result = await scenario.run();
print(result.allPassed ? 'All steps passed' : 'Failed at step ${result.failedAtStep}');

Widget Testing

Integrate with flutter_test:

testWidgets('Login flow test', (WidgetTester tester) async {
  SelfTestManager().setTestMode(true);

  await tester.pumpWidget(MyApp());
  await tester.pumpAndSettle();

  SelfTestManager().enterText('username_field', 'testuser');
  SelfTestManager().trigger('login_button');
  await tester.pump();

  expect(find.text('Login successful!'), findsOneWidget);
});

Enabling Self-Test Mode

// In debug/profile builds
SelfTestManager().setSelfTestModeActive(true);
SelfTestManager().restartWidgetTree();

// In test environments
SelfTestManager().setTestMode(true);

API Reference

SelfTestManager

Singleton managing test nodes and actions.

Method	Description
trigger(id)	Tap a button by ID
enterText(id, text)	Enter text in a field by ID
waitForAnimations()	Wait for UI updates
restartWidgetTree()	Force widget tree rebuild
captureScreenshot([name])	Capture a screenshot
registerTestNode(node)	Register a test node
unregisterTestNode(id)	Unregister a test node
setSelfTestModeActive(bool)	Enable/disable in debug/profile
setTestMode(bool)	Enable/disable in test environments

SelfTestableWidget

SelfTestableWidget({
  required String id,
  required Widget child,
  VoidCallback? onTap,
  ValueSetter<String>? onTextChange,
})

Annotations

@SelfTestButton(String id)  // For tappable widgets
@SelfTestInput(String id)   // For text input widgets

Architecture

The self_test ecosystem consists of three components:

┌─────────────────────────────────────────────────────────────────┐
│                      AI Agent (Claude)                           │
└─────────────────────────────┬───────────────────────────────────┘
                              │ MCP Protocol
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                   self_test_mcp Server                           │
│  • 60+ Playwright-equivalent tools                               │
│  • Actions: tap, type, scroll, drag                             │
│  • Assertions: expect, visual regression                        │
│  • State inspection: Riverpod, Bloc, Provider                   │
│  • Network mocking & monitoring                                 │
└─────────────────────────────┬───────────────────────────────────┘
                              │ WebSocket
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                Flutter App + self_test_bridge                    │
│  • Receives commands from MCP                                    │
│  • Executes via self_test callbacks                             │
│  • Works on iOS, Android, Web, Desktop                          │
└─────────────────────────────────────────────────────────────────┘

Components

Package	Description	When to Use
self_test (this package)	Core: locators, real pointer events, recording	Always - enables runtime testing in your Flutter app
self_test_bridge	WebSocket bridge connecting MCP to Flutter	When using AI-powered testing with Claude
self_test_mcp	MCP server with 60+ tools for AI agents	When using AI agents for automated testing

AI-Powered Testing with MCP

The self_test MCP (Model Context Protocol) server enables AI agents like Claude to test your Flutter apps with Playwright feature parity.

Platform Support

Platform	Bridge Required	How It Works
iOS	✅ Yes	Bridge runs WebSocket server on device, MCP connects
Android	✅ Yes	Bridge runs WebSocket server on device, MCP connects
Web (with bridge)	✅ Yes	Bridge embedded in web app, MCP connects
Web (bridgeless)	❌ No	MCP uses Playwright + Flutter semantics tree
Desktop	✅ Yes	Bridge runs WebSocket server in app, MCP connects

Quick Start with MCP

1. Install MCP Server

npx self-test-mcp --help

Add it to ~/.claude/settings.json. The token is the one your app prints at startup, or, better for anything automated, the one you passed to SelfTestBridge(token:) yourself: the printed one only exists while you are watching a flutter run console.

{
  "mcpServers": {
    "flutter-self-test": {
      "command": "npx",
      "args": ["self-test-mcp"],
      "env": {
        "FLUTTER_APP_HOST": "127.0.0.1",
        "FLUTTER_APP_PORT": "9999",
        "SELF_TEST_TOKEN": "<the token the app printed>"
      }
    }
  }
}

2. Add Bridge to Flutter App

Add to pubspec.yaml. Both packages, not just the bridge: the bridge is the socket, self_test is the binding, the root widget and the recording store.

dependencies:
  path_provider: ^2.0.0 # for the directory the recordings and kept files live in
  self_test: ^2.0.0
  self_test_bridge: ^2.0.0

Add to main.dart. Every line below is there because leaving it out breaks something that still reports success:

import 'package:flutter/foundation.dart';
import 'package:path_provider/path_provider.dart';
import 'package:self_test/self_test.dart';
import 'package:self_test_bridge/self_test_bridge.dart';

final navigatorKey = GlobalKey<NavigatorState>();
NavigatorStateBridgeNavigator? bridgeNavigator;
SelfTestBridge? selfTestBridge;

void main() async {
  // self_test's own binding, and first. A plugin takes the default messenger
  // from whichever binding exists the first time it is used, and only this one
  // can answer a channel (picker, camera, permission) in place of the
  // platform. With the plain binding everything still runs, and every native
  // picker opens for real, swallows the taps that follow, and the run reports
  // them as passed.
  SelfTestWidgetsFlutterBinding.ensureInitialized();

  // Debug only. The bridge refuses to start in a release build anyway: it is
  // a remote control for the app, and it listens on a socket.
  if (kDebugMode) {
    bridgeNavigator = NavigatorStateBridgeNavigator(navigatorKey);
    // Name the token. The generated one is announced only through debugPrint,
    // which `simctl launch` sends neither to the console nor to the device
    // log, so on a simulator a random token is unreachable and every
    // connection is refused.
    final bridge = SelfTestBridge(
      navigator: bridgeNavigator,
      port: 9999,
      token: 'probe-token',
    );
    selfTestBridge = bridge;

    // Recordings go to a file, so a flow captured today is still in the list
    // tomorrow. The default store is in memory and loses everything on exit.
    final supportDir = await getApplicationSupportDirectory();
    SelfTestManager().useRecordingStore(FileRecordingStore(supportDir.path));

    // A picker answers with a path into a directory the app empties, and iOS
    // renames the data container on every install, so a recording that kept
    // only the path replays against a file that is gone. Keep a copy instead.
    // Only the app knows which of its own directories outlives a run, which is
    // why self_test will not pick one for you.
    SelfTestManager().keepRecordedFilesIn('${supportDir.path}/self_test_files');
    await SelfTestManager().initializeRecordingStore();

    await bridge.start();
    debugPrint(bridge.url);
  }

  runApp(SelfTestRoot(navigatorKey: navigatorKey, child: MyApp()));
}

And in the MaterialApp:

MaterialApp(
  navigatorKey: navigatorKey,
  navigatorObservers: [
    if (bridgeNavigator != null) bridgeNavigator!.observer,
    // navigationStack reads this one, not the BridgeNavigator's. Without it
    // the bridge answers depth 0 for an app three routes deep, and a locator
    // index that counts the route stack lands on the wrong screen.
    if (selfTestBridge != null) selfTestBridge!.navigationObserver,
  ],
  // ...
)

3. Test with Claude

Open Claude Code and ask:

The agent's first call is flutter_describe_screen, which answers with every widget on screen and a ready-made locator for each. It does not need your app to have been prepared in any way.

Test the login flow in my Flutter app:
1. Enter username "test@example.com"
2. Enter password "password123"
3. Tap login button
4. Verify we navigated to the dashboard

Claude will use the MCP tools to interact with your app!

Web-External Mode (No Bridge Required!)

Test any Flutter web app without code changes using Playwright:

# Configure for web-external mode
export BRIDGE_MODE=web-external
export FLUTTER_APP_URL=https://your-app.com
export PLAYWRIGHT_HEADLESS=false

# Run MCP server
npm start

Perfect for:

• Production web apps
• Third-party Flutter apps
• CI/CD smoke tests
• Quick exploratory testing

Available MCP Tools

The MCP server provides 60+ tools with Playwright feature parity:

Locators & Queries:

• flutter_snapshot - Get widget tree
• flutter_get_by_role - Find by semantic role
• flutter_get_by_text - Find by text content

Actions:

• flutter_tap, flutter_type, flutter_clear, flutter_scroll
• flutter_drag, flutter_hover, flutter_focus
• flutter_long_press, flutter_double_tap

Assertions:

• flutter_expect - Assert widget state (toBeVisible, toHaveText, etc.)
• flutter_expect_screenshot - Visual regression testing

State Management:

• flutter_get_state - Inspect Riverpod/Bloc/Provider state
• flutter_dispatch_action - Dispatch events/actions
• flutter_watch_state - Subscribe to state changes

Network:

• flutter_mock_http - Mock API responses
• flutter_block_http - Block requests
• flutter_network_log - Monitor network traffic

Platform Mocking:

• flutter_set_geolocation - Mock GPS
• flutter_set_permission - Mock permissions
• flutter_mock_channel - Mock platform channels

See full tool list in packages/self_test_mcp/README.md

Ecosystem

Sponsors

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Run the test suite: flutter test
6. Submit a pull request

License

Copyright (c) 2025-2026 Ari Silva, Daniel Carneiro. All rights reserved.

This software may be used and modified in your own products and services, but may not be sold or redistributed as a standalone product. See the LICENSE file for details.