flutter_wright

Drive a running Flutter app over HTTP — snapshot the UI, then tap, type, and scroll it. Built for AI agents and automated end-to-end testing, in the spirit of Playwright.

The SDK runs a tiny control server inside your app. It is off by default and only binds a socket when you explicitly enable it, so the call is safe to leave in production code — release builds become a no-op.

Companion to the flutter-wright Claude Code skill. But any HTTP client works just as well — curl, Postman, or your own script.

Install

dev_dependencies:
  flutter_wright: ^0.11.0

Or run flutter pub add dev:flutter_wright.

Quick start

import 'package:flutter_wright/flutter_wright.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await FlutterWright.start(enabled: kDebugMode); // binds in debug, no-op in release
  runApp(const MyApp());
}

That is the whole setup. The control server listens on 127.0.0.1:9123, and /snapshot, /tap, /type, and /scroll work right away.

How it works: snapshot first

1. GET /snapshot returns a semantic tree of the current screen. Every actionable node carries a [ref=sN] handle.
2. Find the ref of the element you want.
3. Send it to an action endpoint, e.g. POST /tap {"ref":"s10"}.

Refs are temporary handles. They change the moment the screen does — navigation, list refresh, rebuild. Take a fresh snapshot before each interaction, and never reuse a ref across screens.

adb forward tcp:9123 tcp:9123                      # Android: reach the device from your laptop
curl localhost:9123/snapshot                       # see the tree and its refs
curl -X POST localhost:9123/tap -d '{"ref":"s10"}' # act on one

HTTP API

Method	Path	Body	Returns
GET	/snapshot	—	semantic tree with [ref=sN]
POST	/tap	{"ref":"sN"}	latest snapshot
POST	/type	{"ref":"sN","text":"…"}	latest snapshot
POST	/scroll	{"ref":"sN", …}	latest snapshot
POST	/long_press	{"ref":"sN"}	latest snapshot
GET	/wait_for	?text=…	latest snapshot
GET	/network	?since=…&clear=…	recorded HTTP requests
POST	/navigate	{"route":"/x","args":{…}}	route + latest snapshot
POST	/reset	{}	latest snapshot
GET	/routes	—	known route names
GET	/screenshot	—	PNG bytes
GET	/health	—	status JSON

Action endpoints wait for the next frame after the action, then return that settled snapshot in their response — so you rarely need a separate /snapshot call after each step. (During a navigation transition the old and new screens may briefly coexist; use /wait_for for a deterministic sync.) Errors always come back as {"ok":false,"error":"…"}.

GET /network (0.11.0+) serves requests recorded at the dart:io HttpClient layer — the http package and Dio's default adapter both sit on it. Each entry carries method/url/status/duration/bytes plus a 4KB-capped body for text-ish responses; ring buffer of 200. Captured bodies are redacted by key name before storage (token/password/secret/api_key/session/cookie → [REDACTED], JSON and form shapes) — a heuristic, not exhaustive: treat /network output as potentially sensitive, especially since its typical consumer is an AI transcript. Recording is on by default (dev-only builds make that safe); opt out with FlutterWrightConfig(recordNetwork: false). Native-channel HTTP (cronet/OkHttp) and WebView-internal traffic are not visible. GET /snapshot?bounds=true appends rect=[l,t,w,h] in device pixels to every node.

Navigation (optional)

/snapshot and the action endpoints work with no extra wiring. To also expose /navigate, /reset, and /routes, tell the SDK how to move between screens.

Navigator 1.0 named routes — share the key:

MaterialApp(navigatorKey: FlutterWright.navigatorKey, /* … */)

GoRouter or GetX — pass an adapter, and the SDK just calls your closures:

await FlutterWright.start(
  enabled: kDebugMode,
  navigationAdapter: CallbackNavigationAdapter(
    // Void block on purpose: Get.toNamed's Future completes only when the
    // pushed route is popped — returning it would hang /navigate.
    // GoRouter: onNavigate: (route, args, _) => router.go(route, extra: args)
    onNavigate: (route, args, _) { Get.toNamed(route, arguments: args); },
    onReset: () => Get.until((route) => route.isFirst),
    routesProvider: () => GetPages.getPages.map((p) => p.name),
  ),
);

Screenshots (optional)

/screenshot only needs setup for Flutter-rendered captures. Wrap your app once:

runApp(FlutterWrightRoot(child: const MyApp()));

Prefer to skip it? Capture externally with adb exec-out screencap and set screenshotMode: ScreenshotMode.external.

WebView H5 (optional)

If your app embeds a WebView, its H5 content is opaque to the snapshot by default. Register a one-line "run this JS" closure and the H5 nodes join the tree like native ones — drivable by the same tap, type, and scroll:

FlutterWright.registerWebView((js) => controller.evaluateJavascript(source: js));

No unregister needed; weak references clean themselves up. Skip it and WebViews simply stay opaque — nothing else changes.

Auth (optional)

Pass a token to require it on every endpoint except /health:

await FlutterWright.start(
  enabled: kDebugMode,
  token: const String.fromEnvironment('FW_TOKEN'), // read from env, never commit
);

Then send it as a header: curl … -H "X-FW-Token: <token>". Without a token, the server is protected only by its loopback binding.

Why "off by default"?

start(enabled: false) does nothing — no socket, no attack surface. You decide when to turn it on: kDebugMode for debug builds, or your own flag such as AppEnv.isTestBuild for QA release builds. That is what makes it safe to ship the call in production code.

License

MIT