Maestro: The Open-Source E2E Testing Framework AI Agents Can Write and Run

Mobile Testing Is the Work Nobody Does

Every mobile team knows it should ship UI tests. Almost none do. Appium setups eat a day, test code has to compile, flaky tests page you at 2 a.m., and maintaining the tests costs more than fixing the bugs they catch. So most teams do the rational thing: skip the tests, regress by hand before every release.

Coding agents made that hole deeper. Your agent ships features fast, but it has never seen your simulator screen. It has no idea whether the login flow it just rewrote actually opens. Coding speed went up; the verification gap went up with it.

Maestro is betting the other way. It’s an open-source UI testing framework from mobile.dev, 15.4k stars on GitHub, Apache 2.0 licensed. The pitch isn’t “yet another test framework.” It’s two design choices: tests are YAML, readable by humans and machines alike, and an official MCP server plugs your emulator’s screen and controls straight into your coding agent. E2E tests stop being a chore nobody maintains and become a verification layer agents write, run, and repair themselves.

What Maestro Is

Maestro is mobile.dev’s (mobile-dev-inc) open-source UI and end-to-end testing framework for Android, iOS, and web apps, including React Native, Flutter, and hybrid apps. The README promises your first test in under five minutes.

It was built on lessons from the generation before it: Appium, Espresso, UIAutomator, XCTest, Selenium, Playwright. The repo has 1,739 commits and 934 forks, with a Kotlin core.

Four surfaces, and the open-source line is drawn clearly:

Surface	Open source	Cost	What it does
CLI	Apache 2.0	Free	The core engine that runs your flows
MCP	Bundled with the CLI	Free	Gives coding agents eyes and hands on a device
Studio	Closed	Free	Visual desktop IDE for Mac, Windows, Linux
Cloud	Closed	Paid	Parallel device farm with CI reporting

The open-source part covers a full local workflow. Studio is free but you won’t find its code in the repo. Cloud is where you go when the suite outgrows your laptop, and the official claim is up to 90% shorter execution times.

Three Design Choices That Cut the Cost of Tests

YAML as the test medium

A minimal flow looks like this:

appId: com.example.app
---
- launchApp
- tapOn: "Login"
- inputText: "test@example.com"
- tapOn: "Continue"
- assertVisible: "Welcome back"

One format, three wins. Humans can review it in a pull request and the diffs stay legible. Flows are interpreted, so there’s no compile step between an edit and a run. And LLMs are unusually good at producing structured YAML, which means an agent writing your tests costs almost nothing.

Compare that with Appium-era tests: the same coverage, written in Java or Python, compiled, wrapped in page objects. Different species, not just different verbosity.

Waiting built in

Most flaky UI tests fail because they’re too fast: the element hasn’t rendered and the tap already fired. The traditional patch is sleep, and the suite gets slower every quarter and still flakes.

Maestro commands like tapOn and assertVisible wait until the element is actually tappable or visible. No manual sleeps. That’s the core of what the docs mean by flakiness tolerance.

Black box, on purpose

Maestro drives your app the way a user does, looking at the screen and tapping things. It doesn’t care what’s under the hood, so the same flow runs on native, React Native, and Flutter. The trade: it can only assert what’s visible on screen. More on that in the limits section.

MCP: Closing the Loop for Agents

This is Maestro’s play for the agent era. The MCP server ships inside the CLI and exposes a toolset to your agent:

• list_devices: list local Android emulators, iOS simulators, and Chromium instances
• inspect_screen: read the current screen’s view hierarchy as JSON
• take_screenshot: capture the screen
• run: execute flows, inline YAML or files
• cheat_sheet: the command reference the agent consults before writing unfamiliar syntax
• run_on_cloud: submit runs to Cloud and poll for results

inspect_screen is the eyes; run is the hands. When your agent rewrites the login flow, it can open the emulator, run the test, read the hierarchy when it fails, patch the flow, and run it again. Verification happens before the PR, not after CI goes red.

There’s also Maestro Viewer. Ask your agent to “open the maestro viewer” and a live simulator embeds in your IDE, so you can watch the agent tap through the app in real time.

One premise is easy to miss: AI-generated tests must be auditable. If the agent produced a compiled binary or a knot of scripts, you’d have to trust it. What Maestro produces is a dozen lines of YAML you can read at a glance. Deterministic, repeatable, human-readable. That’s the precondition for handing testing to an agent at all.

Up and Running in Five Minutes

You need Java 17 or higher first. Then one command installs the CLI:

curl -fsSL "https://get.maestro.mobile.dev" | bash

Wire it into your agent. Claude Code:

claude mcp add maestro -- maestro mcp

Codex:

codex mcp add maestro -- maestro mcp

For other agents (Cursor, Gemini CLI, Copilot, and the rest), the docs list per-client configs, but the generic entry is one block:

{
  "mcpServers": {
    "maestro": {
      "command": "maestro",
      "args": ["mcp"]
    }
  }
}

Then give your agent one instruction: open the emulator, write a Maestro test for the login screen, and run it until it passes.

When Maestro Is the Wrong Tool

• Unit and logic tests are still your first line of defense. Maestro covers the UI layer end to end; don’t use it to test pure logic.
• The black-box stance means assertions stop at “what’s visible on screen.” Data correctness still needs API or unit tests behind it.
• The Java 17 runtime is a hard requirement. If your environment can’t take it, you’re done before you start.
• The open-source part is the CLI and MCP. Studio is free but closed, Cloud is paid. Be clear about what you get free and what you’d pay for.
• If your team is deep in Appium with a large existing suite, weigh migration on its own merits. Agentic workflows alone aren’t a reason to switch.

The Takeaway

Mobile testing isn’t skipped because it doesn’t matter. It’s skipped because writing and maintaining tests used to be human work, all of it. Maestro moves that cost: agents write the YAML, agents run it through MCP, and humans review a dozen readable lines.

If you already build mobile features with an agent and still regress by hand, the next step is small: install the CLI, connect the MCP, and hand your agent its first test. That’s the missing piece of the agentic loop.

Repo: https://github.com/mobile-dev-inc/maestro Docs: https://docs.maestro.dev MCP setup: https://docs.maestro.dev/get-started/maestro-mcp