google/artemis translates plain-English test instructions into reliable cross-app automation on real Android hardware via ADB, with Logcat and screenshot collection for authorized QA workflows.

google/artemis is a Python-based automation agent that lets AI assistants and test suites operate real Android phones the way a human tester would. Given a natural-language instruction such as "open Settings, find Battery and tell me the current level," it drives a physical device or emulator over ADB, observes the screen, and performs the UI interactions needed to complete the task. The repository pitches it squarely at the mobile QA and test-engineering crowd, and its roughly 8,100 stars plus topics like test-automation and ai-agents suggest it has landed well with that audience. For security professionals, the interesting angle is that it turns an owned test device into a scriptable, evidence-collecting endpoint for both functional and exploratory testing.

Architecturally, artemis is a reactive observe-and-act loop wrapped around standard Android tooling. The quick-start script detects and installs the system prerequisites it needs — ADB, scrcpy, FFmpeg, and Python dependencies managed through uv — so the operator does not have to assemble the stack manually. A connected device with USB Debugging enabled, or an emulator, is the only real precondition. The tool then layers multimodal perception on top of the device feed: the README describes "multimodal targeting" that prefers element indices when the accessibility tree exposes them, with coordinate and visual locating fallbacks for custom interfaces where indices are unavailable.

That "dynamic-first, coordinate-fallback" locator strategy is one of the more thoughtful design decisions documented in the repo. Rather than depending solely on brittle coordinate taps or solely on accessibility metadata that many apps fail to expose properly, artemis degrades gracefully between the two. The behavioral rules file shipped at mcp_server/rules.md encodes this pattern along with guidance on latency and timing compensation, so an AI agent driving the device does not hallucinate UI interactions or tap before a screen has settled. It is the kind of engineering note that signals the project was built by people who have actually fought flaky mobile test suites.

Two execution profiles are exposed. flash is the fast reactive mode, with asynchronous history summaries that keep step latency in the 3–5 second range, and it is the default for straightforward tasks. pro is the deeper reasoning mode aimed at long-running exploratory and stability testing; the README notes it checks targets before individual actions and returns blocked actions to the operator for recovery rather than silently failing. The distinction matters in practice — a CI regression suite wants flash speed, while an exploratory stability soak on a pre-release build benefits from pro's pre-action verification and its tolerance for apps that misbehave.

The headline benchmark claim is a 99%+ task completion rate on Google Research's AndroidWorld benchmark, which comprises over a hundred multi-step cross-app tasks. Because AndroidWorld is itself a public Google Research artifact, the number is at least reproducible in principle by anyone with the hardware, which is more than can be said for most agent-framework benchmarks. The demo workflow in the README — setting up driving routes in Google Maps, computing durations, then switching to YouTube to play a track — illustrates the cross-app chaining that makes this genuinely harder than single-app UI scripting.

What elevates artemis beyond a standalone CLI toy is its native Model Context Protocol server. uv run artemis mcp --install all wires the device farm into AI IDEs including Antigravity, Claude Code, Cursor, Codex, Windsurf, and VS Code-adjacent setups like Cline/Roo. Once mounted, the IDE agent can call tools such as mobile_run_task, mobile_manage_task, mobile_get_device_state, mobile_inspect_trace, and mobile_diagnose — a verb set that covers dispatch, lifecycle management, state inspection, trace analysis, and post-failure diagnosis. The documented workflow moves from prompt input, to test-plan generation, to autonomous device execution, to a structured final report with audit findings and metric tables.

This MCP integration is where the tool becomes interesting for defenders and appsec teams, because the diagnostic channel is built in. During execution, artemis collects Logcat output and screenshots alongside the action stream, so a failed or anomalous test comes with the evidence needed to determine whether the app crashed, threw an unexpected dialog, or simply rendered slowly. The example IDE prompt in the README — build an APK, install it, open the login screen with a test account, check for unexpected popups, and return screenshots — is effectively a mini smoke-test harness expressed in one sentence.

Operationally, there are four usage modes that map cleanly onto different team roles. The web console, launched with uv run artemis ui on localhost:8000, provides live screen mirroring, a prompt sandbox, execution replay, and a task dashboard, and can be managed from any terminal with uv run artemis restart/stop/status. The MCP server serves IDE-integrated workflows, the developer CLI (uv run artemis run) handles scripted test cases and benchmark runs, and a zero-runtime-dependency Python SDK (artemis-client) plugs the same capability into existing pytest fixtures or CI/CD pipelines with strongly typed Pydantic structured outputs.

The SDK design deserves a note: the client package is deliberately thin, with ADB, the agents, the model backends, and image processing all remaining on the device host rather than the developer workstation. An ArtemisClient instance points at the host over HTTP, optionally pins a device_serial such as emulator-5554, and returns results carrying a trace_id and success status that assertions can key off. That split makes it feasible to centralize a device host and have many CI jobs hit it without shipping the full automation stack into every pipeline runner.

Multimodal model support is broad — the README badges Gemini, Claude, GPT-4o, and Qwen-VL — which means teams are not locked to a single vision-language backend for the perception layer. The first task on a device installs an on-device helper, so it is worth reading that section of the README carefully and understanding exactly what artifacts land on the phone before pointing it at anything sensitive; on a managed test device this is routine, but it is the kind of detail a careful operator verifies rather than assumes.

From a security-auditing standpoint, artemis is best understood as a capability amplifier for authorized testing: it lowers the cost of systematically exercising an app's UI surface, reproducing bugs, and capturing forensic evidence along the way. It requires USB Debugging on the target device, which in practice means a device you control or are contracted to test — the trust boundary is the same as any ADB workflow. Everything it does is observable on the device side, from the installed helper to the Logcat activity, so there is no stealth component and nothing that lends itself to covert use; it is a test tool, and its telemetry trail is a feature, not a leak.

With Apache-2.0 licensing, Python 3.12+ requirements, an active Discord community, and bilingual English/Chinese documentation, the project presents as production-oriented rather than a research throwaway. For mobile QA teams drowning in device-labor, for appsec engineers who want reproducible UI-level evidence attached to every finding, and for anyone building an AI-assisted development loop around real hardware, artemis is one of the more complete entries in the fast-growing device-agent category.

Educational analysis for authorized security professionals. Use only in controlled, authorized environments.