More data won’t make a smartphone agent better on its own. The OpenMobile-2 team ran a controlled test where the data budget stayed the same and only the number of different apps the agent trained on changed. On apps it had never seen before, its success rate went from 60.7 to 67.5 percent.
That result tells you more than any single benchmark score in the release. The open-source side of this race seems to be short on variety, and adding volume won’t solve that.
The project comes from researchers at Nanjing University, the AI company SenseTime and several other universities. Their argument is that AI agents that operate apps on their own learn mostly by practicing, and most of the data that practice needs is held by large companies. That leaves open models behind.
The team’s response is two agent models. One is built on Qwen3.5 with 9 billion parameters, the other on Qwen3.6 with 27 billion. They’re also publishing the full training environment, the data and the instructions to reproduce all of it. According to the authors, their model sits near the top of today’s smartphone agents.
Real phones showed the largest improvement
Emulators let a lot slide. Real Android devices don’t, and that is where OpenMobile-2 gained the most. The larger model climbed from 31.9 to 59.6 points on SPA-Bench, which the researchers say is the highest score for any open model.
Scores on the established benchmarks went up as well. On AndroidWorld, the 27-billion-parameter model went from 67.1 to 79.9 points, and the 9-billion model rose from 54.3 to 74. OpenMobile-1, the earlier version, had scored 64.7 on that test.
MobileWorld is tougher, with longer tasks that cross several apps. There the larger model moved from 35.9 to 50.4 points. It’s a good gain, though the agent still fails roughly half of its multi-app tasks.
Why the researchers rebuilt 35 real apps
Live apps are a poor place to train. Logins, actual purchases and no way to reset state make each run costly and hard to repeat. The researchers rebuilt 35 apps instead, mostly ones that are popular in China (including Taobao, JD, DiDi and Meituan), along with services used around the world such as Slack and Google Drive.
With existing simulations and apps running in an Android emulator added in, the agent trains across more than 110 applications.
Human reviewers checked every rebuilt app against the original and kept tweaking it until the workflows and interfaces lined up. Orders, budgets and travel plans persist from one session to the next, and the clipboard can carry data between apps. Every simulation returns to its starting state automatically.
All the agent sees is screenshots. It has no access to internal app data, so it’s in about the same spot as a person holding the phone.
That setup gave the team roughly 11,600 example workflows and over 2,000 practice tasks with automated success checks.
A closed model still did the teaching
The “open” label comes with a caveat. The examples were generated by Google’s Gemini 3.1 Pro as the teacher model, which means OpenMobile-2 is effectively learning from a proprietary system. Its weights and pipeline are open, but part of the knowledge inside them came from Google.
Reinforcement learning on top of that training added little. Most of the gains appeared in environments that resembled the training setup. The larger model actually slipped a bit on AndroidWorld after RL, scoring 79.9 points instead of 80.8.
Shortcuts pay off once the agent learns them
The most experimental piece of the study drops tapping and swiping altogether. Over 50 of the simulated apps offer direct commands like “add to cart,” so the agent can skip digging through menus. It can move between those commands and regular screen interaction depending on what an app supports.
Without extra work, this did almost nothing. The models mostly ignored the shortcuts unless they had been trained on them.
With targeted training, the agents completed up to 10.3 percentage points more tasks and used about five fewer steps. The researchers say the skill carries over to tools the models didn’t encounter during training.
During the later RL phase, runs that used at least one direct command received a bonus factor, but the reward still depended on finishing the task. Claiming success without actually succeeding earned a penalty. For now these interfaces only exist in the simulated apps, and the researchers describe the approach as experimental.
A benchmark scored by code instead of another model
Some of the results come from a custom benchmark with 215 tasks, close to half of which span multiple apps. No language model judges whether a task worked. A program compares the app’s stored data from before and after each run.
That check is stricter than letting one model grade another. The authors also searched for overlap between training and test tasks and found only small similarities.
How it compares with Microsoft, Oppo and Google
The shortcut work ties into a debate that has been growing. A 2025 Microsoft Research study rated API agents as more reliable and GUI agents as more flexible, and named mobile apps as a typical example of software without external interfaces.
Oppo is taking another path with its open agent X-OmniClaw. It runs on real devices and uses deeplinks to replay paths it was previously shown in apps like Taobao or Meituan, so it doesn’t have to click through the menus again.
On the proprietary side, Google has put smartphone control straight into Gemini 3.5 Flash. That agent now has an open-model benchmark to beat: OpenMobile-2’s 59.6 on real-device SPA-Bench, a test anyone can rerun with the released environment and data.