Coding Agents with Multimodal Browsing
Free while signed in. Answers cite the passages they came from.
Introduces OpenHands-Versa, a unified agent designed to perform strongly across diverse domains, coding, web browsing, and multimodal information access, by equipping a single agent with three general capabilities: code execution, multimodal web browsing, and file/search access. In contrast to specialist or multi-agent systems optimized for narrow domains, OpenHands-Versa aims to solve a wide variety of real-world tasks with minimal architectural complexity. Key highlights:
Unified Toolset, Superior Coverage: OpenHands-Versa integrates visual web browsing, search API access, and multimodal file processing into the OpenHands coding framework. Despite its simplicity, it surpasses specialized agents across three benchmarks: SWE-Bench Multimodal (+9.1%), GAIA (+1.3%), and The Agent Company (+9.1%) in success rates.
Benchmark Generalization: The agent matches or outperforms multi-agent systems like OWL-roleplaying and Magentic-One, which struggle to generalize across domains. For example, OWL-roleplaying, though strong on GAIA, performs poorly on The Agent Company due to limited tool generality.
Domain-Aware Tool Use: Analysis reveals that OpenHands-Versa effectively adapts its tool usage per benchmark (e.g., search APIs in GAIA, browser in The Agent Company, and visual validation in SWE-Bench M), unlike its predecessor, OpenHands, which misuses or lacks crucial tools like search.
Minimal Agent, Strong Results: By relying on a single-agent design and Claude-3.7 or Claude Sonnet-4 as backbone LLMs, OpenHands-Versa achieves SOTA results without per-task tool customization. For example, it attains 64.24% on GAIA val split, outperforming multi-agent baselines by up to +18%.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack