Computer-Use & Browser Agents

Most agents act through APIs — clean, structured connections to software. Computer-use and browser agents take a different, more general approach: they operate a graphical interface directly, looking at the screen, deciding where to click and type, and navigating apps and websites the way a human would. This lets them work with software that has no API at all.

This topic covers that frontier: the model-lab computer-use capabilities and browser agents that can carry out multi-step tasks across real interfaces. It is one of the most consequential and least mature areas of agent research, with enormous potential reach and real safety questions.

The genuine AI is a multimodal model that can perceive a screen, reason about interface state, and produce the right sequence of clicks and keystrokes to accomplish a goal. The technical frontier is reliability across the endless variety of real interfaces, and the safety frontier is just as serious — an agent that can operate your computer can also take harmful actions, so permissioning, sandboxing, and human oversight are central. This is early, powerful, and genuinely load-bearing AI; learners should weigh its reach against its immaturity and the accountability questions it raises.