What is computer use?
"Computer use" is when an AI model operates a computer the way a person does: it looks at the screen, then clicks, types and scrolls to get a task done. It works with any app or website, including old ones that have no API.
How it works
- Look. The agent takes a screenshot. Some agents also read the page structure (the browser's DOM or the operating system's accessibility tree), which is faster and more precise than pixels alone.
- Decide. The model works out the next step towards the goal: "click the Export button", "type the invoice number".
- Act. The step is executed as a real mouse or keyboard action.
- Repeat until the task is done, it gets stuck, or it needs a human to approve something.
Browser agents do this inside a web browser only. Desktop agents control the whole computer: files, desktop apps and the browser.
Where you'll find it
- Inside AI apps: Claude Cowork, ChatGPT Work, Gemini Agent and Manus all include an agent that can operate a computer or browser for you. See the comparison.
- As a model API: Anthropic, Google and OpenAI let developers build their own agents on top of their models.
- In enterprise automation: Microsoft Copilot Studio and RPA vendors such as UiPath are adding computer use to replace brittle scripted bots.
- Open source: frameworks like Browser Use and Cua. See open-source agents.
What it does well, and what it doesn't
One infrastructure vendor's 2026 assessment groups tasks roughly like this:
| Works well | Works with careful design | Still early |
|---|---|---|
| Data extraction, copying data between apps, form filling, software testing | Transactional workflows, legacy ERP systems, vendor portals, customer support | Long tasks with no supervision, drag-and-drop |
Source: Deck, State of Computer Use Agents 2026 (a vendor).
On the main desktop benchmark, OSWorld-Verified, top agents now score in the mid-80s percent, above the human baseline of about 72%. On harder tests they finish well under half of tasks. See benchmarks.
What it costs
a16z estimated in August 2026 that a computer-use agent costs about $6–8 per hour of work, against about $10 for offshore outsourcing and $30–45 for a US back-office worker. Agents are also slower: one example task took 8–10 minutes versus 2–3 for a person.
Source: a16z, Can agents use a computer yet?
The main risk
An agent that reads web pages and emails can be tricked by hidden instructions in them, called prompt injection. Run agents in a sandbox, limit which sites they can reach, and require a human to approve anything irreversible. The security checklist covers this in detail.