Computer use

#How computer use works

The loop has four steps. The application that hosts the model takes a screenshot and sends it to the model. The model returns an action, for example moving the cursor to a point, clicking, typing text or scrolling. The application runs that action on the computer and takes a new screenshot. The loop continues until the model reports that the task is complete or the application stops it.

The Anthropic computer use tool follows this pattern. The developer's application runs each action and returns the next screenshot to the model.

#Why it matters for AI agents

Computer use works on any interface a person can see. The agent does not need an API from the app it controls. The cost is speed and money. Each step needs a model call with a screenshot, so a task with many steps is slow and expensive.

The same property creates risk. The model acts on a real screen, and text on that screen can try to change what the model does. This is prompt injection. Anthropic's documentation recommends a dedicated, sandboxed environment with limited privileges for this tool.

#Computer use, browser agents and APIs

A browser agent is usually a narrower form of computer use. The model controls a web browser instead of the whole desktop. Some browser agents read the page structure as well as screenshots.

#Related terms

• Browser agent
• AI agent
• Prompt injection
• Human in the loop

#FAQ

#Is computer use the same as a browser agent?

No. Computer use is the capability to control a screen with a mouse and keyboard. A browser agent applies that kind of control to a web browser. A browser agent may use screenshots, page content or both.

#What are the risks of computer use?

A model can click the wrong control, and each click can change real data. Text on a page or screen can also contain instructions that the model may follow. Run the tool in a sandboxed environment, and require a person to approve sensitive actions.