GitHub Copilot can now interact with desktop apps with computer use
By Steven Van ·
Available on macOS and Windows in Copilot CLI and the Copilot app, it is off by default and asks approval before controlling an app.
Computer use is now in public preview in GitHub Copilot CLI and the GitHub Copilot app, on macOS and Windows. Copilot can interact with desktop applications by reading accessible app content and visual context, clicking controls, entering and editing text, pressing keys, scrolling, dragging, and moving through workflows across applications.
The aim is software with no API, command-line interface or MCP integration, such as legacy and GUI-only apps. Examples from the docs include summarising information in a legacy desktop application, updating content in a presentation, and moving information between applications in a multi-step workflow. Where an API, terminal command or dedicated tool can do the job, the docs say that tool usually gives more structured and predictable results.
- Off by default. In Copilot CLI, run /computer on, with /computer show to check status and /computer off to disable it. In the Copilot app, open Settings, select Computer Use and turn on Enable Computer Use, or use /computer on.
- Approval. Copilot asks before controlling an app. You can allow it for the session, save the approval, or deny it. Saved "Always allow" decisions are stored locally and shared between the CLI and the app, and can be reviewed and removed in the app.
- macOS permissions. Setup walks through the Accessibility and Screen Recording permissions.
- Stopping a run. Press Esc twice in the CLI. In the app, click Stop or press Esc.
- Organisations. Enterprise administrators can disable the feature through managed settings, and a local setting does not override that policy.
The docs warn that Copilot can pick the wrong control or type in the wrong place, and that unexpected on-screen content can lead to unintended actions. They advise against "Always allow" for apps with sensitive information or high-impact actions.
