Wazza OS User guide Sign in

Computer Control

Computer Control is a Chat whose agent can see and use your own computer (not the VM): its screen, its keyboard and its mouse, in your own apps. It opens only after you allow it with your own click, and it stops as soon as you close its tab.

WhatAn agent on your real computer's screen, keyboard and mouse
WhyWork outside the VM: an app installed only on your computer, your desktop's settings
WhoExperienced users who understand what they allow
EndsWhen you close its tab (Stop halts a turn at once)
You do not need it for work in the VM

Every other Chat works in the VM, a separate computer, where an agent cannot harm your own files. Use Computer Control only for a task that must happen on your own computer, and watch it.

Open Computer Control#

  1. Click + (Show apps), then Computer Control in Main apps (Screen control).

  2. A card asks: Open Computer Control? Read it, then click Start. Cancel, Esc or a click outside the card closes it and nothing opens.

The card Open Computer Control? This Chat will look at and act on this computer's screen, keyboard and mouse (your own apps, not a VM's) without asking again until you close its tab. Its own shell and file tools run without asking: they change files only in this Chat's folder. Your own mouse and keyboard, and Stop, always win; closing the tab ends the control. Start, Cancel
The consent card. Only your own real click on Start opens it: an agent, a script or the command line cannot.

The tab then becomes the Computer Control Chat: its name has a red dot, and a red banner over the Chat says what is allowed:

The Computer Control tab with its red dot, and the banner: Computer Control is active: this Chat can see and use this computer's screen, keyboard and mouse without asking again. Its own shell and file tools run without asking too (Wazza OS has no ask mode) and change files only in this Chat's folder. Stop halts it; closing this tab ends it.
Allowed, without asking againNot allowed
Looking at your computer's screen (every window you have open).Anything in another tab: the consent is for this one Chat only.
Clicking, typing and pressing keys in your own apps.Changing files outside this Chat's own folder with its shell and file tools.
Running its own shell and file tools, which change files only in this Chat's folder.Going on after you close the tab, or after Wazza OS restarts.
  • Your own mouse and keyboard always win: if you move or type, your input goes first.
  • Stop halts the agent's current turn at once.
  • Closing the tab ends the control: the permission is withdrawn before the tab closes.
  • After Wazza OS restarts, a restored Computer Control tab says Computer Control needs your consent again: Wazza OS restarted, so this Chat cannot see or use this computer until you allow it, with Allow… to show the card again.

Ask it something#

Write to it like any Chat. Pick a model that takes images (it works by looking at your screen); the model window is the same as every Chat's (Choose the agent and the model).

A Computer Control Chat asked to look at the screen without clicking: the agent read the Drive tool's description, listed the windows (drive_look, seat host), captured the screen, and analysed the picture
"Look at my screen and tell me which apps and windows are open. Do not click or type anything": the agent lists your windows, takes a picture of the screen and reads it, then answers.

Good first requests: "look at my screen and tell me what is open (do not click)", "open the Files app and show me my Downloads folder", "in my desktop settings, turn on the dark style". Say what it must not do as clearly as what it must do. Describing a screen is not always right the first time: in the test above, the agent took the Wazza OS window for a browser; a stronger model sees better.

How it works, and its limits#

  • It uses Drive on your computer's windows (the "host" seat) instead of the VM's: it lists your windows, looks at one or at the whole screen, then clicks and types.
  • It runs on your computer, inside Wazza OS's sandbox: its shell and file tools are limited to its own folder. It reaches the Internet and Wazza OS; services of your computer (localhost) only when Local services is on.
  • Each Computer Control tab has its own permission: opening another one asks again.
  • It sees the screen Wazza OS runs on: your desktop session, or, on a server, Wazza OS's own virtual screen (On a server).

See also Privacy and security.

Wazza OS user manual · written for Wazza OS 0.2.0 (build 44c5d0ea07c1) · screenshots from a test computer running Wazza OS 0.2.0.