Andrew Ng has announced OpenWorker, an open-source desktop agent that produces finished work instead of interaction. OpenWorker asks the user outcomeNot the signs: a polished document, a Slack reply with real numbers, an updated calendar, a triangular inbox. It then breaks that result into steps, works on local files and connected apps, and checks before anything consequential happens.
The architecture has four layers, and they all run on your machine.
The repository contains 119 Python files (~32,400 lines) coworker/149 TypeScript/TSX files below surfaces/gui/And 78 backend testing modules.
The stack breaks down as follows:
- desktop shell – A Torii 2 native window that wraps the React 18 UI. is the bundle identifier
com.openworker.desktopAnd the shell itself monitors the Python server. - local agent server – Python 3.10+, bindings on FastAPI and Uvicorn
127.0.0.1:8765As a default. The example configuration caps a turn at 12 model↔tool iterations. - capacitance and connector layer – Verified local tools (files, git, repgrep-backed search, shell, todo) plus hosted integration plus MCP.
- model router – One interface on native, OpenAI-compatible, reseller and local providers.
Engine Suite is built on Andrew Ng’s provider-agnostic LLM library.
Bring your own model from a deliberately small curated list
There is no OpenWorker estimation service. The user pastes an API key, or points the app at the local runtime.
The curated model matrix has exactly 30 entries. Native providers cover OpenAI (GPT-5.6 Sol/Terra/Luna and GPT-5.5), Anthropic (Cloud Fable 5, Opus 4.8, Sonnet 4.6, Haiku 4.5) and Google (Gemini 3.1 Pro, 3.6 Flash, 2.5 Pro, 2.5 Flash). OpenAI-compliant vendors add GLM-5.2, DeepSeek V4, Km K2.6, MiniMax M2.5, Qwen3 Max, Grok 4.3, and Mistral Large. Open-source models come through Together AI and Fireworks, and fully local models come through Olama, requiring no keys.
Permission Engine is the real engineering story
Most desktop agents treat project approval as a UI afterthought. OpenWorker treats them as a typed layer.
Each tool call is classified into one of four risk classes: read (no side effects), write_local (changes workspace, path-scope), exec (runs the command), and external (side effects from the machine). Five permission modes then decide what happens: discuss And plan are read-only, interactive Is the default and asks before writes, commands and external actions, auto The path-field permits everything while it exists, and custom Auto-approves user-listed tool sets.
Two design decisions stand out.
- First, Unattended mode does not increase the autonomy limit – It changes only where humans reach. Prompts that appear inline are sent to the inbox, and the session is suspended until a reply is made.
- Second, permanent scope rules are limited to
externalOnly risk. Shell commands ask forever, by design.
The underlying ops persona also instructs the model to treat the content of tools, logs, web, files, and incoming messages as untrusted data rather than instructions. This is a clear signal-injection posture, written in the persona sent.
Privacy: Local-First
Model calls go directly from the machine to the configured provider. Conversations, connector tokens, and model keys remain local, and the secret store is designed so that secrets never enter model contexts, signals, or traces.
The only cloud component is an optional broker that handles the OAuth handshake for the one-click connector using Auth0 authorization codes with PKCE. Connector tokens are assigned directly to the machine and are never stored in the cloud. The app is fully functional signing out using manually pasted credentials.
key takeaways
- OpenWorker is Andrew Ng’s MIT-licensed desktop AI peer that returns ready deliverables instead of chat replies.
- The local Python FastAPI agent built on Stack Suite is a Tauri 2 + React shell on the server.
- Model Access comes with 30 curated tool-calling models, as well as your own keys to a fully localized Olma.
- A typed risk engine (
read/write_local/exec/external) gates each action into five permission modes.
View the GitHub repo, project site, and announcement. All credit for this research goes to the researchers and developers of this project.
Asif Razzaq Marktechpost Media Inc. Is the CEO of. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. Their most recent endeavor is the launch of MarketTechPost, an Artificial Intelligence media platform, known for its in-depth coverage of Machine Learning and Deep Learning news that is technically robust and easily understood by a wide audience. The platform boasts of over 2 million monthly views, which shows its popularity among the audience.
