Clone Me

A lot of office work is the same seven clicks with different values: close the ticket, set the resolution, leave the comment. Writing a macro for each one costs more than doing it, and nobody writes down the workflows they actually repeat.

Learning them by watching is the obvious answer and the dangerous one. A screen recorder sees passwords, bank pages and API keys pasted into a terminal, and an agent that replays clicks can send, delete or pay for something nobody meant it to.

The recorder takes the foreground app, the URL, each click with the accessibility role and name of what was clicked, typing aggregated per field — never raw keystrokes — and screenshots. Before anything touches disk, a privacy layer drops blocked apps, 131 blocked domains and login or checkout URLs, discards anything typed into a password field, redacts keys, tokens and card numbers, and OCRs each screenshot in memory to black out secrets. Without OCR it takes no screenshots at all.

Episodes are split at idle gaps and app switches, compacted into semantic actions, and mined with a gap-tolerant sequential pattern miner. Values that differ between repetitions become inputs. Claude only names the skill and its inputs, and its answer is validated: it can't add, drop or reorder steps, and it can add a confirm-first flag but never clear one.

Approved skills run dry by default. With --execute, a computer-use loop observes the screenshot and accessibility tree, takes one JSON action from Claude, confirms anything that looks irreversible, acts, and verifies the page against where the recording says that step leads. The browser is locked to the skill's own host and port inside Chromium's network stack, before any request leaves.

• make demo records a scripted week through the real recorder, aggregator and privacy filters: about 340 events become 28 episodes, and the miner finds the 3 workflows that repeat 5–7 times while ignoring the one-off noise.
• The executor drives headless Chromium against the bundled Tracklet app, stops to confirm the irreversible click, and checks Tracklet's own API to prove the ticket was closed with the comment.
• The privacy traps in the demo week — a password manager, a bank, a password field, an API key pasted into a terminal — are dropped or redacted from text and blacked out of the screenshot.
• An independent review found 12 issues, including a host lock that redirects could slip past; all fixed with regression tests. 107 tests in CI on Python 3.10–3.13.