cotomi Act: Browser Agent Learns Tasks by Observing the User
A new research paper presents cotomi Act, a browser-based AI agent that autonomously learns user tasks by observing interactions and converting them into executable instructions and organisational knowledge.
What happened?
cotomi Act is an AI agent integrated into the web browser, developed to automate tasks by passively observing user behaviour. The system combines reliable multi-step task execution with the ability to create persistent organisational knowledge from user actions. The agent scored 80.4% on WebArena’s evaluation set of 179 tasks for human evaluation, exceeding the reported human baseline of 78.2%.
Key facts
| Publikationsdatum | 2026-05-30 |
|---|---|
| WebArena prestanda | 80.4% |
| Mänsklig baslinje | 78.2% |
| Antal uppgifter i WebArena | 179 |
”What if a browser agent could learn your work simply by watching you do it? We present cotomi Act, a browser-based computer-using agent that combines reliable multi-step task execution with persistent organizational knowledge learned from user behavior.”
”For execution, an agent scaffold with adaptive lazy observation, verbal-diff-based history compression, coarse-grained actions, and test-time scaling via best-of-N action selection achieves 80.4% on the 179-task WebArena human-evaluation subset, exceeding the reported 78.2% human”
Why it matters
This development is significant as it could potentially revolutionise how software interacts with users, creating automation solutions and building knowledge bases. By having the AI learn directly from observation, the need for explicit programming or extensive configuration is reduced, paving the way for more adaptive and personal digital assistants. The system generates artefacts such as task boards and wikis from observed behaviour.
Who is affected?
Principally, software developers and researchers in AI and automation are affected, as this represents an advancement in cognitive agents and machine learning. In the long run, end-users of browser-based applications may benefit from access to more advanced and self-learning automation tools.
What else you should know
The technology utilises an agentic stance with adaptive latent observation, verbal-diff-based history compression, coarse-grained actions, and test-time scaling via best-of-N action selection. A controlled proxy evaluation confirmed that task success improves as behaviour-based knowledge accumulates.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Vem påverkas främst?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
AI-verktyg i artikeln
Topics
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "cotomi Act: Browser Agent Learns Tasks by Observing the User"