Files
AstrBot/astrbot/core/computer/olayer/gui.py
エイカク cb5c172e69 feat: add CUA computer-use sandbox support (#7828)
* feat: add CUA computer-use sandbox support

* fix: add CUA config metadata translations

* fix: address CUA sandbox review feedback

* fix: default CUA sandbox to local mode

* fix: harden CUA SDK method compatibility

* fix: harden CUA GUI and permission handling

* fix: refine CUA capability and shell handling

* fix: avoid inline CUA screenshot image results by default

* fix: guide CUA browser startup workflow

* feat: add CUA browser and key press tools

* fix: launch CUA browser as sandbox user

* fix: stabilize CUA browser screenshots

* fix: simplify CUA browser launch command

* fix: remove CUA open browser tool

* fix: align CUA desktop control guidance

* fix: harden CUA shell background handling

* fix: harden CUA runtime adapters

* fix: surface CUA filesystem failures

* fix: clarify CUA shell fallback support

* fix: harden CUA shell helpers

* fix: guard CUA file fallbacks

* fix: redact sensitive config log paths

* fix: guard CUA download fallback

* test: cover CUA GUI and shell env wiring

* fix: preserve CUA command result output

* fix: normalize CUA return codes

* fix: preserve foreground shell behavior

* fix: clean up failed CUA boots

* docs: add CUA sandbox runtime guide

* test: cover CUA GUI tool registration

* refactor: simplify CUA fallback handling

* refactor: simplify CUA shell helpers

* test: cover CUA screenshot result shapes
2026-04-28 01:40:14 +09:00

26 lines
681 B
Python

"""
GUI automation component.
"""
from typing import Any, Protocol
class GUIComponent(Protocol):
"""Desktop GUI operations component."""
async def screenshot(self, path: str | None = None) -> dict[str, Any]:
"""Capture a screenshot, optionally saving it to path."""
...
async def click(self, x: int, y: int, button: str = "left") -> dict[str, Any]:
"""Click at screen coordinates."""
...
async def type_text(self, text: str) -> dict[str, Any]:
"""Type text into the active UI target."""
...
async def press_key(self, key: str) -> dict[str, Any]:
"""Press a keyboard key or shortcut."""
...