Computer Use Agent Node

C

Automation/Computer/Agent

Lets a vision model operate the desktop until a goal is reached: it looks at a screenshot (optionally with numbered accessibility marks), calls mouse and keyboard tools, waits for the screen to settle, checks the result and repeats. Works with any vision model that supports tool calling. Ends when the model reports done or asks the user, or when it is stuck or out of steps or time

computer_use_agentautomationLocal onlyLong running

Inputs13

Outputs9

Security exposure3/10

Packageautomation

Ratings

Scores range from 0 to 10. Higher values mean more impact, exposure, or operational weight.

SecurityAttack surface and exposure impact.

3/10Low

PrivacyPotential sensitivity of processed data.

2/10Low

PerformanceRuntime or resource pressure.

3/10Low

GovernancePolicy, audit, or compliance impact.

4/10Medium

ReliabilityOperational stability considerations.

5/10Medium

CostExternal or compute cost impact.

3/10Low

Input Pins

13

▶

Execution

exec_in

Trigger

Session

Struct

session

Computer session handle

SchemaAutomationSessionAutomationSessionAutomationSession11 fields

session_refstringrequired

platformPlatformrequired

enum "Windows", "MacOS", "Linux"

default_delay_msinteger:uint64required

format uint64min 0

click_delay_msinteger:uint64required

format uint64min 0

debug_modebooleanrequired

browser_typeanyOf (2)

Browser context info if browser is attached

variant 1BrowserTypevariant

enum "Chrome", "Firefox", "Edge", "Safari"

variant 2nullvariant

browser_headlessboolean | null

browser_user_data_dirstring | null

current_page_refstring | null

Current page info if a page is open

current_window_handlestring | null

browser_frame_selectorsArray<Selector>

default []

itemsSelectorarray item

kindSelectorKindrequired

variant 1enumvariant

enum "Css", "Xpath", "TextExact", "TestId"...

variant 2constvariant

Visible text contained in the element; script, style and head content is ignored.

const "Text"

variant 3constvariant

ARIA role, optionally filtered by accessible name: `button`, `button|Sign in` (case-insensitive substring), `button|=Sign in` (exact) or `button|/^sign/i` (regex).

const "Role"

variant 4constvariant

Element ref from the latest Browser Snapshot, such as `e12`.

const "Ref"

valuestringrequired

confidencenumber | null

format double

scopestring | null

Model

Struct

model

Vision model with tool calling that operates the desktop

SchemaBitBitBit19 fields

idstring

default ""

typeBitTypes

enum "Llm", "Vlm", "Tts", "Stt"...default "Other"

metaMap<string, Metadata>

default {}

*Metadatamap value

namestringrequired

descriptionstringrequired

long_descriptionstring | null

release_notesstring | null

tagsArray<string>required

itemsstringarray item

+11 more fields

authorsArray<string>

default []

itemsstringarray item

repositorystring | null

default null

download_linkstring | null

default null

file_namestring | null

default null

hashstring

default ""

sizeinteger | null

format uint64default nullmin 0

hubstring

default ""

parametersvalue

default null

versionstring | null

default null

licensestring | null

default null

dependenciesArray<string>

default []

itemsstringarray item

dependency_tree_hashstring

default ""

createdstring

default ""

updatedstring

default ""

model_slugstring | null

default null

+1 more fields

Schema enforced

Goal

String

goal

The task in plain language, e.g. 'Rename report.txt on the Desktop to final.txt'

Display Index

Integer

display_index

Display the agent sees and acts on; -1 is the primary display

Default -1

Target Window

String

target_window

Optional window title (or part of it): the agent sees the display showing this window and its elements are the numbered ones. Empty uses the focused window outside Flow-Like

Perception

String

perception

screenshot: the plain screenshot; screenshot_marks: numbered boxes on accessibility elements plus an element list; screenshot_marks_ocr: also number recognized text

Default screenshot_marks

screenshotscreenshot_marksscreenshot_marks_ocr

Max Steps

Integer

max_steps

Most model turns before the agent stops with max_steps (1–500)

Default 30

Range 1 to 500Step 1

Max Duration (s)

Integer

max_duration_s

Wall-clock budget in seconds before the agent stops with timeout (1–86400)

Default 600

Range 1 to 86400Step 1

Max Actions per Step

Integer

max_actions_per_step

Most tool calls executed per model turn (1–20)

Default 5

Range 1 to 20Step 1

Settle Time (ms)

Integer

settle_ms

Longest wait after actions for the screen to stop changing before the next screenshot (0–10000)

Default 600

Range 0 to 10000Step 50

Forbidden Text

String Array

forbidden_text

Regular expressions the agent may never type; a matching type call is refused

Default ["rm\\s+-[a-zA-Z]*[rR][a-zA-Z]*\\s+(/|~)","(curl|wget)\\b[^|\\n]*\\|\\s*(sudo\\s+)?(ba|...

Extra Instructions

String

extra_instructions

Optional guidance added to the task, e.g. which app to use or what to avoid

Output Pins

9

Done

Execution

exec_out

The model reported the goal as achieved

Failed

Execution

exec_failed

The model reported failure, the agent was stuck, ran out of steps or time, or hit an error

Needs Input

Execution

exec_needs_input

The model asked the user a question; it is in Answer

Session

Struct

session_out

Computer session handle (pass-through)

SchemaAutomationSessionAutomationSessionAutomationSession11 fields

session_refstringrequired

platformPlatformrequired

enum "Windows", "MacOS", "Linux"

default_delay_msinteger:uint64required

format uint64min 0

click_delay_msinteger:uint64required

format uint64min 0

debug_modebooleanrequired

browser_typeanyOf (2)

Browser context info if browser is attached

variant 1BrowserTypevariant

enum "Chrome", "Firefox", "Edge", "Safari"

variant 2nullvariant

browser_headlessboolean | null

browser_user_data_dirstring | null

current_page_refstring | null

Current page info if a page is open

current_window_handlestring | null

browser_frame_selectorsArray<Selector>

default []

itemsSelectorarray item

kindSelectorKindrequired

variant 1enumvariant

enum "Css", "Xpath", "TextExact", "TestId"...

variant 2constvariant

Visible text contained in the element; script, style and head content is ignored.

const "Text"

variant 3constvariant

ARIA role, optionally filtered by accessible name: `button`, `button|Sign in` (case-insensitive substring), `button|=Sign in` (exact) or `button|/^sign/i` (regex).

const "Role"

variant 4constvariant

Element ref from the latest Browser Snapshot, such as `e12`.

const "Ref"

valuestringrequired

confidencenumber | null

format double

scopestring | null

Status

String

status

done, failed, stuck, max_steps, timeout or needs_input

Answer

String

answer

The model's result or summary, the question for the user, or why the agent stopped

Steps

Struct Array

steps

Trajectory: per turn the screenshot seen, the model's text, each action with its arguments, desktop coordinates and result, and the duration

SchemaAgentStepAgentStepAgentStep5 fields

indexinteger:uint32required

1-based turn number.

format uint32min 0

observationstringrequired

The screenshot the model looked at.

assistantstringrequired

Text the model wrote next to its tool calls.

actionsArray<AgentAction>required

itemsAgentActionarray item

One tool call of a turn.

toolstringrequired

argsvaluerequired

Arguments as the model sent them; coordinates are pixels of its screenshot.

input_pointsArray<InputPoint>

Desktop input coordinates the action used.

default []

itemsInputPointarray item

skippedboolean

Not run: an earlier call of the turn failed, the turn limit was reached or the task ended.

default false

okbooleanrequired

+1 more fields

duration_msinteger:uint64required

format uint64min 0

Final Screenshot

Struct

final_image

The last screenshot, without marks

SchemaNodeImageNodeImageNodeImage1 fields

image_refstringrequired

Final Frame

Struct

final_frame

Desktop rectangle and pixel size of the final screenshot

SchemaScreenFrameScreenFrameScreenFrame7 fields

display_indexinteger | null

format uint32default nullmin 0

xinteger:int32required

format int32

yinteger:int32required

format int32

widthinteger:uint32required

format uint32min 0

heightinteger:uint32required

format uint32min 0

pixel_widthinteger:uint32required

format uint32min 0

pixel_heightinteger:uint32required

format uint32min 0

Node Info

Internal name

computer_use_agent

Category

Automation/Computer/Agent

Version

1