UI Automation activities

v26.10

link

Agentic UIAutomation​

Semantic selectors​

Semantic activities​

ScreenPlay​

• SecureString and ScreenPlayImage support in coded workflows — ScreenPlay coded workflows now handle two additional argument types: SecureString, for passing sensitive string values without exposing them in plain text, and ScreenPlayImage, for including a screenshot directly in the ScreenPlay prompt to give the model additional visual context. Previously, coded workflows had no native handling for these types; this change brings them to parity with low-code workflows.
• Distributed tracing SpanId — ScreenPlay now adds an X-UIPATH-SEMANTIC-SPAN-ID header to the HTTP calls it makes to backend services, such as the Semantic Proxy and LLM Gateway. The span identifier lets a single ScreenPlay execution and its requests be correlated end-to-end across services, improving observability and troubleshooting.
• Application instance attach mode — ScreenPlay now supports the Application instance window attach mode on the Use Application/Browser activity. Inner activities search across the entire application instance — all parent and child windows, alerts, and popups — instead of a single window, and ScreenPlay captures and records traces correctly across all of them, in addition to the existing Single window support.
• Refreshed HTML trace file — The ScreenPlay HTML trace file has a refreshed visual design. The trace viewer output now has an updated layout and styling that make execution traces easier to read and navigate, aligned with current UiPath design standards.

Extract UI Data (Preview)​

Healing Agent​

UI Automation support for coding agents​

• Configure a target (uia-configure-target) — Point the agent at a screen and the elements you care about ("set up the Submit button on the checkout page"). It captures the live application, locates each element, generates a reliable selector, registers the screen and elements in the Object Repository, and verifies each one against a screenshot before handing back the reference IDs your workflow can use.
• Interact with a live app (uia-interact) — Ask the agent to interact with a running application — click, type, read values, fill forms, navigate menus, extract tables, or take screenshots. Useful for exploring an app before you automate it, or for confirming that a change behaves as expected.
• Improve a selector (uia-improve-selector) — Fix, harden, or recover a selector from runtime data. See Selector intelligence below.

Selector intelligence (uia-improve-selector)​

• Recover — the selector has stopped working ("element not found", a runtime failure). The agent rewrites it against the current state of the application so it matches the element again.
• Improve — the selector still works but is fragile, because it leans on position, an auto-generated ID, or a value that reflects changing content. The agent produces a more robust version that targets the same element.

• Driven by runtime data. It works from a live application or from data captured at the moment of failure — so a selector that broke on an unattended robot can be recovered later, on a different machine, even when the app is no longer running.
• Activity-aware. The agent knows, for example, that a Get Text target should not lean on the text it is meant to read, and that a Check target should not depend on whether the box is currently ticked. Each rewrite follows the rules that keep a selector stable for its specific use.
• Validated, not guessed. Every candidate selector is checked and scored before anything is written back. If the first attempt doesn't hold up, the agent refines it automatically and tries again, then keeps the strongest result.
• Explained. When a selector is recovered, you get a one-line root cause (why it broke) and the strategy used to make it durable — so you can trust the change at a glance.

Chromium Automation​

• Chromium Automation - Best for development, debugging, unattended automation with a visible browser
• Chromium Automation Headless - Best for unattended automation on servers and CI/CD pipelines

Oracle Automation​

macOS UI Automation​

• Enhanced support for macOS menu-based controls in Select Item activities. You can now use Select Item with AXPopUpButton and AXMenuButton controls, enabling reliable interaction with dropdown and menu-based UI elements that require opening a popup to access their items.
• Added support for triple-click actions. This capability is available on:
  • Windows and macOS using Hardware Events
  • Windows and macOS using ChromiumAPI
• Added support for mouse drag actions on macOS.

UI Automation Modern​

• Enabled the Capture Elements feature in Object Repository in cross-platform projects.

• The Set Runtime Browser activity can now resolve the default system browser at runtime in cross-platform projects. A new Use default system browser option allows selecting between the system default browser or a specific browser:

  • Windows projects: Chrome, Edge, Firefox
  • Cross-platform projects: Chrome, Edge, Safari

  The activity’s display name now reflects the selected browser (literal value, Default system browser, or a truncated expression).

• Added the Accessibility Check activity which enables users to perform Accessibility Testing in Test Manager instances within Test Cloud. This activity loads a web browser page, and performs accessibility checks as per the WCAG 2.1 (or higher) standard. Examples of accessibility checks include DOM and visual accessibility, contrast ratio, alt tags, tab order, improper ARIA roles.

Advanced Selector Editor​

• Added zoom controls to the Advanced Selector Editor and Selection Helper, allowing you to zoom in or out between 80% and 110% using +, -, and Reset options.

• Added case-sensitive support for attributes in the Advanced Selector Editor.

• Added an Edit target option to Use Application/Browser, exposing:

  • Window selector
  • Application properties (such as Browser URL, browser arguments, application path, application arguments, and working directory)
  • The option to Update the window selector in child activities. This allows any changes made to the screen’s Window Selector to be automatically applied to all child elements.

• The Advanced Selector Editor can now be used for editing Window selectors in addition to UI element Selectors.

• In the UI Tree panel of UI Explorer, added a toggle to show or hide properties with empty values in the Selector compatible and Other(s) sections, helping display only relevant information.

• Added a Filter properties option to the UI Tree panel, enabling filtering of properties for the selected node by name in both the Selector compatible and Other(s) sections.

Improvements​

• Optimized Active Accessibility selectors to improve Extract Table Data performance for large tables.

Bug fixes​

• For link type nodes, the text was not retrieved and the tree path was generated using the internal SAP id. With this fix, the proper node text is used. This affected the SAP Expand Tree activity.

• Fixed an issue where selectors could not be identified when interacting with HTML views in the CAMBIO Cosmic application (a Java-based healthcare application using JxBrowser).

• Fixed an issue where the cross-platform Type Into activity became unresponsive on Automation Cloud Robot VMs.

• Fixed native capture for glyphs and Japanese characters.

• Fixed an issue where Java extensions continued to be reported as installed after uninstalling them.