starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

[PR #14097] [MERGED] [wpf] Add UIA events #29928

New Issue
Open
opened 2026-01-31 09:37:40 +00:00 by claunia · 0 comments
claunia commented 2026-01-31 09:37:40 +00:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/microsoft/terminal/pull/14097
Author: @carlos-zamora
Created: 9/28/2022
Status: ✅ Merged
Merged: 10/7/2022
Merged by: @undefined

Base: main ← Head: dev/cazamor/wpf/uia-events

📝 Commits (7)

  • 2aa3f3f [wpf] Add UIA events
  • bf6f0ff enable/disable on focus changed; remove unnecessary bool
  • 9fd73c7 add UIA properties
  • b717ebf Hook up HwndTerminal UIA layer to WPFTermControl
  • adf9aa9 static analysis (and one warning)
  • 1e86ac8 fix PR feedback
  • 967e04f make bstr nothrow

📊 Changes

12 files changed (+370 additions, -45 deletions)

View changed files

📝 OpenConsole.sln (+12 -6)
📝 src/cascadia/PublicTerminalCore/HwndTerminal.cpp (+32 -34)
📝 src/cascadia/PublicTerminalCore/HwndTerminal.hpp (+4 -4)
➕ src/cascadia/PublicTerminalCore/HwndTerminalAutomationPeer.cpp (+159 -0)
➕ src/cascadia/PublicTerminalCore/HwndTerminalAutomationPeer.hpp (+42 -0)
📝 src/cascadia/PublicTerminalCore/PublicTerminalCore.vcxproj (+6 -1)
📝 src/cascadia/PublicTerminalCore/PublicTerminalCore.vcxproj.filters (+6 -0)
📝 src/cascadia/PublicTerminalCore/pch.h (+1 -0)
📝 src/cascadia/WpfTerminalControl/TerminalContainer.cs (+26 -0)
📝 src/cascadia/WpfTerminalControl/TerminalControl.xaml.cs (+32 -0)
📝 src/types/TermControlUiaProvider.cpp (+46 -0)
📝 src/types/TermControlUiaProvider.hpp (+4 -0)

📄 Description

Adds UIA events to the WPF control for the following items:

  • selection changed
  • text changed (and output)
  • cursor changed

Automation Peer

Similar to the architecture of the UWP TermControl, we added a
HwndTerminalAutomationPeer which acts as the
TermControlAutomationPeer in UWP. However, we don't need a XAML
wrapper here, so really we just need it to inherit from
TermControlUiaProvider (the ITextProvider implementation shared
across conhost and WT) and IUiaEventDispatcher (the event dispatching
interface that is responsible for signaling the screen reader that
something has changed).

Removing the local echo

As with WT, we need to record key events to remove the local echo. These
recorded events are matched up with the output text. Each sequential
match is removed in the output text so that it's not read by the screen
reader.

Detecting what to send events for

As with WT, a UiaEngine was added to the renderer and it is set up
when a UIA client is detected. WT would normally stop sending events
when focus was lost from the control. We do the same here.

Automation properties

TermControlUiaProvider was upgraded to support property values. Such
properties include class name and control type. These align with those
set in TermControlAutomationPeer. Realistically, those should point to
these, but that requires a lot more work and a localization burden
(because we need to move the localized word "terminal").

HwndTerminalAutomationPeer takes this a step further and overrides the
class name to be WPFTermControl. This allows screen readers to provide
special handling for the WPFTermControl vs the UWP term control since
they will be updating at different speeds.

Build fixes

To build the WPF test app, I had to mess with the dependencies a little
bit. Really just add the atlas engine and uia renderer to the build
steps.

HwndTerminal initialization

The initialization order with WM_NCCREATE was changed to match that of
Windows Terminal (BaseWindow/IslandWindow). This is safer now. I also
removed the static window because it was unnecessary.

Handling WM_GETOBJECT

WPF's HwndHost likes to mark the WM_GETOBJECT message as handled to
force the usage of the WPF automation peer. We now explicitly mark it as
not handled and don't return an automation peer. This forces the message
to go down to the HwndTerminal where we return terminal's UiaProvider.

Remove TermControl layer from UIA tree

TermContol (the top-most layer in the UIA tree) would pop up and not do
anything. This PR also overrides the automation peer at that layer and
marks IsContentElement/IsControlElement=false (the equivalent to
AccessibilityView=Raw). This makes the layer only appear in the UIA tree
if you are using the raw view (i.e. you know what you're doing and you
want to see each individual layer even if you can't directly interact
with it).

Validation Steps Performed

Tested with Narrator/NVDA using WpfTerminalTestNetCore project in our
repo.

  • New output is read out (not just key events, but also other output
    text)
  • Local echo does not occur (i.e. pressing 'A' should only read 'A'
    once, not twice [key event and rendered letter]).
  • selection events are read out properly
  • cursor change events are read out properly (tested with text
    cursor indicator preview in Settings App > Accessibility > Text
    Cursor)

NOTE: test this with Release builds. Debug builds may be too slow and
not read out properly

Closes #12642

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/microsoft/terminal/pull/14097 **Author:** [@carlos-zamora](https://github.com/carlos-zamora) **Created:** 9/28/2022 **Status:** ✅ Merged **Merged:** 10/7/2022 **Merged by:** [@undefined](undefined) **Base:** `main` ← **Head:** `dev/cazamor/wpf/uia-events` --- ### 📝 Commits (7) - [`2aa3f3f`](https://github.com/microsoft/terminal/commit/2aa3f3fce7615c432fa1ef5d4ae851ad3a590d06) [wpf] Add UIA events - [`bf6f0ff`](https://github.com/microsoft/terminal/commit/bf6f0ff865777fa17e139b354b9ae920d19f68f0) enable/disable on focus changed; remove unnecessary bool - [`9fd73c7`](https://github.com/microsoft/terminal/commit/9fd73c7cbd33e0a7d97ee26ef7c12689333b3bc2) add UIA properties - [`b717ebf`](https://github.com/microsoft/terminal/commit/b717ebf5d6c603b0f724da56d35268baef5d482a) Hook up HwndTerminal UIA layer to WPFTermControl - [`adf9aa9`](https://github.com/microsoft/terminal/commit/adf9aa98c7bb5097bd1dcdabb89c370b64391db8) static analysis (and one warning) - [`1e86ac8`](https://github.com/microsoft/terminal/commit/1e86ac8ee7f3fadb0d62d0202163987ec64aca16) fix PR feedback - [`967e04f`](https://github.com/microsoft/terminal/commit/967e04ff4be0d4e55c51563f6f8e8613fd38ab63) make bstr nothrow ### 📊 Changes **12 files changed** (+370 additions, -45 deletions) <details> <summary>View changed files</summary> 📝 `OpenConsole.sln` (+12 -6) 📝 `src/cascadia/PublicTerminalCore/HwndTerminal.cpp` (+32 -34) 📝 `src/cascadia/PublicTerminalCore/HwndTerminal.hpp` (+4 -4) ➕ `src/cascadia/PublicTerminalCore/HwndTerminalAutomationPeer.cpp` (+159 -0) ➕ `src/cascadia/PublicTerminalCore/HwndTerminalAutomationPeer.hpp` (+42 -0) 📝 `src/cascadia/PublicTerminalCore/PublicTerminalCore.vcxproj` (+6 -1) 📝 `src/cascadia/PublicTerminalCore/PublicTerminalCore.vcxproj.filters` (+6 -0) 📝 `src/cascadia/PublicTerminalCore/pch.h` (+1 -0) 📝 `src/cascadia/WpfTerminalControl/TerminalContainer.cs` (+26 -0) 📝 `src/cascadia/WpfTerminalControl/TerminalControl.xaml.cs` (+32 -0) 📝 `src/types/TermControlUiaProvider.cpp` (+46 -0) 📝 `src/types/TermControlUiaProvider.hpp` (+4 -0) </details> ### 📄 Description Adds UIA events to the WPF control for the following items: - selection changed - text changed (and output) - cursor changed ### Automation Peer Similar to the architecture of the UWP TermControl, we added a `HwndTerminalAutomationPeer` which acts as the `TermControlAutomationPeer` in UWP. However, we don't need a XAML wrapper here, so really we just need it to inherit from `TermControlUiaProvider` (the `ITextProvider` implementation shared across conhost and WT) and `IUiaEventDispatcher` (the event dispatching interface that is responsible for signaling the screen reader that something has changed). ### Removing the local echo As with WT, we need to record key events to remove the local echo. These recorded events are matched up with the output text. Each sequential match is removed in the output text so that it's not read by the screen reader. ### Detecting what to send events for As with WT, a `UiaEngine` was added to the renderer and it is set up when a UIA client is detected. WT would normally stop sending events when focus was lost from the control. We do the same here. ### Automation properties `TermControlUiaProvider` was upgraded to support property values. Such properties include class name and control type. These align with those set in `TermControlAutomationPeer`. Realistically, those should point to these, but that requires a lot more work and a localization burden (because we need to move the localized word "terminal"). `HwndTerminalAutomationPeer` takes this a step further and overrides the class name to be `WPFTermControl`. This allows screen readers to provide special handling for the `WPFTermControl` vs the UWP term control since they will be updating at different speeds. ### Build fixes To build the WPF test app, I had to mess with the dependencies a little bit. Really just add the atlas engine and uia renderer to the build steps. ### HwndTerminal initialization The initialization order with `WM_NCCREATE` was changed to match that of Windows Terminal (BaseWindow/IslandWindow). This is safer now. I also removed the `static` window because it was unnecessary. ### Handling `WM_GETOBJECT` WPF's HwndHost likes to mark the `WM_GETOBJECT` message as handled to force the usage of the WPF automation peer. We now explicitly mark it as not handled and don't return an automation peer. This forces the message to go down to the HwndTerminal where we return terminal's UiaProvider. ### Remove TermControl layer from UIA tree TermContol (the top-most layer in the UIA tree) would pop up and not do anything. This PR also overrides the automation peer at that layer and marks IsContentElement/IsControlElement=false (the equivalent to AccessibilityView=Raw). This makes the layer only appear in the UIA tree if you are using the raw view (i.e. you know what you're doing and you want to see each individual layer even if you can't directly interact with it). ## Validation Steps Performed Tested with Narrator/NVDA using WpfTerminalTestNetCore project in our repo. - [X] New output is read out (not just key events, but also other output text) - [X] Local echo does not occur (i.e. pressing 'A' should only read 'A' once, not twice [key event and rendered letter]). - [X] selection events are read out properly - [X] cursor change events are read out properly (tested with text cursor indicator preview in Settings App > Accessibility > Text Cursor) NOTE: test this with Release builds. Debug builds may be too slow and not read out properly Closes #12642 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-31 09:37:40 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#29928
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?