Add a means by which UIA clients can determine whether a build of Windows Terminal does or doesn't support notifications #18045

Open
opened 2026-01-31 06:02:12 +00:00 by claunia · 8 comments
Owner

Originally created by @codeofdusk on GitHub (Jul 26, 2022).

Originally assigned to: @carlos-zamora on GitHub.

Description of the new feature/enhancement

#12358 adds support for UIA notifications, allowing screen readers and other UIA clients to easily determine exactly what new text to report without doing any of the heavy lifting themselves (diffing, etc..). However, as not all terminals support notifications, UIA clients need a way to determine when this support is available and when they should fall back to performing diffing themseles. (if clients diff even when notifications are available, we risk doing extra work and/or double-reporting incoming text).

Proposed technical implementation details (optional)

  • In #12358 and over email at various times, we discussed using a different class name (TermCtrl2) for situations when notifications are available. NVDA supports this new ID today, but there were concerns from JAWS.
  • We also discussed sending an "API version" of sorts through a custom property. This is necessary since wt and conhost versions aren't necessarily tied to particular Windows releases/aren't reliable for feature detection.
Originally created by @codeofdusk on GitHub (Jul 26, 2022). Originally assigned to: @carlos-zamora on GitHub. <!-- 🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨 I ACKNOWLEDGE THE FOLLOWING BEFORE PROCEEDING: 1. If I delete this entire template and go my own path, the core team may close my issue without further explanation or engagement. 2. If I list multiple bugs/concerns in this one issue, the core team may close my issue without further explanation or engagement. 3. If I write an issue that has many duplicates, the core team may close my issue without further explanation or engagement (and without necessarily spending time to find the exact duplicate ID number). 4. If I leave the title incomplete when filing the issue, the core team may close my issue without further explanation or engagement. 5. If I file something completely blank in the body, the core team may close my issue without further explanation or engagement. All good? Then proceed! --> # Description of the new feature/enhancement #12358 adds support for UIA notifications, allowing screen readers and other UIA clients to easily determine exactly what new text to report without doing any of the heavy lifting themselves (diffing, etc..). However, as not all terminals support notifications, UIA clients need a way to determine when this support is available and when they should fall back to performing diffing themseles. (if clients diff even when notifications are available, we risk doing extra work and/or double-reporting incoming text). <!-- A clear and concise description of what the problem is that the new feature would solve. Describe why and how a user would use this new functionality (if applicable). --> # Proposed technical implementation details (optional) * In #12358 and over email at various times, we discussed using a different class name (TermCtrl2) for situations when notifications are available. NVDA supports this new ID today, but there were concerns from JAWS. * We also discussed sending an "API version" of sorts through a custom property. This is necessary since wt and conhost versions aren't necessarily tied to particular Windows releases/aren't reliable for feature detection.
Author
Owner

@codeofdusk commented on GitHub (Jul 26, 2022):

Blocking nvaccess/nvda#13781.

CC @carlos-zamora, @leonardder.

@codeofdusk commented on GitHub (Jul 26, 2022): Blocking nvaccess/nvda#13781. CC @carlos-zamora, @leonardder.
Author
Owner

@LeonarddeR commented on GitHub (Jul 26, 2022):

It could also be considered to add an automationID (it is currently empty)

@LeonarddeR commented on GitHub (Jul 26, 2022): It could also be considered to add an automationID (it is currently empty)
Author
Owner

@codeofdusk commented on GitHub (Jul 26, 2022):

It could also be considered to add an automationID (it is currently empty)

@leonardder That idea was rejected during initial implementation.

@codeofdusk commented on GitHub (Jul 26, 2022): > It could also be considered to add an automationID (it is currently empty) @leonardder That idea [was rejected](https://github.com/microsoft/terminal/pull/12358#issuecomment-1034328089) during initial implementation.
Author
Owner

@carlos-zamora commented on GitHub (Jul 27, 2022):

Chatted with @codeofdusk. This is a feature/task. We'll chat more about it at sync but the short version is this:

  • In a world where Terminal doesn't send notifications for text output...
    • NVDA reads the text buffer, performs a diff on what was stored, and reads out what changed.
    • Diffing is (unsurprisingly) pretty expensive! This is a common practice among screen readers.
  • Today, Terminal sends notifications for text output to tell the screen reader exactly what is relevant...
    • NVDA would normally read these notifications, resulting in a "local echo" where it would read the notification and the result of the diff. Not good! So NVDA straight up ignores those notifications.
    • That said, these notifications uncovered a massive opportunity for screen readers! What if they just trusted the notification instead of diffing themselves? That would be a huge performance enhancement!
  • The Problem: NVDA needs some kind of check to know if Terminal will send out these notifications. If that check succeeds, then they know they can trust our notifications, and they can avoid diffing altogether.
    • NOTE: NVDA already does these "checks" on UIA. There's 3 "API Levels":
      • 0 --> "GetVisibleRange" returns one range per line of text --> initial UIA implementation (pre-UIA provider refactor)
      • 1 --> "GetVisibleRange" returns one range total --> after my UIA provider refactor
      • 2 --> text formatting (i.e. color/font) is exposed --> most recent UIA implementation
@carlos-zamora commented on GitHub (Jul 27, 2022): Chatted with @codeofdusk. This is a feature/task. We'll chat more about it at sync but the short version is this: - In a world where Terminal doesn't send notifications for text output... - NVDA reads the text buffer, performs a diff on what was stored, and reads out what changed. - Diffing is (unsurprisingly) pretty expensive! This is a common practice among screen readers. - Today, Terminal sends notifications for text output to tell the screen reader exactly what is relevant... - NVDA would normally read these notifications, resulting in a "local echo" where it would read the notification and the result of the diff. Not good! So NVDA straight up ignores those notifications. - That said, these notifications uncovered a _massive_ opportunity for screen readers! What if they just trusted the notification instead of diffing themselves? That would be a huge performance enhancement! - The Problem: NVDA needs some kind of check to know if Terminal will send out these notifications. If that check succeeds, then they know they can trust our notifications, and they can avoid diffing altogether. - NOTE: NVDA already does these "checks" on UIA. There's 3 "API Levels": - 0 --> "GetVisibleRange" returns one range per line of text --> initial UIA implementation (pre-UIA provider refactor) - 1 --> "GetVisibleRange" returns one range total --> after my UIA provider refactor - 2 --> text formatting (i.e. color/font) is exposed --> most recent UIA implementation
Author
Owner

@zadjii-msft commented on GitHub (Aug 15, 2022):

discussion: sure let's do it

@zadjii-msft commented on GitHub (Aug 15, 2022): discussion: sure let's do it
Author
Owner

@carlos-zamora commented on GitHub (Aug 16, 2022):

@codeofdusk here's an idea actually. So, all versions of Windows Terminal in active support going back to v1.12 send notifications. At this point, why not just assume that if NVDA is connected to Windows Terminal, it has support for notifications?

The key part here is "connected to Windows Terminal" because Conhost won't support notifications. So if the class name is TermControl, you know you're connected to Windows Terminal, and by extension, you know we support notifications.

CC @DHowett

@carlos-zamora commented on GitHub (Aug 16, 2022): @codeofdusk here's an idea actually. So, all versions of Windows Terminal in active support going back to v1.12 send notifications. At this point, why not just assume that if NVDA is connected to Windows Terminal, it has support for notifications? The key part here is "connected to Windows Terminal" because Conhost won't support notifications. So if the class name is `TermControl`, you know you're connected to Windows Terminal, and by extension, you know we support notifications. CC @DHowett
Author
Owner

@DHowett commented on GitHub (Aug 16, 2022):

Conhost won't support notifications

Until SV2

@DHowett commented on GitHub (Aug 16, 2022): > Conhost won't support notifications Until SV2
Author
Owner

@carlos-zamora commented on GitHub (Aug 16, 2022):

Conhost won't support notifications

Until SV2

Nah. Conhost doesn't use the UiaRenderer. So the whole notifications thing doesn't affect Conhost. It operates the same way it always has (I believe it's attached to the Write API but don't quote me on that one, I'd have to look into it).

@carlos-zamora commented on GitHub (Aug 16, 2022): > > Conhost won't support notifications > > Until SV2 Nah. Conhost doesn't use the UiaRenderer. So the whole notifications thing doesn't affect Conhost. It operates the same way it always has (I believe it's attached to the `Write` API but don't quote me on that one, I'd have to look into it).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#18045