starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-03 21:25:34 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

Alt-clicking a profile doesn't open in a pane when Narrator is turned on #17781

New Issue
Open
opened 2026-01-31 05:53:17 +00:00 by claunia · 15 comments
claunia commented 2026-01-31 05:53:17 +00:00
Owner
Copy Link

Originally created by @j4james on GitHub (Jun 23, 2022).

Windows Terminal version

1.13.10336.0

Windows build number

10.0.19044.1706

Other Software

Windows Narrator

Steps to reproduce

  1. Open any shell in Windows Terminal.
  2. Open the profile menu and click on an entry while pressing the Alt key.
  3. Note that the profile is opened in a pane of the current window. Close that pane.
  4. Now turn on the Windows Narrator screen reader.
  5. Open the profile menu again and click on an entry while pressing the Alt key.
  6. Note that the profile is now opened in a separate tab.

Expected Behavior

Alt-clicking on a profile should always open it in a pane of the current window.

Actual Behavior

When the narrator is turned on, alt-clicking opens the profile in a separate tab.

Btw, if there's some accessibility reason for this, that's fine. I was just testing something with the narrator and was surprised by the behavior, but it's not something that affects me personally.

Originally created by @j4james on GitHub (Jun 23, 2022). ### Windows Terminal version 1.13.10336.0 ### Windows build number 10.0.19044.1706 ### Other Software Windows Narrator ### Steps to reproduce 1. Open any shell in Windows Terminal. 2. Open the profile menu and click on an entry while pressing the <kbd>Alt</kbd> key. 3. Note that the profile is opened in a pane of the current window. Close that pane. 4. Now turn on the Windows Narrator screen reader. 5. Open the profile menu again and click on an entry while pressing the <kbd>Alt</kbd> key. 6. Note that the profile is now opened in a separate tab. ### Expected Behavior Alt-clicking on a profile should always open it in a pane of the current window. ### Actual Behavior When the narrator is turned on, alt-clicking opens the profile in a separate tab. Btw, if there's some accessibility reason for this, that's fine. I was just testing something with the narrator and was surprised by the behavior, but it's not something that affects me personally.
claunia commented 2026-01-31 05:53:18 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jun 23, 2022):

WOW. Yea that's wild. Why doesn't that work?

Thanks for finding this one!

@zadjii-msft commented on GitHub (Jun 23, 2022): WOW. Yea that's wild. Why doesn't that work? Thanks for finding this one!
claunia commented 2026-01-31 05:53:18 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Jun 23, 2022):

I did some logging of the alt state being tested here:
24a53d4968/src/cascadia/TerminalApp/TerminalPage.cpp (L1003-L1006)

And it seems that alt key is just not registering as Down when the Narrator is on. Sometimes the state shows as Locked, so I thought maybe we were supposed to be testing for either Locked or Down, but sometimes it's just 0. Maybe this is a bug in winrt?

@j4james commented on GitHub (Jun 23, 2022): I did some logging of the alt state being tested here: https://github.com/microsoft/terminal/blob/24a53d4968f8a1c5648cbc49a4be2d0bc0b28a72/src/cascadia/TerminalApp/TerminalPage.cpp#L1003-L1006 And it seems that alt key is just not registering as `Down` when the Narrator is on. Sometimes the state shows as `Locked`, so I thought maybe we were supposed to be testing for either `Locked` or `Down`, but sometimes it's just 0. Maybe this is a bug in winrt?
claunia commented 2026-01-31 05:53:19 +00:00
Author
Owner
Copy Link

@carlos-zamora commented on GitHub (Oct 27, 2022):

Issue still repros after #14221 merged.

However, I'd bet that we could fix by leveraging the IDirectKeyListener. Assigning to myself and giving it a try this cycle if I have time. If it doesn't work, back to the 22H2 milestone it goes!

@carlos-zamora commented on GitHub (Oct 27, 2022): Issue still repros after #14221 merged. However, I'd bet that we could fix by leveraging the `IDirectKeyListener`. Assigning to myself and giving it a try this cycle if I have time. If it doesn't work, back to the 22H2 milestone it goes!
claunia commented 2026-01-31 05:53:19 +00:00
Author
Owner
Copy Link

@carlos-zamora commented on GitHub (Oct 27, 2022):

Ok well, played around with this for a bit. Surprised to see that the Alt key doesn't get detected by TerminalPage::OnDirectKeyEvent(). I went as far as to make a class that implements MenuFlyoutItem and IDirectKeyListener and use that for the profiles, but the Alt key isn't hit there either.

So yeah, that's weird. No idea where the Alt key is being picked up. Back to 22H2, I guess.

@carlos-zamora commented on GitHub (Oct 27, 2022): Ok well, played around with this for a bit. Surprised to see that the `Alt` key doesn't get detected by `TerminalPage::OnDirectKeyEvent()`. I went as far as to make a class that implements `MenuFlyoutItem` and `IDirectKeyListener` and use that for the profiles, but the `Alt` key isn't hit there either. So yeah, that's weird. No idea where the `Alt` key is being picked up. Back to 22H2, I guess.
claunia commented 2026-01-31 05:53:19 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Oct 27, 2022):

I thought TerminalPage::OnDirectKeyEvent only got triggered for very specific keypresses involving alt... IIRC it was only for alt key releases, alt+space, and for f7.

I bet it's this:

    bool AppLogic::OnDirectKeyEvent(const uint32_t vkey, const uint8_t scanCode, const bool down)
    {
        if (_root)
        {
            // Manually bubble the OnDirectKeyEvent event up through the focus tree.
            auto xamlRoot{ _root->XamlRoot() };
            auto focusedObject{ Windows::UI::Xaml::Input::FocusManager::GetFocusedElement(xamlRoot) };
            do {...}

A MenuFlyout isn't a child of the XamlRoot, so we'd never bubble through this OnDirectKeyEvent call to a MenuFlyoutItem that did implement it.

@zadjii-msft commented on GitHub (Oct 27, 2022): I thought `TerminalPage::OnDirectKeyEvent` only got triggered for very specific keypresses involving alt... IIRC it was only for alt key releases, alt+space, and for f7. I bet it's this: ```c++ bool AppLogic::OnDirectKeyEvent(const uint32_t vkey, const uint8_t scanCode, const bool down) { if (_root) { // Manually bubble the OnDirectKeyEvent event up through the focus tree. auto xamlRoot{ _root->XamlRoot() }; auto focusedObject{ Windows::UI::Xaml::Input::FocusManager::GetFocusedElement(xamlRoot) }; do {...} ``` A `MenuFlyout` isn't a child of the `XamlRoot`, so we'd never bubble through this `OnDirectKeyEvent` call to a `MenuFlyoutItem` that did implement it.
claunia commented 2026-01-31 05:53:19 +00:00
Author
Owner
Copy Link

@marcelwgn commented on GitHub (Jan 8, 2023):

I'm inclined to say this is just an issue with the CoreWindow API and Narrator interfering. Both GetKeyEvent and GetAsyncKeyEvent don't seem to reliably return the status of that element.

So is the solution to manually listen for keydown/keyup to know if the key is currently pressed? Do we need to do that for other keys beside alt then? Regarding OnDirectKeyEvent, while it currently only raises for Alt and F7 being released, we could also modify the behavior to raise for alt being pressed.

@marcelwgn commented on GitHub (Jan 8, 2023): I'm inclined to say this is just an issue with the CoreWindow API and Narrator interfering. Both GetKeyEvent and GetAsyncKeyEvent don't seem to reliably return the status of that element. So is the solution to manually listen for keydown/keyup to know if the key is currently pressed? Do we need to do that for other keys beside alt then? Regarding OnDirectKeyEvent, while it currently only raises for Alt and F7 being released, we could also modify the behavior to raise for alt being pressed.
claunia commented 2026-01-31 05:53:20 +00:00
Author
Owner
Copy Link

@marcelwgn commented on GitHub (Jan 8, 2023):

Hmm I think there isn't really a way for us to sensibly work around this. I tried adding an "Alt key down" check in here but that didn't seem to get triggered if Narrator is open. My guess would be that Narrator is more powerful than us here, after all Narrator has the shortcuts Alt+Narrator-Key+Left/Alt+Narrator-Key++Right to move the narration.

@marcelwgn commented on GitHub (Jan 8, 2023): Hmm I think there isn't really a way for us to sensibly work around this. I tried adding an "Alt key down" check in [here](https://github.com/microsoft/terminal/blob/main/src/cascadia/WindowsTerminal/main.cpp#L149-L179) but that didn't seem to get triggered if Narrator is open. My guess would be that Narrator is more powerful than us here, after all Narrator has the shortcuts Alt+Narrator-Key+Left/Alt+Narrator-Key++Right to move the narration.
claunia commented 2026-01-31 05:53:20 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jan 9, 2023):

Welp, that's surely annoying.... Wait there's gotta be a way, right? Like, tapping alt in Edge still focuses the menu, even with narrator open, so there's gotta be some way of getting alt keypresses.

Unless there's some weird interplay between XAML islands and narrator here, like... the input not even making it to the root HWND. That seems insane though...

Yea, there's gotta be a way. PowerToys can still get Alt keypresses even with narrator........

@zadjii-msft commented on GitHub (Jan 9, 2023): Welp, that's surely annoying.... Wait there's gotta be a way, right? Like, tapping `alt` in Edge still focuses the menu, even with narrator open, so there's gotta be _some_ way of getting alt keypresses. Unless there's some weird interplay between XAML islands and narrator here, like... the input not even making it to the root HWND. That seems insane though... Yea, there's gotta be a way. PowerToys can still get `Alt` keypresses even with narrator........
claunia commented 2026-01-31 05:53:20 +00:00
Author
Owner
Copy Link

@marcelwgn commented on GitHub (Jan 9, 2023):

Not sure how the alt hotkey works in Edge, but maybe Edge is not listening for Alt Key down but rather Alt Key up to activate?

The issue in our case is that we receive the alt down event after the button was clicked. It seems that the AltKeyDown gets triggered when the alt key has been lifted or you click on a button. In what scenario does powertoys handle the alt key?

@marcelwgn commented on GitHub (Jan 9, 2023): Not sure how the alt hotkey works in Edge, but maybe Edge is not listening for Alt Key down but rather Alt Key up to activate? The issue in our case is that we receive the alt down event after the button was clicked. It seems that the AltKeyDown gets triggered when the alt key has been lifted or you click on a button. In what scenario does powertoys handle the alt key?
claunia commented 2026-01-31 05:53:21 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jan 9, 2023):

I was using the page that lets you rebind the Powertoys Run keybinding, but lo and behold, it doesn't work on key down with narrator enabled either. I guess I wasn't paying close enough attention 😅

@zadjii-msft commented on GitHub (Jan 9, 2023): I was using the page that lets you rebind the Powertoys Run keybinding, but lo and behold, it doesn't work on key _down_ with narrator enabled either. I guess I wasn't paying close enough attention 😅
claunia commented 2026-01-31 05:53:21 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jan 10, 2023):

/2am_thought: if we manually track alt-down/up, that won't work. If Someone alt-tabs away from us, we'll get an alt down, but not an alt up, and permanently think alt was pressed. yikes.

@zadjii-msft commented on GitHub (Jan 10, 2023): /2am_thought: if we manually track alt-down/up, that won't work. If Someone alt-tabs away from us, we'll get an alt down, but not an alt up, and permanently think alt was pressed. yikes.
claunia commented 2026-01-31 05:53:21 +00:00
Author
Owner
Copy Link

@marcelwgn commented on GitHub (Jan 10, 2023):

Great...

So are we really left to find a way to get GetKeyState/GetAsynKeyState to work reliably? Or we find another way that does not involve us manually tracking key presses/key lifts for alt key (and possible other keys that might not work) but I can't think of any other API beside the already mentioned ones.

Another (hacky and not that elegant) workaround would be to detect narrator being started and show a warning indicating that alt hotkeys might not properly working.

@marcelwgn commented on GitHub (Jan 10, 2023): Great... So are we really left to find a way to get GetKeyState/GetAsynKeyState to work reliably? Or we find another way that does not involve us manually tracking key presses/key lifts for alt key (and possible other keys that might not work) but I can't think of any other API beside the already mentioned ones. Another (hacky and not that elegant) workaround would be to detect narrator being started and show a warning indicating that alt hotkeys might not properly working.
claunia commented 2026-01-31 05:53:21 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jan 10, 2023):

We're reaching out to the Narrator team (internally) for some help on this one. I'm starting to worry this is a Hard problem / entirely unsolvable :(

oh god I had an immensely dumb idea, but we could filter all the win32 mouse events in winMain, look for alt in the modifiers there, and if we get a mouse+click event there, stash a boolean in AppLogic/TerminalPage as a one-time-use "hey was alt held for this louse event".

That would be crazy dumb though. Let's hope narrator has a better idea.

@zadjii-msft commented on GitHub (Jan 10, 2023): We're reaching out to the Narrator team (internally) for some help on this one. I'm starting to worry this is a **H**ard problem / entirely unsolvable :( oh god I had an immensely dumb idea, but we could filter all the win32 mouse events in `winMain`, look for `alt` in the modifiers there, and if we get a mouse+click event there, stash a boolean in AppLogic/TerminalPage as a one-time-use "hey was alt held for this louse event". That would be crazy dumb though. Let's hope narrator has a better idea.
claunia commented 2026-01-31 05:53:21 +00:00
Author
Owner
Copy Link

@marcelwgn commented on GitHub (Jan 10, 2023):

I don't want to sound too pessimistic but how would you look at the alt modifiers inside the message loop? The WM_LBUTTONDOWN event mentions that you should use GetKeyState to check for alt being pressed, similar is also said for the WM_POINTERUP event. And well, if GetKeyState would work correctly, we wouldn't have that problem in the first place.

Really looking forward to what the narrator team is saying.

PS: I'm really amazed how much I learn as part of this issue, thank you @zadjii-msft, @carlos-zamora and @j4james !

@marcelwgn commented on GitHub (Jan 10, 2023): I don't want to sound too pessimistic but how would you look at the alt modifiers inside the message loop? The [WM_LBUTTONDOWN event](https://learn.microsoft.com/en-us/windows/win32/inputdev/wm-lbuttondown#remarks) mentions that you should use GetKeyState to check for alt being pressed, similar is also said for the [WM_POINTERUP event](https://learn.microsoft.com/en-us/windows/win32/inputmsg/wm-pointerup#remarks). And well, if GetKeyState would work correctly, we wouldn't have that problem in the first place. Really looking forward to what the narrator team is saying. PS: I'm really amazed how much I learn as part of this issue, thank you @zadjii-msft, @carlos-zamora and @j4james !
claunia commented 2026-01-31 05:53:22 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jan 10, 2023):

psh pessimism is like our main export here on the Terminal 😝 We're all learning a lot about the platform this week, that's for sure

@zadjii-msft commented on GitHub (Jan 10, 2023): psh pessimism is like our main export here on the Terminal 😝 We're all learning a lot about the platform this week, that's for sure
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label Area-Accessibility Area-UserInterface Help Wanted Issue-Bug Priority-1 Product-Terminal
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#17781
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?