ENABLE_VIRTUAL_TERMINAL_INPUT and child processes #20273

Closed
opened 2026-01-31 07:08:43 +00:00 by claunia · 6 comments
Owner

Originally created by @amoldeshpande on GitHub (Jul 21, 2023).

Windows Terminal version

1.17.11461.0

Windows build number

10.0.22621.1992]

Other Software

windows tcsh, setting the following:

SetConsoleMode(ghstdin, dwmode | ENABLE_VIRTUAL_TERMINAL_INPUT | ENABLE_WINDOW_INPUT))

Steps to reproduce

This is not directly related to running in Windows Terminal, but IIRC you own the conhost as well, so I hope this question/bug belongs here.

I have a Windows native version of tcsh that sets ENABLE_VIRTUAL_TERMINAL_INPUT to handle escape codes and such.

What I find is that when you use ReadConsoleInput on a console handle with this setting, the Virtual Key code for ESCAPE is not set (i.e., INPUT_RECORD.KeyEvent.wVirtualKeyCode is 0, not 0x1b)

The uChar.AsciiChar is set to 0x1b.

Now, this is not a problem for this shell as it is not looking for VK_xxx, but just looking at ascii chars.

However, this setting seems to be inherited by the child process, so a console application that is too old to know (or does not care) about this behavior will get shafted if it expects VK_xyz to be populated.

In my specific case, the child process is a very old hex editor that seems (by looking at disassembly, since I don't have source) to be expecting a non-zero wVirtualKeyCode (or scan code, not quite sure).

I can work around this issue by disabling virtual key processing every time I call createprocess and then re-enabling it after the call.

The issue/bug is whether this behavior is correct.

Expected Behavior

Child process should be unaware of parent's VT behavior

Actual Behavior

as described above.

Originally created by @amoldeshpande on GitHub (Jul 21, 2023). ### Windows Terminal version 1.17.11461.0 ### Windows build number 10.0.22621.1992] ### Other Software windows tcsh, setting the following: SetConsoleMode(ghstdin, dwmode | ENABLE_VIRTUAL_TERMINAL_INPUT | ENABLE_WINDOW_INPUT)) ### Steps to reproduce This is not directly related to running in Windows Terminal, but IIRC you own the conhost as well, so I hope this question/bug belongs here. I have a Windows native version of tcsh that sets ENABLE_VIRTUAL_TERMINAL_INPUT to handle escape codes and such. What I find is that when you use ReadConsoleInput on a console handle with this setting, the Virtual Key code for ESCAPE is not set (i.e., INPUT_RECORD.KeyEvent.wVirtualKeyCode is 0, not 0x1b) The uChar.AsciiChar is set to 0x1b. Now, this is not a problem for this shell as it is not looking for VK_xxx, but just looking at ascii chars. However, this setting seems to be inherited by the child process, so a console application that is too old to know (or does not care) about this behavior will get shafted if it expects VK_xyz to be populated. In my specific case, the child process is a very old hex editor that seems (by looking at disassembly, since I don't have source) to be expecting a non-zero wVirtualKeyCode (or scan code, not quite sure). I can work around this issue by disabling virtual key processing every time I call createprocess and then re-enabling it after the call. The issue/bug is whether this behavior is correct. ### Expected Behavior Child process should be unaware of parent's VT behavior ### Actual Behavior as described above.
claunia added the Issue-BugResolution-Duplicate labels 2026-01-31 07:08:43 +00:00
Author
Owner

@DHowett commented on GitHub (Jul 21, 2023):

Hey, thanks for the report! You're in the right place for conhost as well, since conhost is built out of this repo!

You're totally right when you say that child processes should be unaware of their parents' behavior (and vice-versa)... but unfortunately, that's the way the console API was designed ages upon ages ago.
I have some significant worries about changing that now, but I'm happy to say that we're tracking making the input modes per-process (and solving the overall story) in #4954. We can chat about what it'll mean to change this over there.

In general, shells on Windows have had to sanitize the console modes before they spawn applications. CMD uses VT output but disables it before spawning an application for this exact reason. PowerShell is in the middle, and it changes the modes all the time (which is a big surprise in all of my trace logging!).

If you don't mind, I'm going to mark this as a /duplicate of #4954. Let me know if you disagree!

@DHowett commented on GitHub (Jul 21, 2023): Hey, thanks for the report! You're in the right place for conhost as well, since conhost is built out of this repo! You're totally right when you say that child processes should be unaware of their parents' behavior (and vice-versa)... but unfortunately, that's the way the console API was designed ages upon ages ago. I have some significant worries about changing that now, _but_ I'm happy to say that we're tracking making the input modes per-process (and solving the overall story) in #4954. We can chat about what it'll mean to change this over there. In general, shells on Windows have had to sanitize the console modes before they spawn applications. CMD uses VT _output_ but disables it before spawning an application for this exact reason. PowerShell is in the middle, and it changes the modes _all the time_ (which is a big surprise in all of my trace logging!). If you don't mind, I'm going to mark this as a /duplicate of #4954. Let me know if you disagree!
Author
Owner

@microsoft-github-policy-service[bot] commented on GitHub (Jul 21, 2023):

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

@microsoft-github-policy-service[bot] commented on GitHub (Jul 21, 2023): Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!
Author
Owner

@amoldeshpande commented on GitHub (Jul 22, 2023):

Can you explain why the KEY_EVENT_RECORD does not have the virtual key code in this mode?
It seems like a VT app could ignore the key code while old apps could continue to use it, but I'm probably missing some nuance.

@amoldeshpande commented on GitHub (Jul 22, 2023): Can you explain why the KEY_EVENT_RECORD does not have the virtual key code in this mode? It seems like a VT app could ignore the key code while old apps could continue to use it, but I'm probably missing some nuance.
Author
Owner

@DHowett commented on GitHub (Jul 22, 2023):

You know, that's an excellent question! I'll have to dig a bit to figure it out.

I think there's some compatibility behavior here. Perhaps that applications that are expecting VKs won't do anything with the "synthetic" escape key from the control sequence if it's zero?

@DHowett commented on GitHub (Jul 22, 2023): You know, that's an excellent question! I'll have to dig a bit to figure it out. I think there's some compatibility behavior here. Perhaps that applications that are expecting VKs won't do anything with the "synthetic" escape key from the control sequence if it's zero?
Author
Owner

@amoldeshpande commented on GitHub (Jul 22, 2023):

hmm, I changed my code so I can't look at the behavior right now. But if I recall correctly, pressing ESC only generated the single key code, not a sequence. So it should ideally have just worked if the VK code was set.

Anyway, I would love to know if you ever discover the reason. thanks!

@amoldeshpande commented on GitHub (Jul 22, 2023): hmm, I changed my code so I can't look at the behavior right now. But if I recall correctly, pressing ESC only generated the single key code, not a sequence. So it should ideally have just worked if the VK code was set. Anyway, I would love to know if you ever discover the reason. thanks!
Author
Owner

@amoldeshpande commented on GitHub (Jul 22, 2023):

The more I think about it, the more it seems like ESC should be passed through with Virtual Keycode as before. If it's part of a multi-byte escape sequence, the application should be able to handle that.

@amoldeshpande commented on GitHub (Jul 22, 2023): The more I think about it, the more it seems like ESC should be passed through with Virtual Keycode as before. If it's part of a multi-byte escape sequence, the application should be able to handle that.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#20273