ENABLE_VIRTUAL_TERMINAL_PROCESSING is set by default in pseudo console on Windows 10 20H2. #12943

Closed
opened 2026-01-31 03:29:34 +00:00 by claunia · 14 comments
Owner

Originally created by @tyan0 on GitHub (Mar 10, 2021).

Environment

Windows 10 20H2 (Version 10.0.19042.804)

Steps to reproduce

  1. Compile the following test code.
  2. Run the test program in pseudo console (e.g. in WSL).
#include <windows.h>
#include <stdio.h>

int main()
{
	DWORD mode;
	GetConsoleMode(GetStdHandle(STD_OUTPUT_HANDLE), &mode);
	printf("%d\n", mode & ENABLE_VIRTUAL_TERMINAL_PROCESSING);
	return 0;
}

Expected behavior

At least on older Windows 10 such as 1809, the result is:

0

Actual behavior

On Windows 20H2, the result is:

4

Is this the intentional behavior?

Originally created by @tyan0 on GitHub (Mar 10, 2021). # Environment Windows 10 20H2 (Version 10.0.19042.804) # Steps to reproduce 1. Compile the following test code. 2. Run the test program in pseudo console (e.g. in WSL). ``` #include <windows.h> #include <stdio.h> int main() { DWORD mode; GetConsoleMode(GetStdHandle(STD_OUTPUT_HANDLE), &mode); printf("%d\n", mode & ENABLE_VIRTUAL_TERMINAL_PROCESSING); return 0; } ``` # Expected behavior At least on older Windows 10 such as 1809, the result is: ``` 0 ``` # Actual behavior On Windows 20H2, the result is: ``` 4 ``` Is this the intentional behavior?
claunia added the Resolution-Duplicate label 2026-01-31 03:29:34 +00:00
Author
Owner

@eryksun commented on GitHub (Mar 10, 2021):

Is this the intentional behavior?

Yes. See src/host/srvinit.cpp.

@eryksun commented on GitHub (Mar 10, 2021): >Is this the intentional behavior? Yes. See [src/host/srvinit.cpp](https://github.com/microsoft/terminal/blob/8f9ccc55d16d1f803f01488f8b78d3cf09503fe5/src/host/srvinit.cpp#L155).
Author
Owner

@zadjii-msft commented on GitHub (Mar 10, 2021):

Yep, this is by design.

/dup #1965

@zadjii-msft commented on GitHub (Mar 10, 2021): Yep, this is by design. /dup #1965
Author
Owner

@ghost commented on GitHub (Mar 10, 2021):

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

@ghost commented on GitHub (Mar 10, 2021): Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!
Author
Owner

@tyan0 commented on GitHub (Mar 10, 2021):

Thank you for your quick response. Just a question from my interest, is it possible to disable this flag by default in pseudo console? Setting HKCU\Console\VirtualTerminalLevel to 0 does not take effect.

@tyan0 commented on GitHub (Mar 10, 2021): Thank you for your quick response. Just a question from my interest, is it possible to disable this flag by default in pseudo console? Setting ```HKCU\Console\VirtualTerminalLevel``` to ```0``` does not take effect.
Author
Owner

@zadjii-msft commented on GitHub (Mar 10, 2021):

A better question might be: why do you want to disable VT processing? It might help to better understand your scenario.

@zadjii-msft commented on GitHub (Mar 10, 2021): A better question might be: _why do you want to disable VT processing_? It might help to better understand your scenario.
Author
Owner

@tyan0 commented on GitHub (Mar 10, 2021):

I asked just because I was interested. I don't have the scenario in which this behavior causes any problem so far.

@tyan0 commented on GitHub (Mar 10, 2021): I asked just because I was interested. I don't have the scenario in which this behavior causes any problem so far.
Author
Owner

@vefatica commented on GitHub (Mar 10, 2021):

I followed the link provided by @eryksun and I'm just curious ... is the conditional expression below (from srvinit.cpp) well-formed?

bool isEnabled = false;
if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(isEnabled) && isEnabled))
@vefatica commented on GitHub (Mar 10, 2021): I followed the link provided by @eryksun and I'm just curious ... is the conditional expression below (from srvinit.cpp) well-formed? ``` bool isEnabled = false; if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(isEnabled) && isEnabled)) ```
Author
Owner

@zadjii-msft commented on GitHub (Mar 10, 2021):

@vefatica Sure is. That statement is basically two parts:

  1. CheckDefaultAppPolicy will get the state of the "default app policy", and stick it into the bool passed in. CheckDefaultAppPolicy will return true if it succeeded in looking that value up.
  2. Was the "default app policy" enabled?
@zadjii-msft commented on GitHub (Mar 10, 2021): @vefatica Sure is. That statement is basically two parts: 1. `CheckDefaultAppPolicy` will get the state of the "default app policy", and stick it into the bool passed in. `CheckDefaultAppPolicy` will return true if it succeeded in looking that value up. 2. Was the "default app policy" enabled?
Author
Owner

@vefatica commented on GitHub (Mar 10, 2021):

In that case I'd expect it to look like this:

if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(&isEnabled)) && isEnabled)

@vefatica commented on GitHub (Mar 10, 2021): In that case I'd expect it to look like this: `if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(&isEnabled)) && isEnabled)`
Author
Owner

@zadjii-msft commented on GitHub (Mar 10, 2021):

Meh, stylistic difference. Taking a param by reference means the callee doesn't need to worry about if the out param is null or not. Most of our newer code with out params (as few as those are) uses params by reference, rather than pointers.

@zadjii-msft commented on GitHub (Mar 10, 2021): Meh, stylistic difference. Taking a param by reference means the callee doesn't need to worry about if the out param is null or not. Most of our newer code with out params (as few as those are) uses params by reference, rather than pointers.
Author
Owner

@eryksun commented on GitHub (Mar 10, 2021):

bool isEnabled = false;
if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(isEnabled) && isEnabled))

The only argument of the SUCCEEDED macro should be the HRESULT that's returned by CheckDefaultAppPolicy, in order to get a boolean result of success or not. Negative HRESULT values indicate failure. Doing a logical && on an HRESULT and a bool is incorrect.

@eryksun commented on GitHub (Mar 10, 2021): > ``` > bool isEnabled = false; > if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(isEnabled) && isEnabled)) > ``` The only argument of the `SUCCEEDED` macro should be the `HRESULT` that's returned by `CheckDefaultAppPolicy`, in order to get a boolean result of success or not. Negative `HRESULT` values indicate failure. Doing a logical `&&` on an `HRESULT` and a `bool` is incorrect.
Author
Owner

@vefatica commented on GitHub (Mar 10, 2021):

OK, didn't know that was by reference. And the second isEnabled being inside SUCCEEDED(). It just looks really funny to me.

@vefatica commented on GitHub (Mar 10, 2021): OK, didn't know that was by reference. And the second isEnabled being inside SUCCEEDED(). It just looks really funny to me.
Author
Owner

@zadjii-msft commented on GitHub (Mar 10, 2021):

oh no

@zadjii-msft commented on GitHub (Mar 10, 2021): oh no
Author
Owner

@miniksa commented on GitHub (Mar 10, 2021):

WELP. I am fixing this as MSFT:32071839. We will replicate that back out to fix my parenthesis error in the next day or so. Good catch and thank you.

@miniksa commented on GitHub (Mar 10, 2021): WELP. I am fixing this as MSFT:32071839. We will replicate that back out to fix my parenthesis error in the next day or so. Good catch and thank you.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#12943