Custom default colors are problematic for legacy apps #8202

Open
opened 2026-01-31 01:23:18 +00:00 by claunia · 0 comments
Owner

Originally created by @j4james on GitHub (May 17, 2020).

Environment

Windows build number: Version 10.0.18362.719
Windows Terminal version (if applicable): commit b46d393061

Steps to reproduce

  1. Build a recent version of the OpenConsole solution.
  2. From the tools directory run opencon, to get a cmd shell.
  3. Open the Properties dialog and set the Screen Background to color index 3 (cyan).
  4. And on the Terminal tab, check Use Separate Background, and set some shade of red, e.g. 100/0/0.
  5. Execute cls.

Expected behavior

I'd expect the screen to be cleared with the same colors used to render the prompt. This is what it looks like in my existing Windows cmd shell.

image

Actual behavior

The screen is cleared with a cyan background, while the prompt is rendered with a red background.

image

The problem here is the active background color has ColorType::IsDefault, but the console APIs only deal in legacy attributes, which can't represent a "default" color. So when cls looks up the active background color, the legacy API returns the Screen Background index, which is cyan. And that's the color that cls then uses to fill the screen. The prompt in the meantime is just using the active background color, which is a genuine default color that maps to red.

The reason why this used to work, is the FillConsoleOutputAttribute API had a hack (since removed in PR #3100) that magically converted a legacy color that matched the legacy version of the active color into that equivalent TextColor value. However, that only applied when VT mode was enabled, so it wouldn't work for most legacy apps, and also relied on the active color just happening to match the color you wanted to fill with, which is by no means guaranteed.

So one option is we could add that hack back (or something similar), but I don't think that's the right solution. Other than the limitations mentioned already, it's only solving part of the problem. I know from working on issue #2661 that this is likely to effect us in other ways once conpty is passing through the full attributes correctly (the fact that conpty maps black to default is actually shielding us from these issues).

I have an idea for how we might address this, but I wanted to think it through some more before writing up the details. In the meantime, though, I thought it best to get this issue filed so you're at least aware of the problem.

Originally created by @j4james on GitHub (May 17, 2020). # Environment Windows build number: Version 10.0.18362.719 Windows Terminal version (if applicable): commit b46d39306124363a4104afb74fa9b6657b1d17d0 # Steps to reproduce 1. Build a recent version of the _OpenConsole_ solution. 2. From the _tools_ directory run `opencon`, to get a cmd shell. 3. Open the _Properties_ dialog and set the _Screen Background_ to color index 3 (cyan). 4. And on the _Terminal_ tab, check _Use Separate Background_, and set some shade of red, e.g. 100/0/0. 5. Execute `cls`. # Expected behavior I'd expect the screen to be cleared with the same colors used to render the prompt. This is what it looks like in my existing Windows cmd shell. ![image](https://user-images.githubusercontent.com/4181424/82157346-d11c1f00-9878-11ea-96a6-ae8ad3571d98.png) # Actual behavior The screen is cleared with a cyan background, while the prompt is rendered with a red background. ![image](https://user-images.githubusercontent.com/4181424/82157349-d7120000-9878-11ea-8dcf-7f26de717bae.png) The problem here is the active background color has `ColorType::IsDefault`, but the console APIs only deal in legacy attributes, which can't represent a "default" color. So when `cls` looks up the active background color, the legacy API returns the _Screen Background_ index, which is cyan. And that's the color that `cls` then uses to fill the screen. The prompt in the meantime is just using the active background color, which is a genuine default color that maps to red. The reason why this used to work, is the `FillConsoleOutputAttribute` API had a hack (since removed in PR #3100) that magically converted a legacy color that matched the legacy version of the active color into that equivalent `TextColor` value. However, that only applied when VT mode was enabled, so it wouldn't work for most legacy apps, and also relied on the active color just happening to match the color you wanted to fill with, which is by no means guaranteed. So one option is we could add that hack back (or something similar), but I don't think that's the right solution. Other than the limitations mentioned already, it's only solving part of the problem. I know from working on issue #2661 that this is likely to effect us in other ways once conpty is passing through the full attributes correctly (the fact that conpty maps black to default is actually shielding us from these issues). I have an idea for how we might address this, but I wanted to think it through some more before writing up the details. In the meantime, though, I thought it best to get this issue filed so you're at least aware of the problem.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#8202