starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

[PR #1964] [MERGED] Fix display of the "low ASCII" glyphs in PC code pages #24739

New Issue
Open
opened 2026-01-31 09:05:05 +00:00 by claunia · 0 comments
claunia commented 2026-01-31 09:05:05 +00:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/microsoft/terminal/pull/1964
Author: @j4james
Created: 7/14/2019
Status: ✅ Merged
Merged: 9/20/2019
Merged by: @DHowett-MSFT

Base: master ← Head: fix-control-glyphs

📝 Commits (2)

  • 0a9f47e Fix the character type check in WriteCharsLegacy, which needs to be a bit test rather than an equality comparison, otherwise it won't match anything.
  • aaf9f79 Move the null check before the character type check, otherwise it won't be reached now that the type check is working correctly.

📊 Changes

1 file changed (+18 additions, -15 deletions)

View changed files

📝 src/host/_stream.cpp (+18 -15)

📄 Description

Summary of the Pull Request

In the legacy console, it used to be possible to write out characters from the C0 range of a PC code page (e.g. CP437), and get the actual glyphs defined for those code points (at least those that weren't processed as control codes). In the v2 console this stopped working so you'd get an FFFD replacement glyph (�) for those characters instead. This PR fixes the issue so the correct glyphs are displayed again.

PR Checklist

  • Closes mc display messed up (#166)
  • CLA signed. If not, go over here and sign the CLA
  • Tests added/passed
  • Requires documentation to be updated
  • I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #166

Detailed Description of the Pull Request / Additional comments

There was already code in place to achieve this in the WriteCharsLegacy method. It used the GetStringTypeW method to determine the character type of the value being output, and if it was a C1_CNTRL character it performed the appropriate mapping. The problem was that the test of the character type flag was done as a direct comparision, when it should have been a bit test, so the condition was never met.

With this condition fixed, the code also needed to be reordered slightly to handle the null character. That had a special-case mapping to space, which was previously performed after the control test, but since a null character now successfully matches C1_CNTRL, it no longer falls through to that special case. To address that, I've had to move the null check above the control test.

Validation Steps Performed

I've tested this manually, by trying to output all the characters in the affected range (ASCII values 0 to 31, and 127, excluding the actual control codes 8,9,10 and 13). In all cases they now match the output that the legacy console produced.

Note that this only applies to PC code pages that have glyphs defined for the C0 range, so it won't work with the UTF-8 code page, but that was to be expected - the legacy console behaved the same way.

Also, note that this only works when the ENABLE_PROCESSED_OUTPUT console mode is set. That seems wrong to me (I'd expect the glyphs to work in both cases), but that's the way the legacy console behaved as well, so if that's a bug it's a separate issue.

I haven't added any unit tests, because I expect the behaviour of some of these characters to change over time (as support is added for more control codes), which could then cause the tests to fail. But if that's not a concern, I could probably add something to the ScreenBufferTests (perhaps with a comment warning that the tests might be expected to fail in the future).

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/microsoft/terminal/pull/1964 **Author:** [@j4james](https://github.com/j4james) **Created:** 7/14/2019 **Status:** ✅ Merged **Merged:** 9/20/2019 **Merged by:** [@DHowett-MSFT](https://github.com/DHowett-MSFT) **Base:** `master` ← **Head:** `fix-control-glyphs` --- ### 📝 Commits (2) - [`0a9f47e`](https://github.com/microsoft/terminal/commit/0a9f47edee4bc1ca195ffd63e9e9ad322ee9f304) Fix the character type check in WriteCharsLegacy, which needs to be a bit test rather than an equality comparison, otherwise it won't match anything. - [`aaf9f79`](https://github.com/microsoft/terminal/commit/aaf9f7960269de24992f9b6f0fbc975937236ed7) Move the null check before the character type check, otherwise it won't be reached now that the type check is working correctly. ### 📊 Changes **1 file changed** (+18 additions, -15 deletions) <details> <summary>View changed files</summary> 📝 `src/host/_stream.cpp` (+18 -15) </details> ### 📄 Description <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request In the legacy console, it used to be possible to write out characters from the C0 range of a PC code page (e.g. CP437), and get the actual glyphs defined for those code points (at least those that weren't processed as control codes). In the v2 console this stopped working so you'd get an FFFD replacement glyph (�) for those characters instead. This PR fixes the issue so the correct glyphs are displayed again. <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [x] Closes #166 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #166 <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments There was already code in place to achieve this in the `WriteCharsLegacy` method. It used the `GetStringTypeW` method to determine the character type of the value being output, and if it was a `C1_CNTRL` character it performed the appropriate mapping. The problem was that the test of the character type flag was done as a direct comparision, when it should have been a bit test, so the condition was never met. With this condition fixed, the code also needed to be reordered slightly to handle the null character. That had a special-case mapping to space, which was previously performed after the control test, but since a null character now successfully matches `C1_CNTRL`, it no longer falls through to that special case. To address that, I've had to move the null check above the control test. <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed I've tested this manually, by trying to output all the characters in the affected range (ASCII values 0 to 31, and 127, excluding the actual control codes 8,9,10 and 13). In all cases they now match the output that the legacy console produced. Note that this only applies to PC code pages that have glyphs defined for the C0 range, so it won't work with the UTF-8 code page, but that was to be expected - the legacy console behaved the same way. Also, note that this only works when the `ENABLE_PROCESSED_OUTPUT` console mode is set. That seems wrong to me (I'd expect the glyphs to work in both cases), but that's the way the legacy console behaved as well, so if that's a bug it's a separate issue. I haven't added any unit tests, because I expect the behaviour of some of these characters to change over time (as support is added for more control codes), which could then cause the tests to fail. But if that's not a concern, I could probably add something to the ScreenBufferTests (perhaps with a comment warning that the tests might be expected to fail in the future). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-31 09:05:05 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#24739
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?