Regression with display of some wide glyphs/DBCS unicode characters #6168

Closed
opened 2026-01-31 00:31:27 +00:00 by claunia · 5 comments
Owner

Originally created by @ramsay-vibes on GitHub (Jan 28, 2020).

Environment

Windows build number: [10.0.18363.592]
Windows Terminal version: [0.8.10261.0]

Steps to reproduce

From an Ubuntu shell cat the attached file (cat utf82.txt).

utf82.txt

Expected behavior

before

Actual behavior

after

Description

With WT 0.8.10091.0 and recent earlier versions, the expected behavior occurs. Since release of WT 0.8.10261.0, the actual behavior occurs. Although there are inconsistencies elsewhere, observe in particular that the Khmer, Lao and Thai languages do not display properly.

Originally created by @ramsay-vibes on GitHub (Jan 28, 2020). <!-- 🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨 I ACKNOWLEDGE THE FOLLOWING BEFORE PROCEEDING: 1. If I delete this entire template and go my own path, the core team may close my issue without further explanation or engagement. 2. If I list multiple bugs/concerns in this one issue, the core team may close my issue without further explanation or engagement. 3. If I write an issue that has many duplicates, the core team may close my issue without further explanation or engagement (and without necessarily spending time to find the exact duplicate ID number). 4. If I leave the title incomplete when filing the issue, the core team may close my issue without further explanation or engagement. 5. If I file something completely blank in the body, the core team may close my issue without further explanation or engagement. All good? Then proceed! --> <!-- This bug tracker is monitored by Windows Terminal development team and other technical folks. **Important: When reporting BSODs or security issues, DO NOT attach memory dumps, logs, or traces to Github issues**. Instead, send dumps/traces to secure@microsoft.com, referencing this GitHub issue. If this is an application crash, please also provide a Feedback Hub submission link so we can find your diagnostic data on the backend. Use the category "Apps > Windows Terminal (Preview)" and choose "Share My Feedback" after submission to get the link. Please use this form and describe your issue, concisely but precisely, with as much detail as possible. --> # Environment ```none Windows build number: [10.0.18363.592] Windows Terminal version: [0.8.10261.0] ``` # Steps to reproduce <!-- A description of how to trigger this bug. --> From an Ubuntu shell cat the attached file (`cat utf82.txt`). [utf82.txt](https://github.com/microsoft/terminal/files/4121097/utf82.txt) # Expected behavior <!-- A description of what you're expecting, possibly containing screenshots or reference material. --> ![before](https://user-images.githubusercontent.com/8571098/73243180-e02e7280-419e-11ea-9a29-3e2b925b2170.jpg) # Actual behavior <!-- What's actually happening? --> ![after](https://user-images.githubusercontent.com/8571098/73243217-f76d6000-419e-11ea-8bd4-cd5ae34f8e89.jpg) # Description With WT 0.8.10091.0 and recent earlier versions, the expected behavior occurs. Since release of WT 0.8.10261.0, the actual behavior occurs. Although there are inconsistencies elsewhere, observe in particular that the Khmer, Lao and Thai languages do not display properly.
Author
Owner

@DHowett-MSFT commented on GitHub (Jan 28, 2020):

@milizhang this looks like it could be a regression from #4081. I'm particularly concerned that during rendering we're now ignoring the calculated backing buffer size... this may even cause a recrudescence of #633...

@DHowett-MSFT commented on GitHub (Jan 28, 2020): @milizhang this looks like it could be a regression from #4081. I'm particularly concerned that during rendering we're now ignoring the calculated backing buffer size... this may even cause a recrudescence of #633...
Author
Owner

@milizhang commented on GitHub (Feb 2, 2020):

Yes, this is indeed an issue caused by #4081 . That change was designed to only handle one-to-n mapping between glyph and character, but in reality for many scripts it can be a n-to-m mapping.

As we dicussed before, with the current column buffer system it is simply not possible to get an ideal solution for this issue.

The solution I am leaning toward is to respect both glyph cluster and column width. This means we place the glyph cluster at the center of expected space, so that the glyph clusters can be kept together likely in a recognizable way (though things can still get bad in cases that fontScale is applied). The side effect, however, is that it could left whitespace around the cluster.

@encladeus @DHowett-MSFT I have a prototype for this solution. Please let me know if you think this is reasonable.

Without fontScale:
image

With fontScale:
image

Screenshot that matches original post:
image

@milizhang commented on GitHub (Feb 2, 2020): Yes, this is indeed an issue caused by #4081 . That change was designed to only handle one-to-n mapping between glyph and character, but in reality for many scripts it can be a n-to-m mapping. As we dicussed before, with the current column buffer system it is simply not possible to get an ideal solution for this issue. The solution I am leaning toward is to respect both glyph cluster and column width. This means we place the glyph cluster at the center of expected space, so that the glyph clusters can be kept together likely in a recognizable way (though things can still get bad in cases that fontScale is applied). The side effect, however, is that it could left whitespace around the cluster. @encladeus @DHowett-MSFT I have a prototype for this solution. Please let me know if you think this is reasonable. Without fontScale: ![image](https://user-images.githubusercontent.com/1834979/73604472-4112cd80-4546-11ea-8d96-13b23a6630e5.png) With fontScale: ![image](https://user-images.githubusercontent.com/1834979/73604482-5ee03280-4546-11ea-85ff-087036531edd.png) Screenshot that matches original post: ![image](https://user-images.githubusercontent.com/1834979/73604494-97800c00-4546-11ea-9dc2-02865eafc8aa.png)
Author
Owner

@DHowett-MSFT commented on GitHub (Feb 2, 2020):

I love it. I know we used to try to crush glyphs into either one or two columns when we shipped the May renderer changes. Perhaps this will make it work more generally?

@DHowett-MSFT commented on GitHub (Feb 2, 2020): I love it. I know we used to try to crush glyphs into either one or two columns when we shipped the May renderer changes. Perhaps this will make it work more generally?
Author
Owner

@DHowett-MSFT commented on GitHub (Feb 2, 2020):

As an aside: what font is that? It looks pretty true to the standard VGA font.

@DHowett-MSFT commented on GitHub (Feb 2, 2020): As an aside: what font is that? It looks pretty true to the standard VGA font.
Author
Owner

@milizhang commented on GitHub (Feb 2, 2020):

If I understand what you are talking correctly, "crushing glyphs into columns" is exactly what code around Run::fontScale is trying to do. Though I think there's an arguably better way to implement the scaling, as far as we want the display to match the column grid some scaling is likely still needed.
By implementing support of glyph cluster, this change might reduce the usage of fontScale, but sadly it does not eliminate it (as you can see in the screenshots). I think only purposefully designed fonts plus correct column width assignment can land us there.

BTW The font is Fixedsys Excelsior, which is basically Fixedsys + programming ligature. You can find it at kika/fixedsys.

@milizhang commented on GitHub (Feb 2, 2020): If I understand what you are talking correctly, "crushing glyphs into columns" is exactly what code around Run::fontScale is trying to do. Though I think there's an arguably better way to implement the scaling, as far as we want the display to match the column grid some scaling is likely still needed. By implementing support of glyph cluster, this change might reduce the usage of fontScale, but sadly it does not eliminate it (as you can see in the screenshots). I think only purposefully designed fonts plus correct column width assignment can land us there. BTW The font is Fixedsys Excelsior, which is basically [Fixedsys](https://en.wikipedia.org/wiki/Fixedsys) + programming ligature. You can find it at [kika/fixedsys](https://github.com/kika/fixedsys).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#6168