Incorrect display of characters written on top of the wide emojis #6123

@DHowett-MSFT commented on GitHub (Jan 24, 2020):

Emoji cannot be split in half. Can you point to an example of a terminal that handles emoji in the way your "expected behavior" image reports?
Thanks!

@DHowett-MSFT commented on GitHub (Jan 24, 2020): Emoji cannot be split in half. Can you point to an example of a terminal that handles emoji in the way your "expected behavior" image reports? Thanks!

claunia commented

@o-sdn-o commented on GitHub (Jan 24, 2020):

Thank you for your attension!

I do not know any terminal emulator that would correctly display this case, but I believe that Windows Terminal should do this correctly, unlike others, and others will need to focus on it.

That is why I believe that it is "expected behavior":
When a user prints a regular character with preliminary indication of the coordinates for displaying in a terminal window, he expects to see it in the cell with these coordinates, and he cannot know in advance that a wide character is located at these coordinates and take this into account before displaying a regular character.

Also, taking this case into account when rendering the contents of its buffer is in the interests of the terminal emulator itself.
Since the width of the terminal window, as well as the cursor coordinates, is a multiple of the width of an narrow character, and not the width of a wide character, then only one cell can remain on the right border for displaying a wide character, and the terminal, in case of line wrapping, requires or a way to depict the presence of half of a wide character in this single cell, and display the second half on the next line.
The terminal displays a Unicode Replacement Character "�", when wide emoji, like 😋, wrapped to the next line, unlike CJK (which are also wide char) that are entirely placed to the next line.

There is no need to operate on the emoji halves separately, the halves are obtained in the output process only in certain cases, such as line wrapping or overlapping text of another high-level object, such as a frame with the message text over the already displayed text.

In the screen buffer, wide characters are usually represented by two adjacent cells, and if one of these two cells is overwritten with a narrow character (as it should happen when the text cursor is positioned manually and its new position falls just over the wide character), then the rendering subsystem should determine a separate way of rendering such a combination of cells.
For instance,
A wide character is represented in the buffer as:
👨 = [subcell1] [subcell2]
note, both subcells should have a reference to the whole grapheme cluster "👨" or be it with the exception of some attribute - the first part or second part, or they are entirely equal, but the first cell has a width=2 and the second cell has a width=0, to distinguish between them.
Narrow character:
X = [normcell]

Two cases of the cell combinations in the buffer:
[subcell1] [normcell]
[normcell] [subcell2]

first, the rendering procedure should draw a whole wide character in its two occupied cells, and then draw a narrow character on top of the wide one, given its position - left or right.

@o-sdn-o commented on GitHub (Jan 24, 2020): Thank you for your attension! I do not know any terminal emulator that would correctly display this case, but I believe that Windows Terminal should do this correctly, unlike others, and others will need to focus on it. That is why I believe that it is "expected behavior": When a user prints a regular character with preliminary indication of the coordinates for displaying in a terminal window, he expects to see it in the cell with these coordinates, and he cannot know in advance that a wide character is located at these coordinates and take this into account before displaying a regular character. Also, taking this case into account when rendering the contents of its buffer is in the interests of the terminal emulator itself. Since the width of the terminal window, as well as the cursor coordinates, is a multiple of the width of an narrow character, and not the width of a wide character, then only one cell can remain on the right border for displaying a wide character, and the terminal, in case of line wrapping, requires or a way to depict the presence of half of a wide character in this single cell, and display the second half on the next line. The terminal displays a Unicode Replacement Character "�", when wide emoji, like 😋, wrapped to the next line, unlike CJK (which are also wide char) that are entirely placed to the next line. There is no need to operate on the emoji halves separately, the halves are obtained in the output process only in certain cases, such as line wrapping or overlapping text of another high-level object, such as a frame with the message text over the already displayed text. ![crop-emoji-animated2](https://user-images.githubusercontent.com/11535558/73079518-302ee000-3ee6-11ea-9220-d95b9939b487.gif) In the screen buffer, wide characters are usually represented by two adjacent cells, and if one of these two cells is overwritten with a narrow character (as it should happen when the text cursor is positioned manually and its new position falls just over the wide character), then the rendering subsystem should determine a separate way of rendering such a combination of cells. For instance, A wide character is represented in the buffer as: 👨 = [subcell1] [subcell2] note, both subcells should have a reference to the whole grapheme cluster "👨" or be it with the exception of some attribute - the first part or second part, or they are entirely equal, but the first cell has a width=2 and the second cell has a width=0, to distinguish between them. Narrow character: X = [normcell] Two cases of the cell combinations in the buffer: [subcell1] [normcell] [normcell] [subcell2] first, the rendering procedure should draw a whole wide character in its two occupied cells, and then draw a narrow character on top of the wide one, given its position - left or right.

claunia commented

@o-sdn-o commented on GitHub (Jan 24, 2020):

that's how it looks now

@o-sdn-o commented on GitHub (Jan 24, 2020): that's how it looks now ![wt-emoji-crop-animated](https://user-images.githubusercontent.com/11535558/73082115-a3d2ec00-3eea-11ea-8e2c-55f2c4522d4b.gif)

claunia commented

@o-sdn-o commented on GitHub (Jan 24, 2020):

In addition, it seems to me that the features of using DECDWL / DECDHL and friends (#1884, Line Renditions, Double-Width, Double-Height Line https://vt100.net/docs/vt510-rm/DECDWL.html) lie in the same plane as visualization of wide characters, and causes the same rendering difficulties, but there may be other distortions of the visual representation during the flow of text.
These cases may be implemented with the same approach, as well as the visualization of wide characters on a cell-based coordinate grid.

@o-sdn-o commented on GitHub (Jan 24, 2020): In addition, it seems to me that the features of using DECDWL / DECDHL and friends (#1884, Line Renditions, Double-Width, Double-Height Line https://vt100.net/docs/vt510-rm/DECDWL.html) lie in the same plane as visualization of wide characters, and causes the same rendering difficulties, but there may be other distortions of the visual representation during the flow of text. These cases may be implemented with the same approach, as well as the visualization of wide characters on a cell-based coordinate grid.

claunia commented

@DHowett-MSFT commented on GitHub (Jan 24, 2020):

I believe that the only correct way to handle a partial destruction of a double-width character is to remove the remaining half. There is no way for us to properly cut an emoji, or a CJK symbol that spans two cells, in half. Its meaning will be lost, and half of a character is not a unit that is representable in any encoding scheme or language.

RXVT-Unicode, which I believe set the standard for unicode use in terminals, treats your reproduction case as follows:

It doesn't support Emoji, but it does support a double-width glyph. Printing X over half of the double-width glyph destroys it.

The Windows Terminal currently has a bug where it does not destroy the double-width glyph. That, we should fix. I think it's tracked elsewhere, though.

@DHowett-MSFT commented on GitHub (Jan 24, 2020): I believe that the only correct way to handle a partial destruction of a double-width character is to remove the remaining half. There is no way for us to properly cut an emoji, or a CJK symbol that spans two cells, in half. Its meaning will be lost, and half of a character is not a unit that is representable in any encoding scheme or language. RXVT-Unicode, which I believe set the standard for unicode use in terminals, treats your reproduction case as follows: ![image](https://user-images.githubusercontent.com/14316954/73099951-6433e200-3ea1-11ea-8bf0-b022695560dc.png) It doesn't support Emoji, but it does support a double-width glyph. Printing X over half of the double-width glyph destroys it. The Windows Terminal currently has a bug where it does _not_ destroy the double-width glyph. _That_, we should fix. I think it's tracked elsewhere, though.

claunia commented

@DHowett-MSFT commented on GitHub (Jan 24, 2020):

The same applies to wrapping wide glyphs. They just cannot be broken in half.

@DHowett-MSFT commented on GitHub (Jan 24, 2020): The same applies to wrapping wide glyphs. They just cannot be broken in half.

claunia commented

@o-sdn-o commented on GitHub (Jan 24, 2020):

Do I understand correctly that is possible to divide a wide character only by doubling the amount of memory allocated for one cell?

In that case, then the memory consumption can be neglected for the consistent visualization of all types of characters, since memory is not such a big problem these days, for example, tens of gigabytes of memory are needed only to compile the Windows Terminal.

@o-sdn-o commented on GitHub (Jan 24, 2020): Do I understand correctly that is possible to divide a wide character only by doubling the amount of memory allocated for one cell? In that case, then the memory consumption can be neglected for the consistent visualization of all types of characters, since memory is not such a big problem these days, for example, tens of gigabytes of memory are needed only to compile the Windows Terminal.

claunia commented

@DHowett-MSFT commented on GitHub (Jan 24, 2020):

I'm not concerned about storage or memory for wide characters. I'm concerned about how half-glyphs are incomprehensible and do not mean anything, and there is no prior art for cutting an emoji or an ideograph in half. 😄

@DHowett-MSFT commented on GitHub (Jan 24, 2020): I'm not concerned about storage or memory for wide characters. I'm concerned about how half-glyphs are incomprehensible and do not mean anything, and there is no prior art for cutting an emoji or an ideograph in half. :smile:

claunia commented

@o-sdn-o commented on GitHub (Jan 24, 2020):

It would be great to become pioneers 🚀 in this matter, since this problem exists.
No one cares about half glyphs, but text-based user interfaces cannot be rendered correctly without them in any way, and Windows Terminal is not least intended to display text user interfaces.

@o-sdn-o commented on GitHub (Jan 24, 2020): It would be great to become pioneers 🚀 in this matter, since this problem exists. No one cares about half glyphs, but text-based user interfaces cannot be rendered correctly without them in any way, and Windows Terminal is not least intended to display text user interfaces.

claunia commented

@o-sdn-o commented on GitHub (Jan 24, 2020):

It is also possible that at some point when glyphs with a width of three cells (e.g.: 👨‍👩‍👦‍👦 Family: Man, Woman, Boy, Boy) are needed to support any specific scripts or complex grapheme clusters, the problem will become even more obvious.

@o-sdn-o commented on GitHub (Jan 24, 2020): It is also possible that at some point when glyphs with a width of three cells (e.g.: 👨‍👩‍👦‍👦 Family: Man, Woman, Boy, Boy) are needed to support any specific scripts or complex grapheme clusters, the problem will become even more obvious.

claunia commented

@o-sdn-o commented on GitHub (Jan 24, 2020):

I see no reason why the wide character should be completely destroyed when another symbol hits it.

I understand that it must be destroyed when copying the contents of cells with this state to the clipboard (until an Unicode standard will allow to form half glyphs by attaching the corresponding modifying codepoint), but such half cells in the terminal have full right to be displayed as is.

@o-sdn-o commented on GitHub (Jan 24, 2020): I see no reason why the wide character should be completely destroyed when another symbol hits it. I understand that it must be destroyed when copying the contents of cells with this state to the clipboard (until an Unicode standard will allow to form half glyphs by attaching the corresponding modifying codepoint), but such half cells in the terminal have full right to be displayed as is.

claunia commented

@DHowett-MSFT commented on GitHub (Jan 25, 2020):

I'd like to understand this scenario a bit more.

Consider an application drawing a UI over top of a two-cell character.

The two-cell character is split in half, and the half in the left cell remains on-screen.
How does the application know that this terminal supports half-characters?
When that UI is eventually dismissed, how does the application restore the missing half character? Remember: the terminal cannot save/restore the contents of the buffer for all applications reliably, so the application must keep track of it.
If the application must keep track of it, and it must support terminals that do not have half-characters, it should move to the correct cell and re-print the entire character. That's the only way to ensure rendering stability.
If an application must re-print the entire glyph, what is the value to the application? It must invalidate the same number of elements on the screen as it always has.
If we standardize a new way for an application to print "the right half of a 2-cell character", it still must do feature detection on this terminal.
How does this extend to characters that take up more than one cell? There is an almost infinite number of cell invalidation schemes that an application would need to be aware of.

Consider the case of an application wrapping a two-cell character at the edge of the screen.

At the right side of my screen, I see:

and on the left, I see

It does not seem trivial as a human to (mentally) reconstruct a symbol from an ideographic language split and moved to the other side of the screen. It's a readability disaster, and no other application anywhere (even complicated word processors!) implements wrapping that breaks a character in half. I'm sure there's a very good reason why.

I think we need to let UI libraries, ones that operate on pixel buffers instead of cellular text buffers, solve this problem for graphical UIs and not try to add single-cell occlusion to terminals to bring them closer to word processors.

@DHowett-MSFT commented on GitHub (Jan 25, 2020): I'd like to understand this scenario a bit more. Consider an application drawing a UI over top of a two-cell character. * The two-cell character is split in half, and the half in the left cell remains on-screen. * How does the application know that this terminal supports half-characters? * When that UI is eventually dismissed, how does the application restore the _missing half character_? Remember: the terminal cannot save/restore the contents of the buffer for all applications reliably, so the application _must_ keep track of it. * If the application must keep track of it, and it must support terminals that do not have half-characters, it **should** move to the correct cell and re-print the entire character. That's the only way to ensure rendering stability. * If an application must re-print the entire glyph, what is the value to the application? It must invalidate the same number of elements on the screen as it always has. * If we standardize a new way for an application to print "the right half of a 2-cell character", it still must do feature detection on this terminal. * How does this extend to characters that take up more than one cell? There is an almost infinite number of cell invalidation schemes that an application would need to be aware of. Consider the case of an application wrapping a two-cell character at the edge of the screen. At the right side of my screen, I see: ![image](https://user-images.githubusercontent.com/14316954/73113050-783e0a80-3ec6-11ea-9d8f-6d8bd68397fa.png) and on the left, I see ![image](https://user-images.githubusercontent.com/14316954/73113119-cc48ef00-3ec6-11ea-8506-e826b00c48c9.png) It does not seem trivial as a human to (mentally) reconstruct a symbol from an ideographic language split and moved to the other side of the screen. It's a readability disaster, and _no other application anywhere_ (even complicated word processors!) implements wrapping that breaks a character in half. I'm sure there's a very good reason why. I think we need to let UI libraries, ones that operate on pixel buffers instead of cellular text buffers, solve this problem for graphical UIs and not try to add single-cell occlusion to terminals to bring them closer to word processors.

claunia commented

@DHowett-MSFT commented on GitHub (Jan 25, 2020):

(Thank you for taking your time to explain this!)

@DHowett-MSFT commented on GitHub (Jan 25, 2020): (Thank you for taking your time to explain this!)

claunia commented

@o-sdn-o commented on GitHub (Jan 25, 2020):

we need to let UI libraries, ones that operate on pixel buffers instead of cellular text buffers, solve this problem for graphical UIs and not try to add single-cell occlusion to terminals to bring them closer to word processors.

I do not agree with this, Unicode allows you to visualize UI elements as exquisitely as never before before the Unicode era. now a huge number of graphic elements are available in the form of separate codepoints, which would be a big mistake not to use this and not to do sophisticated programs in text-based mode.

@o-sdn-o commented on GitHub (Jan 25, 2020): > we need to let UI libraries, ones that operate on pixel buffers instead of cellular text buffers, solve this problem for graphical UIs and not try to add single-cell occlusion to terminals to bring them closer to word processors. I do not agree with this, Unicode allows you to visualize UI elements as exquisitely as never before before the Unicode era. now a huge number of graphic elements are available in the form of separate codepoints, which would be a big mistake not to use this and not to do sophisticated programs in text-based mode.

claunia commented

@o-sdn-o commented on GitHub (Jan 25, 2020):

There is an almost infinite number of cell invalidation schemes that an application would need to be aware of.

An application offering a user interface naturally has its own buffer, which contains a picture of the world, and this application must duplicate all transactions in the terminal into this buffer, and moreover, changes to the buffer must be recorded taking into account that the terminal may not support half-characters and other functionality, e.g.: even if the terminal does not correctly handle character widths, the application may always emit an ansi-sequence to correct cursor coordinates after the printing of each potentially wide characters, despite the awarnes of terminal capabilities.

@o-sdn-o commented on GitHub (Jan 25, 2020): > There is an almost infinite number of cell invalidation schemes that an application would need to be aware of. An application offering a user interface naturally has its own buffer, which contains a picture of the world, and this application must duplicate all transactions in the terminal into this buffer, and moreover, changes to the buffer must be recorded taking into account that the terminal may not support half-characters and other functionality, e.g.: even if the terminal does not correctly handle character widths, the application may always emit an ansi-sequence to correct cursor coordinates after the printing of each potentially wide characters, despite the awarnes of terminal capabilities.

claunia commented

@o-sdn-o commented on GitHub (Jan 25, 2020):

Thank you for your clarification and for your work on the Windows Terminal, and I hope that my thoughts were at least a little useful 🙏🏻

@o-sdn-o commented on GitHub (Jan 25, 2020): Thank you for your clarification and for your work on the Windows Terminal, and I hope that my thoughts were at least a little useful 🙏🏻

claunia commented

@egmontkob commented on GitHub (Jan 25, 2020):

I'm mostly with @DHowett-MSFT on this.

No terminal I'm aware of supports half of a double-width character (half-CJK or half-emoji). When one half is overwritten, I believe most terminals explicitly replace the other half with a space (whether retaining its previous background color, or taking the currently active one, I have no clue).

In addition to that, when the cursor is at the rightmost column (i.e. there's still room for a final single character in the row) and a double wide is printed, the new double-wide character is placed entirely in the next line, leaving an empty cell at the end of the previous. Ideally this empty cell doesn't get copy-pasted, and disappears at a rewrap-on-resize; and similarly, if the CJK would cross the line boundary after the rewrapped position then it's placed entirely in the next line. Just like with every decent piece of software. (On a side note, this is one of the reasons VTE forces a minimum size of 1 row, 2 columns.)

If there is a need for supporting half-CJKs and half-emojis (which I'm unsure about), the partial overwriting approach shown above is problematic for multiple reasons.

It assumes that the other (the one overwriting half of a CJK/emoji) is a single-width one, for which I don't see a guarantee. If there's a need to display half characters, the need for displaying two halves next to each other can arise very easily. There would also be a need to display a right half in the leftmost column, or a left half in the rightmost column, which would be impossible (and is a natural requirement for a text editor with non-folding lines and horizontal scrolling).

It doesn't allow flicker-free updates: in order to replace the half-emoji with another one, the other character might shortly be replaced by the undesired other half of the emoji.

It makes it necessary to come up with a logic that is very unlikely to be compatible with the current thinking and implementation of screen drawing libraries (e.g. ncurses, slang...). Not only would they also need to support storing half-emojis in their internal buffers, they'd also need to construct non-trivial ways of transmitting them to the terminal. E.g. if the right half of an emoji is visible, they'd need to reprint the entire emoji first, then move the cursor back, and then print whichever character hides its left half (which again might be the right half of an emoji, and then continue backtracking...).

This all falls apart uncontrollably.

If there's a need to display half-emojis, a new escape sequence should be invented which allows to place a half-emoji anywhere, without affecting any other cell. It could perhaps be a new mode to SAPV, or some brand new sequence. Once we have it (with agreement of a couple of leader terminals), it's perhaps okay to change the overwriting mechanism too as part of this story, to keep the other half present upon a partial overwrite.

There'd be a couple of other tricky questions. E.g. what to do on a copy-paste (do you just simply copy it? and how do you make sure not to copy it twice if the entire emoji is visible, maybe emitted by a screen drawing library as separate left + right halves?). Or what to do on rewrap-on-resize, how to decide whether an emoji (or a matching pair of two half-emojis) are allowed to be separated to different lines or not? Would the terminal need to store somehow whether the two halves are joined or not; what escape sequences and what other actions would modify this state, and last but not least, do we see a reasonable chance that various major terminals would come to a conclusion here? I'm afraid not.

It's an interesting and really hard technical challenge if and how this could be solved, in a way that's not specific to just a terminal or two, but is likely followed by the entire ecosystem (most terminals, screen drawing libraries, utilities). A proper solution is sure much-much harder than just coding in WT for a couple of hours to keep the other half there :)

@egmontkob commented on GitHub (Jan 25, 2020): I'm mostly with @DHowett-MSFT on this. No terminal I'm aware of supports half of a double-width character (half-CJK or half-emoji). When one half is overwritten, I believe most terminals explicitly replace the other half with a space (whether retaining its previous background color, or taking the currently active one, I have no clue). In addition to that, when the cursor is at the rightmost column (i.e. there's still room for a final single character in the row) and a double wide is printed, the new double-wide character is placed entirely in the next line, leaving an empty cell at the end of the previous. Ideally this empty cell doesn't get copy-pasted, and disappears at a rewrap-on-resize; and similarly, if the CJK would cross the line boundary after the rewrapped position then it's placed entirely in the next line. Just like with every decent piece of software. (On a side note, this is one of the reasons VTE forces a minimum size of 1 row, 2 columns.) _If_ there is a need for supporting half-CJKs and half-emojis (which I'm unsure about), the partial overwriting approach shown above is problematic for multiple reasons. It assumes that the other (the one overwriting half of a CJK/emoji) is a single-width one, for which I don't see a guarantee. If there's a need to display half characters, the need for displaying two halves next to each other can arise very easily. There would also be a need to display a right half in the leftmost column, or a left half in the rightmost column, which would be impossible (and is a natural requirement for a text editor with non-folding lines and horizontal scrolling). It doesn't allow flicker-free updates: in order to replace the half-emoji with another one, the other character might shortly be replaced by the undesired other half of the emoji. It makes it necessary to come up with a logic that is very unlikely to be compatible with the current thinking and implementation of screen drawing libraries (e.g. ncurses, slang...). Not only would they also need to support storing half-emojis in their internal buffers, they'd also need to construct non-trivial ways of transmitting them to the terminal. E.g. if the right half of an emoji is visible, they'd need to reprint the entire emoji first, then move the cursor back, and then print whichever character hides its left half (which again might be the right half of an emoji, and then continue backtracking...). This all falls apart uncontrollably. _If_ there's a need to display half-emojis, a new escape sequence should be invented which allows to place a half-emoji anywhere, without affecting any other cell. It could perhaps be a new mode to SAPV, or some brand new sequence. Once we have it (with agreement of a couple of leader terminals), it's _perhaps_ okay to change the overwriting mechanism too as part of this story, to keep the other half present upon a partial overwrite. There'd be a couple of other tricky questions. E.g. what to do on a copy-paste (do you just simply copy it? and how do you make sure not to copy it twice if the entire emoji is visible, maybe emitted by a screen drawing library as separate left + right halves?). Or what to do on rewrap-on-resize, how to decide whether an emoji (or a matching pair of two half-emojis) are allowed to be separated to different lines or not? Would the terminal need to store somehow whether the two halves are joined or not; what escape sequences and what other actions would modify this state, and last but not least, do we see a reasonable chance that various major terminals would come to a conclusion here? I'm afraid not. It's an interesting and really hard technical challenge if and how this could be solved, in a way that's not specific to just a terminal or two, but is likely followed by the entire ecosystem (most terminals, screen drawing libraries, utilities). A proper solution is sure much-much harder than just coding in WT for a couple of hours to keep the other half there :)

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

use private codepoint (from Unicode Private Use Areas):
GCLPART: grapheme cluster left part
GCRPART: grapheme cluster right part
as a modifier character
(VT sequences are not suitable for copy-paste)

@o-sdn-o commented on GitHub (Jan 26, 2020): use private codepoint (from Unicode Private Use Areas): `GCLPART: grapheme cluster left part` `GCRPART: grapheme cluster right part` as a modifier character (VT sequences are not suitable for copy-paste)

claunia commented

@egmontkob commented on GitHub (Jan 26, 2020):

How would it work? Prefix for a single following character?

What kinds of problems would it use that it's not an escape sequence, but a regular character?

Or wait, you said "modifier character", like VS16 and friends, so would it come after the base character? Then what if there's a pause in the input stream after the base character? The terminal displays the entire glyph, potentially overwriting something, or overflowing to the next line, in turn potentially scrolling the entire contents, and then a bit later oops it should change its mind, undo these and place only half of it? Clearly cannot work, the special treatment has to be known in advance.

Also, private use means private use, up for you to use it in-house for whatever you want; not for Microsoft or any similar party to start assigning it a value (let alone a control instuction, rather than a glyph).

There's a whole lot more to the story, like how would it be compatible with wcwidth() and similar methods of determining a glyph's width in the terminal, etc.

Trust me please, it's not an area where you, or me, or anyone could such easily come up with a good solution.

@egmontkob commented on GitHub (Jan 26, 2020): How would it work? Prefix for a single following character? What kinds of problems would it use that it's _not_ an escape sequence, but a regular character? Or wait, you said "modifier character", like VS16 and friends, so would it come _after_ the base character? Then what if there's a pause in the input stream after the base character? The terminal displays the entire glyph, potentially overwriting something, or overflowing to the next line, in turn potentially scrolling the entire contents, and then a bit later oops it should change its mind, undo these and place only half of it? Clearly cannot work, the special treatment has to be known in advance. Also, private use means private use, up for _you_ to use it in-house for whatever you want; not for Microsoft or any similar party to start assigning it a value (let alone a control instuction, rather than a glyph). There's a whole lot more to the story, like how would it be compatible with wcwidth() and similar methods of determining a glyph's width in the terminal, etc. Trust me please, it's not an area where you, or me, or anyone could such easily come up with a good solution.

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

How would it work?

like VS16. I suppose

what if there's a pause in the input stream after the base character?

If such a modifier appears first in the input stream the terminal should be triggered to text reflowing as in the case of window resize.

@o-sdn-o commented on GitHub (Jan 26, 2020): > How would it work? like VS16. I suppose > what if there's a pause in the input stream after the base character? If such a modifier appears first in the input stream the terminal should be triggered to text reflowing as in the case of window resize.

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

...not for Microsoft or any similar party to start assigning it a value (let alone a control instuction, rather than a glyph)

So they need to add these modifiers to the Unicode Standard.

@o-sdn-o commented on GitHub (Jan 26, 2020): > ...not for Microsoft or any similar party to start assigning it a value (let alone a control instuction, rather than a glyph) So they need to add these modifiers to the Unicode Standard.

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

to support double-height (DECDHL, 2x2 cells) characters use

GCNWPART: grapheme cluster north-west part
GCNEPART: grapheme cluster north-east part 
GCSWPART: grapheme cluster south-west part 
GCSEPART: grapheme cluster south-east part

which are preserve cwidth value of the base character.

@o-sdn-o commented on GitHub (Jan 26, 2020): to support double-height (DECDHL, 2x2 cells) characters use ``` GCNWPART: grapheme cluster north-west part GCNEPART: grapheme cluster north-east part GCSWPART: grapheme cluster south-west part GCSEPART: grapheme cluster south-east part ``` which are preserve cwidth value of the base character.

claunia commented

@egmontkob commented on GitHub (Jan 26, 2020):

So they need to add these modifiers to the Unicode Standard.

That's sure one possible way, with a couple of problems:

It wouldn't solve the problem that Unicode appends modifiers (becase they assume the text is available in its entirety, or it's okay to reprocess it if it changes). This model isn't suitable for the terminal, a terminal potentially needs to apply it retroactively without the possibility of replaying the previous events. If Unicode adds it as a suffix character, it hardcodes that terminals won't be able to support it.
Unicode hardly cares about the width in terminal, only has some weak hint that is sometimes ambiguous, sometimes changes over Unicode versions. Unicode's width model is already incompatible with what terminals and terminal-based applications expect, namely that the width of an entire string is always the sum of the width of the individual codepoints, VS16 breaks this and terminals have not yet figured out how to handle this. Anyway, if Unicode added such "left half", "right half" modifiers, they'd sure apply to all the glyphs, not just the double wide ones. And then one asks for half of a narrow glyph in a terminal, what to do?
It doesn't answer the copy-paste problem; and in the mean time introduces another surface for homoglyph attacks, e.g. the URL you see as "microsoft.com" might actually be "[left half of m][right half of m]icrosoft.com"

This approach would create tons new of problems and wouldn't solve anything in the Unicode world, and would solve certain things and create new problems in the terminal world.

There's no problem with Unicode, there's a problem with terminals right now. Trying to solve it in Unicode is in my firm opinion necessarily the wrong approach. The problem needs to be solved wherever it is: in terminals. That is, the solution is not new codepoints, the solution is new escape sequences.

A new escape sequence could switch to a mode where the following single character, or the following characters until the mode is switched off, are forced to a single cell, displaying the given half in case of double characters (but this might need to be further generalized, see below).

to support double-height (DECDHL, 2x2 cells) characters use

DECDWL/DECDHL themselves apply to entire lines, and as such they are utterly broken by design and useless, see e.g. VTE 195.

If there's a need for double-height or double-width (as in stretched) characters, it needs to go per-character, according to your proposal (maybe using brand new GCNWPART etc. escape sequences).

As you can see in Tilix 1782, some Unicode modifiers increase the width of a character in a way that it can be 4 or even more cells wide. Yet another sign of Unicode not caring about terminals, and terminals having no idea how to follow the specs they come up with. Another similar case is if a fixed variant (i.e. per-character) of DECDWL is invented: a horizontally stretched character which is double-wide to begin with becomes quadruple-wide. And if we allow these (i.e. units that occupy more than 2 cells horizontally), you'll need a solution for displaying any of its cells and only those, e.g. third or fourth or three fourths of such a grapheme...

Backwards compatibility is another important issue. It's unclear to me how to design an escape sequence that doesn't break in non-supporting terminals. A supporting and a non-supporting terminal (assuming that non-supporting ones silently ignore the escape sequence) need to advance the cursor by the same amount, even in special cases where the cursor is near one of the margins, otherwise subsequently printed text won't be displayed at the same location in them. If an app cannot reliably print either a half-emoji in supporting terminals, or a space (or preferably: an app-specified single-wide fallback character) in non-supporting terminals, it even significantly decreases the chance of apps caring enough to bother.

Realistically, I'm afraid all this is just too much work to fix (including not just multiple terminals but also ncurses and friends, and apps building on top of them), for quite small benefits, that it's unlikely to happen. Probably the terminal is just going to remain a platform that can display entire glyphs, and if only half of a glyph would fit than that glyph is entirely gone. If you need partial glyphs, graphical applications can do that for you (and a whole bunch of other things that terminals can't and won't ever).

@egmontkob commented on GitHub (Jan 26, 2020): > So they need to add these modifiers to the Unicode Standard. That's sure one possible way, with a couple of problems: - It wouldn't solve the problem that Unicode _appends_ modifiers (becase they assume the text is available in its entirety, or it's okay to reprocess it if it changes). This model isn't suitable for the terminal, a terminal potentially needs to apply it retroactively without the possibility of replaying the previous events. If Unicode adds it as a _suffix_ character, it hardcodes that terminals won't be able to support it. - Unicode hardly cares about the width in terminal, only has some weak hint that is sometimes ambiguous, sometimes changes over Unicode versions. Unicode's width model is already incompatible with what terminals and terminal-based applications expect, namely that the width of an entire string is always the sum of the width of the individual codepoints, VS16 breaks this and terminals have not yet figured out how to handle this. Anyway, if Unicode added such "left half", "right half" modifiers, they'd sure apply to all the glyphs, not just the double wide ones. And then one asks for half of a narrow glyph in a terminal, what to do? - It doesn't answer the copy-paste problem; and in the mean time introduces another surface for homoglyph attacks, e.g. the URL you see as "microsoft.com" might actually be "[left half of m][right half of m]icrosoft.com" This approach would create tons new of problems and wouldn't solve anything in the Unicode world, and would solve certain things and create new problems in the terminal world. There's no problem with Unicode, there's a problem with terminals right now. Trying to solve it in Unicode is in my firm opinion necessarily the wrong approach. The problem needs to be solved wherever it is: in terminals. That is, the solution is not new codepoints, the solution is new escape sequences. A new escape sequence could switch to a mode where the following single character, or the following characters until the mode is switched off, are forced to a single cell, displaying the given half in case of double characters (but this might need to be further generalized, see below). > to support double-height (DECDHL, 2x2 cells) characters use DECDWL/DECDHL themselves apply to entire lines, and as such they are utterly broken by design and useless, see e.g. [VTE 195](https://gitlab.gnome.org/GNOME/vte/issues/195). If there's a need for double-height or double-width (as in stretched) characters, it needs to go per-character, according to your proposal (maybe using brand new GCNWPART etc. escape sequences). As you can see in [Tilix 1782](https://github.com/gnunn1/tilix/issues/1782), some Unicode modifiers increase the width of a character in a way that it can be 4 or even more cells wide. Yet another sign of Unicode not caring about terminals, and terminals having no idea how to follow the specs they come up with. Another similar case is if a fixed variant (i.e. per-character) of DECDWL is invented: a horizontally stretched character which is double-wide to begin with becomes quadruple-wide. And if we allow these (i.e. units that occupy more than 2 cells horizontally), you'll need a solution for displaying any of its cells and only those, e.g. third or fourth or three fourths of such a grapheme... Backwards compatibility is another important issue. It's unclear to me how to design an escape sequence that doesn't break in non-supporting terminals. A supporting and a non-supporting terminal (assuming that non-supporting ones silently ignore the escape sequence) need to advance the cursor by the same amount, even in special cases where the cursor is near one of the margins, otherwise subsequently printed text won't be displayed at the same location in them. If an app cannot reliably print either a half-emoji in supporting terminals, or a space (or preferably: an app-specified single-wide fallback character) in non-supporting terminals, it even significantly decreases the chance of apps caring enough to bother. Realistically, I'm afraid all this is just too much work to fix (including not just multiple terminals but also ncurses and friends, and apps building on top of them), for quite small benefits, that it's unlikely to happen. Probably the terminal is just going to remain a platform that can display entire glyphs, and if only half of a glyph would fit than that glyph is entirely gone. If you need partial glyphs, graphical applications can do that for you (and a whole bunch of other things that terminals can't and won't ever).

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

Thank you for your thoughts, everything is very clear, but let's dream a little, it would certainly be very cool to use such things in the terminals 😊

$LF = [char]0x0a
$ESC = [char]0x1b
$CSI = "$($ESC)["
$RGB = "$($CSI)38;2;"
$DECDHLT = "$($ESC)#3"
$DECDHLB = "$($ESC)#4"
$DECSWL  = "$($ESC)#5"
$RESET = "$($CSI)0m"           # reset rendition
$WTCLR = "$($RGB)255;255;255m" # white  fg color
$RDCLR = "$($RGB)243;83;37m"   # red    fg color
$GRCLR = "$($RGB)129;188;6m"   # green  fg color
$BLCLR = "$($RGB)5;166;240m"   # blue   fg color
$YLCLR = "$($RGB)255;186;8m"   # yellow fg color
#    ____▄ 
# Windows ▀
#  Terminal
$logo = "$RESET$WTCLR    _____$RDCLR▄$GRCLR▄
 $WTCLR $($DECDHLB)W$($DECSWL)indows$BLCLR▀$YLCLR▀
 $WTCLR  $($DECDHLB)T$($DECSWL)erminal$RESET$LF"

"Default mode on"
$logo
"Legacy mode on"
$logo

Of course, I imagined it much cooler until I drew 🤔

@o-sdn-o commented on GitHub (Jan 26, 2020): Thank you for your thoughts, everything is very clear, but let's dream a little, it would certainly be very cool to use such things in the terminals 😊 ``` $LF = [char]0x0a $ESC = [char]0x1b $CSI = "$($ESC)[" $RGB = "$($CSI)38;2;" $DECDHLT = "$($ESC)#3" $DECDHLB = "$($ESC)#4" $DECSWL = "$($ESC)#5" $RESET = "$($CSI)0m" # reset rendition $WTCLR = "$($RGB)255;255;255m" # white fg color $RDCLR = "$($RGB)243;83;37m" # red fg color $GRCLR = "$($RGB)129;188;6m" # green fg color $BLCLR = "$($RGB)5;166;240m" # blue fg color $YLCLR = "$($RGB)255;186;8m" # yellow fg color # ____▄ # Windows ▀ # Terminal $logo = "$RESET$WTCLR _____$RDCLR▄$GRCLR▄ $WTCLR $($DECDHLB)W$($DECSWL)indows$BLCLR▀$YLCLR▀ $WTCLR $($DECDHLB)T$($DECSWL)erminal$RESET$LF" "Default mode on" $logo "Legacy mode on" $logo ``` ![image](https://user-images.githubusercontent.com/11535558/73137444-ed057600-4079-11ea-8384-38abec01c234.png) Of course, I imagined it much cooler until I drew 🤔

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

It doesn't answer the copy-paste problem

Why?
When the selected area is copyed from the terminal, each halfed-cell that does not have an adjacent neighbor is appended with a half-modifier, otherwise it is replaced by a single whole character.

...introduces another surface for homoglyph attacks, e.g. the URL you see as "microsoft.com" might actually be "[left half of m][right half of m]icrosoft.com"

Such adjacent user-perceived characters should be replaced by the application as a whole as soon as possible (immediately after input from outside and before output), in the case when the application shows the user single glyph while internally represents it as a combination of two halves should be treated as application security bug. If an application does not know about such a Unicode standard, then it can't show the halves as a single whole.

The problem needs to be solved wherever it is: in terminals.

The copy-paste activity is an area that is outside of the terminals.
Escape sequences are useless outside the terminal, so they break the copy-paste mechanics, unlike codepoint modifiers.

@o-sdn-o commented on GitHub (Jan 26, 2020): > It doesn't answer the copy-paste problem Why? When the selected area is copyed from the terminal, each halfed-cell that does not have an adjacent neighbor is appended with a half-modifier, otherwise it is replaced by a single whole character. > ...introduces another surface for homoglyph attacks, e.g. the URL you see as "microsoft.com" might actually be "[left half of m][right half of m]icrosoft.com" Such adjacent user-perceived characters should be replaced by the application as a whole as soon as possible (immediately after input from outside and before output), in the case when the application shows the user single glyph while internally represents it as a combination of two halves should be treated as application security bug. If an application does not know about such a Unicode standard, then it can't show the halves as a single whole. > The problem needs to be solved wherever it is: in terminals. The copy-paste activity is an area that is outside of the terminals. Escape sequences are useless outside the terminal, so they break the copy-paste mechanics, unlike codepoint modifiers.

claunia commented

@egmontkob commented on GitHub (Jan 26, 2020):

Such adjacent user-perceived characters should be replaced by the application

By "the application" you pretty much every application in the world that supports Unicode (not restricted to terminal based ones), which would then have to be updated to your behavior. Absolutely hopeless.

The copy-paste activity is an area that is outside of the terminals.

Sorry if I wasn't clear, this sentence wasn't supposed to focus on copy-pasting, which is just a tiny part of the entire story. I meant this sentence for our main problem: half-cut CJKs and emojis. They are not a problem anywhere else, only in terminals. Trying to address this problem anywhere else (e.g. in Unicode) would create a giant headache for magnitudes more people than those who are affected by the problem in terminals. It's a wrong approach. Half-cut CJKs are a problem of the terminal world; if it is worth addressing at all (which I don't think is) then it should be addressed here, in the terminal world, not in some "upper entity". No one outside terminals wants or needs these.

@egmontkob commented on GitHub (Jan 26, 2020): > Such adjacent user-perceived characters should be replaced by the application By "the application" you pretty much every application in the world that supports Unicode (not restricted to terminal based ones), which would then have to be updated to your behavior. Absolutely hopeless. > > The copy-paste activity is an area that is outside of the terminals. Sorry if I wasn't clear, this sentence wasn't supposed to focus on copy-pasting, which is just a tiny part of the entire story. I meant this sentence for our main problem: half-cut CJKs and emojis. They are not a problem anywhere else, only in terminals. Trying to address this problem anywhere else (e.g. in Unicode) would create a giant headache for magnitudes more people than those who are affected by the problem in terminals. It's a wrong approach. Half-cut CJKs are a problem of the terminal world; if it is worth addressing at all (which I don't think is) then it should be addressed here, in the terminal world, not in some "upper entity". No one outside terminals wants or needs these.

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

...would create a giant headache for magnitudes more people than those who are affected by the problem in terminals.

There are many modifiers in Unicode today that can corrupt glyphs, but no one forces them to use and corrupt CJK. The application that will claim Unicode Standard compliance should ensure that the incorrect use of this technique does not go beyond the application.

...which would then have to be updated to your behavior.

Absolutely not, because at the moment there are successfully running applications that do not know anything about Unicode, or if they aware, but don’t know all aspects of its appliance.

Half-cut CJKs are a problem of the terminal world; if it is worth addressing at all.

This can be done with the optional behavior of whether or not to copy-paste halves when copying a selected area of text or not. Let users decide what types of characters they want deal with outside the terminal.

@o-sdn-o commented on GitHub (Jan 26, 2020): > ...would create a giant headache for magnitudes more people than those who are affected by the problem in terminals. There are many modifiers in Unicode today that can corrupt glyphs, but no one forces them to use and corrupt CJK. The application that will claim Unicode Standard compliance should ensure that the incorrect use of this technique does not go beyond the application. > ...which would then have to be updated to your behavior. Absolutely not, because at the moment there are successfully running applications that do not know anything about Unicode, or if they aware, but don’t know all aspects of its appliance. > Half-cut CJKs are a problem of the terminal world; if it is worth addressing at all. This can be done with the optional behavior of whether or not to copy-paste halves when copying a selected area of text or not. Let users decide what types of characters they want deal with outside the terminal.

claunia commented

@egmontkob commented on GitHub (Jan 26, 2020):

So you imagine that thousands (millions?) of apps out there, which let's assume support a future Unicode version 15, will then suddenly no longer be fully compliant when version 16 comes out, or not fully compliant with whichever UAX or whatnot; and will have to be updated to at least prevent those homoglyph attacks, have reasonable copy-pasting, etc. All this for the sake of fixing a rare problem in terminals. Thousands of developers will thank you for that. (No.)

But let's also mention again that currently in Unicode all the modifiers are suffix characters, whereas in order to fix the problem in terminals we'd definitely need a prefix one, a suffix modifier that decreases the width literally cannot be reliably handled when the input slowly arrives over a stream. What do you think the chances are of Unicode accepting such a proposal? I believe it's pretty much zero.

On one hand I find the Unicode approach technicaly a bad choice, on the other hand, even if decided to go for it, I don't see a reasonable chance for getting buy-in from the Unicode folks.

Anyway, I believe we have to agree to disagree at this point. We've shared out thoughts with each other, and continuing this discussion doesn't seem to take us any further.

[Disclaimer: I'm not a WT developer and I'm not affiliated with Microsoft, not commenting on their behalf.]

@egmontkob commented on GitHub (Jan 26, 2020): So you imagine that thousands (millions?) of apps out there, which let's assume support a future Unicode version 15, will then suddenly no longer be fully compliant when version 16 comes out, or not fully compliant with whichever UAX or whatnot; and will have to be updated to at least prevent those homoglyph attacks, have reasonable copy-pasting, etc. All this for the sake of fixing a rare problem in terminals. Thousands of developers will thank you for that. (No.) But let's also mention again that currently in Unicode all the modifiers are suffix characters, whereas in order to fix the problem in terminals we'd definitely need a prefix one, a suffix modifier that decreases the width literally cannot be reliably handled when the input slowly arrives over a stream. What do you think the chances are of Unicode accepting such a proposal? I believe it's pretty much zero. On one hand I find the Unicode approach technicaly a bad choice, on the other hand, even if decided to go for it, I don't see a reasonable chance for getting buy-in from the Unicode folks. Anyway, I believe we have to agree to disagree at this point. We've shared out thoughts with each other, and continuing this discussion doesn't seem to take us any further. [Disclaimer: I'm not a WT developer and I'm not affiliated with Microsoft, not commenting on their behalf.]

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

Anyway, I believe we have to agree to disagree at this point. We've shared out thoughts with each other, and continuing this discussion doesn't seem to take us any further.

Thanks so much for the in-depth discussion. This is a lot of valuable information.

@o-sdn-o commented on GitHub (Jan 26, 2020): > Anyway, I believe we have to agree to disagree at this point. We've shared out thoughts with each other, and continuing this discussion doesn't seem to take us any further. Thanks so much for the in-depth discussion. This is a lot of valuable information.

claunia commented

@o-sdn-o commented on GitHub (Jan 26, 2020):

Another idea how to solve the problem of the allowing the screen drawing libraries (e.g. ncurses, slang ...) to display half characters to users.

Allow the terminal leaves the surviving half visible after splitting it, but only on the screen and only until the next repainting of the terminal window is occurred (text reflowing on some event of something else), and be sure to inform the application that all halves are “flushed”, and the application is responsible to reprint them (or whole screen) if they intended to do so.

Or, the application, at startup, tells to the terminal that the application itself is responsible for repainting entire terminal window when it is needed (by printing all screen lines, for example).

Some type of screen submode of the alternative screen buffer mode (CSI?1049h).

In this special mode, the terminal is not responsible to select text with the mouse, but must forward all mouse events to the application so that it takes care of the selection of text and placing data on the clipboard if the application provides for this.

Terminal should be responsible to emit VT-sequences to the application about the following data:

window (pane) resize events
dirty regions that need to be reprinted
all mouse events, also events outside the pane if mouse is captured (any of mouse button is pressed inside pane before mouse cursor leaves the pane region)
keyboard events
window (pane) focus events

In order to be able to display the right half of the wide character along the left edge of the screen, it should be permissible to place the text cursor one character to the left of the left edge of the screen, location [y; 0]:
~~- CUP (HVP) y; 0~~
~~- CHA 0~~
~~- CUB 1 from [y, 1] position~~

To optimize application performance, the terminal is required to perform the following requests from the application:

copy a certain region to the specified coordinates
clear (fill) the specified region by specified color or by a character

@o-sdn-o commented on GitHub (Jan 26, 2020): Another idea how to solve the problem of the allowing the screen drawing libraries (e.g. ncurses, slang ...) to display half characters to users. Allow the terminal leaves the surviving half visible after splitting it, but *only* on the screen and only until the next repainting of the terminal window is occurred (text reflowing on some event of something else), and be sure to inform the application that all halves are “flushed”, and the application is responsible to reprint them (or whole screen) if they intended to do so. Or, the application, at startup, tells to the terminal that the application itself is responsible for repainting entire terminal window when it is needed (by printing all screen lines, for example). Some type of screen submode of the alternative screen buffer mode (CSI?1049h). In this special mode, the terminal is not responsible to select text with the mouse, but must forward all mouse events to the application so that it takes care of the selection of text and placing data on the clipboard if the application provides for this. Terminal should be responsible to emit VT-sequences to the application about the following data: - window (pane) resize events - dirty regions that need to be reprinted - all mouse events, also events outside the pane if mouse is captured (any of mouse button is pressed inside pane before mouse cursor leaves the pane region) - keyboard events - window (pane) focus events ~~In order to be able to display the right half of the wide character along the left edge of the screen, it should be permissible to place the text cursor one character to the left of the left edge of the screen, location [y; 0]:~~ ~~- CUP (HVP) y; 0~~ ~~- CHA 0~~ ~~- CUB 1 from [y, 1] position~~ To optimize application performance, the terminal is required to perform the following requests from the application: - copy a certain region to the specified coordinates - clear (fill) the specified region by specified color or by a character

claunia commented

@o-sdn-o commented on GitHub (Jan 27, 2020):

Your arguments are very convincing, and I agree in the sense that the decision to use Unicode modifiers is like shooting a sparrow from a cannon, however, I can’t agree that it will do much harm to anyone.

One way or another, this solution is not feasible at a given time.

I consider the proposal to use a special terminal mode to provide a narrow circle of libraries using the terminal window for rendering the user interface more real and effortless, as well as the experience of using this terminal screen mode in the future will clearly show the need for standardization of half modifiers in Unicode, so I'm going to do make feature request this new terminal functionality.

@o-sdn-o commented on GitHub (Jan 27, 2020): Your arguments are very convincing, and I agree in the sense that the decision to use Unicode modifiers is like shooting a sparrow from a cannon, however, I can’t agree that it will do much harm to anyone. One way or another, this solution is not feasible at a given time. I consider the proposal to use a special terminal mode to provide a narrow circle of libraries using the terminal window for rendering the user interface more real and effortless, as well as the experience of using this terminal screen mode in the future will clearly show the need for standardization of half modifiers in Unicode, so I'm going to do make feature request this new terminal functionality.

claunia commented

@egmontkob commented on GitHub (Jan 27, 2020):

Another idea how to solve the problem of the allowing the screen drawing libraries (e.g. ncurses, slang ...) to display half characters to users. [...]

First of all, your proposal allows to place half glyphs with limitations. E.g. there'd be no way to place a left half of X, followed by the right half of Y. What if an app distinguishes UI elements by different background colors, without an explicit character (e.g. line drawing) as borders, and two such items happen to be next to each other? If you need support for half glyphs, you need support for this, too. Also, screen drawing libraries need to know what to do if you create such a layout in their in-memory representation.

Second, I must point out that your proposal breaks plenty of things in terminal emulation:

The emulation behavior is right now the same on the normal and the alternate screen, several apps even offer the users to pick one, expecting identical behavior. With your proposal, the behavior would be different, and those apps that don't switch to the alternate screen would have no chance of achieving your desired layout. You can't suddenly make the behavior different without potentially breaking things.
You can't just say that moving the cursor to column 0 will no longer clamp it to column 1 but will actually move it offscreen, without worrying about breaking plenty of apps. You can't just say the counterpart of this for the right margin, either.
You can't just say that all apps on full screen will have to from now on handle mouse themselves, and you can't forbid terminals from doing their own copy-pasting (e.g. for Shift+mouse).
You can't suddenly change the existing model of how resize events are delivered, or how an application knows which parts to repaint.

and so on... Unix terminals carry a legacy of maybe 50-ish years, and at every step we have to be careful not to break things that were developed in this time. WT developers are in an much even harder situation, as they have to merge two different worlds without breaking anything. We have to be extremely careful with every step.

[...] so I'm going to do make feature request this new terminal functionality.

I'm sorry but I failed to understand this sentence, and I'm not sure where you're about to request this. In Unicode? If I fail to convince you not to push for your broken attempts then at least I sincerely hope that you'll point them to this discussion, so that they have a chance to read our (Dustin's and my) comments.

One of your screenshots shows Midnight Commander, an application I used to develop. It can use either the popular ncurses or the not-so-popular slang screen drawing library. I've been remotely following the development of these two libraries. There's not much happening, and they are really slow in catching up with changes of the terminal world. Even if terminals supported half glyphs, these libraries may not catch up for years (if not decades), and may even need backwards incompatible changes, I'm not sure about that. And then whichever app (e.g. mc) would also need to add support, which, given how its development goes, is again extremely unlikely. Before going ahead, it wouldn't hurt to hear a buy-in from these involved people, which at the very least requires the feature to be implemented by several popular terminals, which, in turn, definitely needs the feature to be well designed. The terminal world has way more popular and technically way easier features that are still not supported by many terminals, as well as these libraries.

It's an enormous amount of work to fix half-emojis, with necessary buy-in and coordination among many parties. It's extremely unlikely to happen. And even if it happens, it won't happen the way you think should, but the way experienced developers of the ecosystem will together design it.

To begin with, from the terminal's point of view, N years of experience with developing a terminal as well as bits of the surrounding infrastucture, I can assert you that the only reasonable way to start is to forget about changing Unicode, and go for an escape sequence that does one and exactly one thing and nothing else: prints a half-character somewhere.

Let me also tell you that N years of developer experience tells me that the terminal ecosystem has way more important problems to fix than this one. Also, fixing this problem would require multiple parties to care as much as you do, which I firmly doubt will happen.

I've been keep an eye on the bugtracker of many terminals and some terminal-based apps, as well as various public forum for years. I haven't come across any report about half-CJKs. It's not something Chinese/Japanese/Korean folks really care about, or really wanted to fix (or maybe they do prefer to see a space rather than a half-glyph? could be, I don't know).

[And finally allow me one comment that is not professional but personal:

Widespread use of emojis made this technical problem more prominient for a much larger user base. For me, the terminal is a tool for getting work done. It's not for playing games, it's not for watching videos, it's not really for having fun, and (some might disagree with me) it's not really for browsing the web, handling e-mail etc. either. There are other great tools for those purposes. In order to get things done (i.e. typical developer and sysadmin tasks), I couldn't care less about emojis.]

@egmontkob commented on GitHub (Jan 27, 2020): > Another idea how to solve the problem of the allowing the screen drawing libraries (e.g. ncurses, slang ...) to display half characters to users. [...] First of all, your proposal allows to place half glyphs _with limitations_. E.g. there'd be no way to place a left half of X, followed by the right half of Y. What if an app distinguishes UI elements by different background colors, without an explicit character (e.g. line drawing) as borders, and two such items happen to be next to each other? If you need support for half glyphs, you need support for this, too. Also, screen drawing libraries need to know what to do if you create such a layout in their in-memory representation. Second, I must point out that your proposal *breaks* plenty of things in terminal emulation: - The emulation behavior is right now the same on the normal and the alternate screen, several apps even offer the users to pick one, expecting identical behavior. With your proposal, the behavior would be different, and those apps that don't switch to the alternate screen would have no chance of achieving your desired layout. You can't suddenly make the behavior different without potentially breaking things. - You can't just say that moving the cursor to column 0 will no longer clamp it to column 1 but will actually move it offscreen, without worrying about breaking plenty of apps. You can't just say the counterpart of this for the right margin, either. - You can't just say that all apps on full screen will have to from now on handle mouse themselves, and you can't forbid terminals from doing their own copy-pasting (e.g. for Shift+mouse). - You can't suddenly change the existing model of how resize events are delivered, or how an application knows which parts to repaint. and so on... Unix terminals carry a legacy of maybe 50-ish years, and at every step we have to be careful not to break things that were developed in this time. WT developers are in an much even harder situation, as they have to merge two different worlds without breaking anything. We have to be extremely careful with every step. > [...] so I'm going to do make feature request this new terminal functionality. I'm sorry but I failed to understand this sentence, and I'm not sure where you're about to request this. In Unicode? If I fail to convince you not to push for your broken attempts then at least I sincerely hope that you'll point them to this discussion, so that they have a chance to read our (Dustin's and my) comments. One of your screenshots shows Midnight Commander, an application I used to develop. It can use either the popular ncurses or the not-so-popular slang screen drawing library. I've been remotely following the development of these two libraries. There's not much happening, and they are really slow in catching up with changes of the terminal world. Even if terminals supported half glyphs, these libraries may not catch up for years (if not decades), and may even need backwards incompatible changes, I'm not sure about that. And then whichever app (e.g. mc) would also need to add support, which, given how its development goes, is again extremely unlikely. Before going ahead, it wouldn't hurt to hear a buy-in from these involved people, which at the very least requires the feature to be implemented by several popular terminals, which, in turn, definitely needs the feature to be well designed. The terminal world has way more popular and technically way easier features that are still not supported by many terminals, as well as these libraries. It's an *enormous* amount of work to fix half-emojis, with necessary buy-in and coordination among many parties. It's extremely unlikely to happen. And even if it happens, it won't happen the way you think should, but the way experienced developers of the ecosystem will together design it. To begin with, from the terminal's point of view, N years of experience with developing a terminal as well as bits of the surrounding infrastucture, I can assert you that the only reasonable way to start is to forget about changing Unicode, and go for an escape sequence that does one and exactly one thing and nothing else: prints a half-character somewhere. Let me also tell you that N years of developer experience tells me that the terminal ecosystem has way more important problems to fix than this one. Also, fixing this problem would require multiple parties to care as much as you do, which I firmly doubt will happen. I've been keep an eye on the bugtracker of many terminals and some terminal-based apps, as well as various public forum for years. I haven't come across any report about half-CJKs. It's not something Chinese/Japanese/Korean folks really care about, or really wanted to fix (or maybe they do prefer to see a space rather than a half-glyph? could be, I don't know). [And finally allow me one comment that is not professional but personal: Widespread use of emojis made this technical problem more prominient for a much larger user base. For me, the terminal is a tool for _getting work done_. It's not for playing games, it's not for watching videos, it's not really for having fun, and (some might disagree with me) it's not really for browsing the web, handling e-mail etc. either. There are other great tools for those purposes. In order to *get things done* (i.e. typical developer and sysadmin tasks), I couldn't care less about emojis.]

claunia commented

@o-sdn-o commented on GitHub (Jan 27, 2020):

My point is that the moment has come when terminal emulators require a certain special mode of operation for the correct presentation of text user interfaces, this is primarily due to the fact that emojis, CJK and many other unicode ones came to the world of terminals. And these things are very difficult to push into the world of terminals, which was formed decades before.

GUIs are always haviest stuff, but TUIs are much lighter, and if the terminals would provide convinient functionality/framework for building TUIs on it, this will be a big leap forward in terms of user experience in cases where using the GUI is impossible or impractical for some reasons.

@o-sdn-o commented on GitHub (Jan 27, 2020): My point is that the moment has come when terminal emulators require a certain special mode of operation for the correct presentation of text user interfaces, this is primarily due to the fact that emojis, CJK and many other unicode ones came to the world of terminals. And these things are very difficult to push into the world of terminals, which was formed decades before. GUIs are always haviest stuff, but TUIs are much lighter, and if the terminals would provide convinient functionality/framework for building TUIs on it, this will be a big leap forward in terms of user experience in cases where using the GUI is impossible or impractical for some reasons.

claunia commented

@egmontkob commented on GitHub (Jan 27, 2020):

I wouldn't call displaying a half emoji, when there's no room for the entire one, instead of showing a space, a "big leap". I'd call it perhaps a minor cosmetic improvement, maybe even a questionable one (some people might prefer not to see glyphs cut in half).

Pair it with up with the fact that it's an enormous task to fix it. It needs a technical solution which has to be discussed, evaluated wrt. backwards compatibility, feasibility of implementation and whatnot. And then it needs to be implemented in many terminals (of which most likely hardly any cares), couple of libraries (presumably none of which cares), followed by each and every single application that cares.

I'm not saying I don't want this to happen. I'm saying I'm pretty sure it won't happen. You're the first person I hear caring about this story. You might find this important, others probably don't.

@egmontkob commented on GitHub (Jan 27, 2020): I wouldn't call displaying a half emoji, when there's no room for the entire one, instead of showing a space, a "big leap". I'd call it perhaps a minor cosmetic improvement, maybe even a questionable one (some people might prefer not to see glyphs cut in half). Pair it with up with the fact that it's an *enormous* task to fix it. It needs a technical solution which has to be discussed, evaluated wrt. backwards compatibility, feasibility of implementation and whatnot. And then it needs to be implemented in *many* terminals (of which most likely hardly any cares), couple of libraries (presumably none of which cares), followed by each and every single application that cares. I'm not saying I don't want this to happen. I'm saying I'm pretty sure it won't happen. You're the first person I hear caring about this story. You might find this important, others probably don't.

claunia commented

@o-sdn-o commented on GitHub (Jan 27, 2020):

I wouldn't call displaying a half emoji, when there's no room for the entire one, instead of showing a space, a "big leap".

I don’t mean right now about half-characters, I mean, on the whole, the problem of displaying wide, complexly designed glyphs, working with the visible surface in the manner of graphic applications as a result of the possibilities of using true-color and various fonts at the same time. But all this graphic variety is strictly within the TUI.

@o-sdn-o commented on GitHub (Jan 27, 2020): > I wouldn't call displaying a half emoji, when there's no room for the entire one, instead of showing a space, a "big leap". I don’t mean right now about half-characters, I mean, on the whole, the problem of displaying wide, complexly designed glyphs, working with the visible surface in the manner of graphic applications as a result of the possibilities of using true-color and various fonts at the same time. But all this graphic variety is strictly within the TUI.

claunia commented

@o-sdn-o commented on GitHub (Jan 28, 2020):

Character segmentation and scaling

VT-sequence:
SCALE Nx; Dx; Ny; Dy
- turn scale mode ON and bind cursor coordinates with N and D
- D and N value range: [1-4]
- 1 byte/cell overhead in screen buffer

Unicode: 
- modifier name <GCSCALE1>..<GCSCALE256> (like VS1..256)
- in text: 
  <basechar><GCSCALE1..256>

Description:
- Storage: 
  byte = (Nx-1) + (Dx-1) * 4 + (Ny-1) * 16 + (Dy-1) * 64
- Meaning:
  e.g. 4x3 character in screen buffer:
  [1-4,3-3][2-4,3-3][3-4,3-3][4-4,3-3]
  [1-4,2-3][2-4,2-3][3-4,2-3][4-4,2-3]
  [1-4,1-3][2-4,1-3][3-4,1-3][4-4,1-3]
  e.g. 1x1 character: 
  [1-1,1-1]
  e.g. 2x1 character:
  [1-2,1-1][2-2,1-1]
  a) N <= D: select part N of character from D available and put it to single cell.
  b) N>D and D=any: stretch whole character on N cells
  c) N=D=0: reset scale mode to default (OFF)
- Coordinate binding:
  x0, y0 - cursor coord values when SCALE emitted
  x, y - current cursor position
  In case N<=D, Nx and Ny are function of x and y respectively: 
  Nx(x) = (x - x0 + (Nx0-1)) mod 4 ...
  Ny(y) = (y - y0 + (Ny0-1)) mod 4. ..
if (N < 0) N = 4 - N else N += 1 ...sleep 😕

This technique allows to work with characters ranging in size from 1 to 4 cells along both coordinate axes.

It doesn’t matter what size (cwidth) the character has, it allows put a wide character to a single cell if you want.

Output examples (VT sequence <SCALE;;;>)
- cout “a” produce  1x1 in buffer:
  [1/1,1/1]
- cout “😊” produce 2x1 in buffer:
  [1/2,1/1][2/2,1/1]
- cout “👨‍👩‍👧‍👦” produce 3x1 in buffer: 
  [1/3,1/1][2/3,1/1][3/3,1/1]
- cout “<SCALE1;1;1;1>😀” produce 1x1
- cout “<SCALE3;1;3;1>😀” produce 3x3
- cout “<SCALE1;1;1;1>👨‍👩‍👧‍👦” produce 1x1
- cout “<SCALE1;1;1;1>😀X” produce 1x1, 1x1
- cout “<SCALE2;1;1;1>😀X<SCALE0;0;0;0>H” produce 2x1(😀), 2x1(X), 1x1(H)
- cout “<SCALE1;2;1;1>😀🌎XH😀😀” produce 
  [1/2,1/1](left half 😀), [2/2,1/1](right half 🌎), [1/2,1/1](left half X) , [2/2,1/1](right half H), [1/2,1/1](left half 😀) , [2/2,1/1](right half 😀)

It is also possible with this technique to print out mathematical expressions and multi-level formulas (monospaced text documents with formulas, CJK, wide emoji and so on - are the Unicode problems that outside terminal world).

@o-sdn-o commented on GitHub (Jan 28, 2020): Character segmentation and scaling ``` VT-sequence: SCALE Nx; Dx; Ny; Dy - turn scale mode ON and bind cursor coordinates with N and D - D and N value range: [1-4] - 1 byte/cell overhead in screen buffer Unicode: - modifier name <GCSCALE1>..<GCSCALE256> (like VS1..256) - in text: <basechar><GCSCALE1..256> Description: - Storage: byte = (Nx-1) + (Dx-1) * 4 + (Ny-1) * 16 + (Dy-1) * 64 - Meaning: e.g. 4x3 character in screen buffer: [1-4,3-3][2-4,3-3][3-4,3-3][4-4,3-3] [1-4,2-3][2-4,2-3][3-4,2-3][4-4,2-3] [1-4,1-3][2-4,1-3][3-4,1-3][4-4,1-3] e.g. 1x1 character: [1-1,1-1] e.g. 2x1 character: [1-2,1-1][2-2,1-1] a) N <= D: select part N of character from D available and put it to single cell. b) N>D and D=any: stretch whole character on N cells c) N=D=0: reset scale mode to default (OFF) - Coordinate binding: x0, y0 - cursor coord values when SCALE emitted x, y - current cursor position In case N<=D, Nx and Ny are function of x and y respectively: Nx(x) = (x - x0 + (Nx0-1)) mod 4 ... Ny(y) = (y - y0 + (Ny0-1)) mod 4. .. if (N < 0) N = 4 - N else N += 1 ...sleep 😕 ``` This technique allows to work with characters ranging in size from 1 to 4 cells along both coordinate axes. It doesn’t matter what size (cwidth) the character has, it allows put a wide character to a single cell if you want. ``` Output examples (VT sequence <SCALE;;;>) - cout “a” produce 1x1 in buffer: [1/1,1/1] - cout “😊” produce 2x1 in buffer: [1/2,1/1][2/2,1/1] - cout “👨‍👩‍👧‍👦” produce 3x1 in buffer: [1/3,1/1][2/3,1/1][3/3,1/1] - cout “<SCALE1;1;1;1>😀” produce 1x1 - cout “<SCALE3;1;3;1>😀” produce 3x3 - cout “<SCALE1;1;1;1>👨‍👩‍👧‍👦” produce 1x1 - cout “<SCALE1;1;1;1>😀X” produce 1x1, 1x1 - cout “<SCALE2;1;1;1>😀X<SCALE0;0;0;0>H” produce 2x1(😀), 2x1(X), 1x1(H) - cout “<SCALE1;2;1;1>😀🌎XH😀😀” produce [1/2,1/1](left half 😀), [2/2,1/1](right half 🌎), [1/2,1/1](left half X) , [2/2,1/1](right half H), [1/2,1/1](left half 😀) , [2/2,1/1](right half 😀) ``` It is also possible with this technique to print out mathematical expressions and multi-level formulas (*monospaced* text documents with formulas, CJK, wide emoji and so on - are the Unicode problems that outside terminal world).

claunia commented

Accordingly to the Unicode® Standard Annex #29, "UNICODE TEXT SEGMENTATION"

@o-sdn-o commented on GitHub (Jan 29, 2020):

EDIT: 31 Jan 2020

Discussion is moved to the Terminals Working Group\Specifications\Issues.

I want to fill out the paper to a certain level, and then publish.

There will be no more updates here.

click to expand...

Character Segmentation and Scaling (Fractaling)

1. Abstract

This paper provides a general description of the solution to the problem of presenting and processing multi-sized (up to 4x4 cells) characters in a cell-based grid, where each cell one-to-one represents the visilbe part (fraction) of the character.

The following applications are related to multisize characters and character segmentation:

Terminal Emulators.
Linux console (when the X Window System is not running).
Windows Console.
Monospaced text documents.
... your suggestions

The solution allows to manipulate and store individual fractions of the whole character in a single cell for displaying them, as well as displaying multi-size characters in a cell-based grid and even allow their vertical splitting.
The solution also solves the problem of displaying wide characters in terminals by letting the terminal or the application running in it decide how wide the character will be, rather than relying on external data sources of these values that are subject to regular changes.

2. Solution

2.1 Definitions

It is important to recognize that what the user thinks of as a “character”—a basic unit of a writing system for a language—may not be just a single Unicode code point. Instead, that basic unit may be made up of multiple Unicode code points. To avoid ambiguity with the computer use of the term character, this is called a user-perceived character. For example, “G” + grave-accent is a user-perceived character: users think of it as a single character, yet is actually represented by two Unicode code points. These user-perceived characters are approximated by what is called a grapheme cluster, which can be determined programmatically.

This paper defines a character as user-perceived character (or grapheme cluster).

2.2 Mathematical Presentation

To correctly display either a whole character of any size (up to 4x4 cells) or any selected character segment, only four numeric parameters Ps = Dx, Nx, Dy, Ny with range values of each from 1 to 4 are required.

Parameters

Dx - count of parts along X-axis
Nx - either width of the whole character or segment selector of the Dx available parts from left to right along the X-axis
Dy - count of parts along Y-axis
Ny - either width of the whole character or segment selector of the Dy available parts from top to bottom along the Y-axis

Interpretation

There are several cases possible (for each axis accordingly)

D = 0
- turn the scale mode off.
N <= D
- select part N of the character from D available parts and use it as a sinle-cell character (along the corresponding axis).
N > D AND D = ANY
- stretch the character to N cells.

2.3 Storing In Memory

Screen Buffer / Monospaced Text File

Each multisize character with a size of n x m that is greater than 1x1 is stored in the screen buffer (or monospaced text file)W x H as a matrix of n x m

Example:

3x2 stretched character "A" is located at x=3, y=2 in the screen buffer (of monospaced text file)

	1	2	3	...	...	...	W
1	...	...	...	...	...	...	...
2	...	...	A+Ps1	A+Ps2	A+Ps3	...	...
...	...	...	A+Ps4	A+Ps5	A+Ps6	...	...
...	...	...	...	...	...	...	...
...	...	...	...	...	...	...	...
H	...	...	...	...	...	...	...

A = "A"
Ps1 = { Dx=3, Nx=1, Dy=2, Ny=1 }
Ps2 = { Dx=3, Nx=2, Dy=2, Ny=1 }
Ps3 = { Dx=3, Nx=3, Dy=2, Ny=1 }
Ps4 = { Dx=3, Nx=1, Dy=2, Ny=2 }
Ps5 = { Dx=3, Nx=2, Dy=2, Ny=2 }
Ps6 = { Dx=3, Nx=3, Dy=2, Ny=2 }

Ps can be packed in one byte and overhead of screen buffer is 1 byte per cell:

byte = (Nx-1) + (Dx-1) * 4 + (Ny-1) * 16 + (Dy-1) * 64

Also there are only 256 variants for the Unicode modifier character value 0 - 255.

Characters with parameters N > D are not allowed to be stored in a cell-based grid. When such a character is to be printed to the grid, it must be segmented for each grid cell, and the parameters are recalculated for each filled cell.

2.4 Naming

2.4.1 VT-Sequence

Variants of the name for the VT-sequence

Grapheme Cluster Scaling
GCS
GCSCALE
GCSC
... your suggestions

2.4.2 Unicode Standard

Name of the Unicode modifier letter

<GCSCALE1>..<GCSCALE256> (like VS1..VS256)
... your suggestions

3. Usage

3.1 Unicode Standard

Latest Unicode Standard defines three types of variation sequences:

Standardized variation sequences.
Emoji variation sequences.
Ideographic variation sequences defined in the Ideographic Variation Database.

Only those three types of variation sequences are sanctioned for use by conformant implementations.

Accorginly to the Standardized variation sequences FAQ

Q: How can I propose a standardized variation sequence?

A: You can initiate the process of requesting a variation sequence by submitting an inquiry via the contact form. A thorough understanding of how Variation Selectors are used will make a proposal more likely to be accepted by the UTC. Read Section 23.4, Variation Selectors, UTR #25 and UAX #34, as well as the rest of this FAQ for background information. [AF]

Accodingly to the Section 23.4, Variation Selectors, UTR #25

A variant form is a different glyph for a character, encoded in Unicode through the mechanism of variation sequences: sequences in Unicode that consist of a base character followed by a variation selector character.

Variation Sequence

In a variation sequence the variation selector affects the appearance of the base character. Such changes in appearance may, in turn, have a visual impact on subsequent characters, particularly combining characters applied to that base character.

The standardization or support of a particular variation sequence does not limit the set of glyphs that can be used to represent the base character alone.

Placement in the Text

<basechar><GCSCALE1..256>

what if there's a pause in the input stream after the base character?

If such a modifier appears the first in the input stream the terminal should be triggered to text reflowing as in the case of window resize.

3.2 VT-Sequence

XTerm Control Sequences, Functions using CSI

Assing VT-sequence as a CSI/SGR command, because it define characters rendition state and sets the appearance of the following characters.

Human readable format
	ESC[ 110; <n1>;<n2>;<n3>;<n4> m

Sequence with "cooked" parameter
	ESC[ 111; <P> m

n1, n2, n3, n4 are from 0 to 4.
n1 = Dx, n2 = Nx, n3 = Dy, n4 = Ny.
P = (Nx-1) + (Dx-1) * 4 + (Ny-1) * 16 + (Dy-1) * 64 from 0 to 255.
SGR code 110: missing numbers are treated as 1.
SGR code 111: missing number is treated as 0.
0 treated as reset the scaling mode (OFF).
ESC[m (all attributes off) also resets the scaling mode.
Instead of SGR codes 110, 111 suggest ... yours.

4. Expected Behavior

It doesn’t matter what size (cwidth) the character has, it allows put a wide character to a single cell if you want.

4.1 Unicode Standard

...

4.2 VT-Sequence

4.2.1 Printing

Output examples (VT sequence <SCALE;;;>)

- cout “a” produce  1x1 in buffer:
  [1/1,1/1]
- cout “😊” produce 2x1 in buffer:
  [1/2,1/1][2/2,1/1]
- cout “👨‍👩‍👧‍👦” produce 3x1 in buffer: 
  [1/3,1/1][2/3,1/1][3/3,1/1]
- cout “<SCALE1;1;1;1>😀” produce 1x1
- cout “<SCALE3;1;3;1>😀” produce 3x3
- cout “<SCALE1;1;1;1>👨‍👩‍👧‍👦” produce 1x1
- cout “<SCALE1;1;1;1>😀X” produce 1x1, 1x1
- cout “<SCALE2;1;1;1>😀X<SCALE0;0;0;0>H” produce 2x1(😀), 2x1(X), 1x1(H)
- cout “<SCALE1;2;1;1>😀🌎XH😀😀” produce 
  [1/2,1/1](left half 😀), [2/2,1/1](right half 🌎), [1/2,1/1](left half X) , [2/2,1/1](right half H), [1/2,1/1](left half 😀) , [2/2,1/1](right half 😀)

It is also possible with this technique to print out mathematical expressions and multi-level formulas (monospace textual documents with formulas, CJK, wide emoji and so on - are the Unicode problems that outside terminal world).

Line Wrap

...

Side effects

...

4.2.2 Capturing

...

5. Applications

5.1 Cost of Initial Implementation

...

6. Existed Infrastructure Compatibility

...

7. Security Issues

7.1 Unicode Security Considerations

Unicode Technical Report #36

This section describes some of the security considerations that programmers, system analysts, standards developers, and users should take into account.

For example, consider visual spoofing, where a similarity in visual appearance fools a user and causes him or her to take unsafe actions.

...

@o-sdn-o commented on GitHub (Jan 29, 2020): ``` EDIT: 31 Jan 2020 ``` Discussion is moved to the [Terminals Working Group\Specifications\Issues](https://gitlab.freedesktop.org/terminal-wg/specifications/issues/23). ``` I want to fill out the paper to a certain level, and then publish. There will be no more updates here. ``` <details><summary>click to expand...</summary><p> ## Character Segmentation and Scaling (Fractaling) ### 1. Abstract This paper provides a general description of the solution to the problem of presenting and processing multi-sized (up to 4x4 cells) characters in a cell-based grid, where each cell one-to-one represents the visilbe part (fraction) of the character. The following applications are related to multisize characters and character segmentation: - Terminal Emulators. - Linux console (when the X Window System is not running). - Windows Console. - Monospaced text documents. - ... your suggestions The solution allows to manipulate and store individual fractions of the whole character in a single cell for displaying them, as well as displaying multi-size characters in a cell-based grid and even allow their vertical splitting. The solution also solves the problem of displaying wide characters in terminals by letting the terminal or the application running in it decide how wide the character will be, rather than relying on external data sources of these values that are subject to regular changes. ### 2. Solution #### 2.1 Definitions [Accordingly to the Unicode® Standard Annex #29, "UNICODE TEXT SEGMENTATION"](https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries) > It is important to recognize that what the user thinks of as a “character”—a basic unit of a writing system for a language—may not be just a single Unicode code point. Instead, that basic unit may be made up of multiple Unicode code points. To avoid ambiguity with the computer use of the term character, this is called a user-perceived character. For example, “G” + grave-accent is a user-perceived character: users think of it as a single character, yet is actually represented by two Unicode code points. These user-perceived characters are approximated by what is called a grapheme cluster, which can be determined programmatically. This paper defines a character as user-perceived character (or grapheme cluster). #### 2.2 Mathematical Presentation To correctly display either a whole character of any size (up to 4x4 cells) or any selected character segment, only four numeric parameters `Ps` = `Dx`, `Nx`, `Dy`, `Ny` with range values of each from 1 to 4 are required. ##### Parameters - `Dx` - count of parts along X-axis - `Nx` - either width of the whole character or segment selector of the `Dx` available parts from left to right along the X-axis - `Dy` - count of parts along Y-axis - `Ny` - either width of the whole character or segment selector of the `Dy` available parts from top to bottom along the Y-axis ##### Interpretation There are several cases possible (for each axis accordingly) - `D = 0` - turn the scale mode off. - `N <= D` - select part `N` of the character from `D` available parts and use it as a sinle-cell character (along the corresponding axis). - `N > D AND D = ANY` - stretch the character to `N` cells. #### 2.3 Storing In Memory ##### Screen Buffer / Monospaced Text File Each multisize character with a size of `n x m` that is greater than `1x1` is stored in the screen buffer (or monospaced text file)`W x H` as a matrix of `n x m` Example: 3x2 stretched character `"A"` is located at `x=3, y=2` in the screen buffer (of monospaced text file) | | 1 | 2 | 3 | ... | ... | ... | W | |:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| | 1 | ... | ... | ... | ... | ... | ... | ... | | 2 | ... | ... |A+Ps1|A+Ps2|A+Ps3| ... | ... | | ... | ... | ... |A+Ps4|A+Ps5|A+Ps6| ... | ... | | ... | ... | ... | ... | ... | ... | ... | ... | | ... | ... | ... | ... | ... | ... | ... | ... | | H | ... | ... | ... | ... | ... | ... | ... | ``` A = "A" Ps1 = { Dx=3, Nx=1, Dy=2, Ny=1 } Ps2 = { Dx=3, Nx=2, Dy=2, Ny=1 } Ps3 = { Dx=3, Nx=3, Dy=2, Ny=1 } Ps4 = { Dx=3, Nx=1, Dy=2, Ny=2 } Ps5 = { Dx=3, Nx=2, Dy=2, Ny=2 } Ps6 = { Dx=3, Nx=3, Dy=2, Ny=2 } ``` `Ps` can be packed in one byte and overhead of screen buffer is 1 byte per cell: ``` byte = (Nx-1) + (Dx-1) * 4 + (Ny-1) * 16 + (Dy-1) * 64 ``` Also there are only 256 variants for the Unicode modifier character value 0 - 255. Characters with parameters `N > D` are not allowed to be stored in a cell-based grid. When such a character is to be printed to the grid, it must be segmented for each grid cell, and the parameters are recalculated for each filled cell. #### 2.4 Naming ##### 2.4.1 VT-Sequence Variants of the name for the VT-sequence - Grapheme Cluster Scaling - GCS - GCSCALE - GCSC - ... your suggestions ##### 2.4.2 Unicode Standard Name of the Unicode modifier letter - \<GCSCALE1>..\<GCSCALE256> (like VS1..VS256) - ... your suggestions ### 3. Usage #### 3.1 Unicode Standard Latest Unicode Standard defines three types of variation sequences: - Standardized variation sequences. - Emoji variation sequences. - Ideographic variation sequences defined in the Ideographic Variation Database. Only those three types of variation sequences are sanctioned for use by conformant implementations. [Accorginly to the Standardized variation sequences FAQ](http://unicode.org/faq/vs.html) > Q: How can I propose a standardized variation sequence? > A: You can initiate the process of requesting a variation sequence by submitting an inquiry via the contact form. A thorough understanding of how Variation Selectors are used will make a proposal more likely to be accepted by the UTC. Read Section 23.4, Variation Selectors, UTR #25 and UAX #34, as well as the rest of this FAQ for background information. [AF] [Accodingly to the Section 23.4, Variation Selectors, UTR #25](http://www.unicode.org/versions/Unicode12.1.0/ch23.pdf#G19053) > A variant form is a different glyph for a character, encoded in Unicode through the mechanism of variation sequences: sequences in Unicode that consist of a base character followed by a variation selector character. ##### Variation Sequence > In a variation sequence the variation selector affects the appearance of the base character. Such changes in appearance may, in turn, have a visual impact on subsequent characters, particularly combining characters applied to that base character. > The standardization or support of a particular variation sequence does not limit the set of glyphs that can be used to represent the base character alone. ##### Placement in the Text ``` <basechar><GCSCALE1..256> ``` > what if there's a pause in the input stream after the base character? If such a modifier appears the first in the input stream the terminal should be triggered to text reflowing as in the case of window resize. #### 3.2 VT-Sequence [XTerm Control Sequences, Functions using CSI](https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h3-Functions-using-CSI-_-ordered-by-the-final-character_s_) Assing VT-sequence as a CSI/SGR command, because it define characters rendition state and sets the appearance of the following characters. ``` Human readable format ESC[ 110; <n1>;<n2>;<n3>;<n4> m Sequence with "cooked" parameter ESC[ 111; <P> m ``` - `n1`, `n2`, `n3`, `n4` are from 0 to 4. - `n1 = Dx`, `n2 = Nx`, `n3 = Dy`, `n4 = Ny`. - `P = (Nx-1) + (Dx-1) * 4 + (Ny-1) * 16 + (Dy-1) * 64` from 0 to 255. - SGR code `110`: missing numbers are treated as 1. - SGR code `111`: missing number is treated as 0. - 0 treated as reset the scaling mode (OFF). - `ESC[m` (all attributes off) also resets the scaling mode. - Instead of SGR codes `110`, `111` suggest `...` yours. ### 4. Expected Behavior It doesn’t matter what size (cwidth) the character has, it allows put a wide character to a single cell if you want. #### 4.1 Unicode Standard ``` ... ``` #### 4.2 VT-Sequence ##### 4.2.1 Printing Output examples (VT sequence <SCALE;;;>) ``` - cout “a” produce 1x1 in buffer: [1/1,1/1] - cout “😊” produce 2x1 in buffer: [1/2,1/1][2/2,1/1] - cout “👨‍👩‍👧‍👦” produce 3x1 in buffer: [1/3,1/1][2/3,1/1][3/3,1/1] - cout “<SCALE1;1;1;1>😀” produce 1x1 - cout “<SCALE3;1;3;1>😀” produce 3x3 - cout “<SCALE1;1;1;1>👨‍👩‍👧‍👦” produce 1x1 - cout “<SCALE1;1;1;1>😀X” produce 1x1, 1x1 - cout “<SCALE2;1;1;1>😀X<SCALE0;0;0;0>H” produce 2x1(😀), 2x1(X), 1x1(H) - cout “<SCALE1;2;1;1>😀🌎XH😀😀” produce [1/2,1/1](left half 😀), [2/2,1/1](right half 🌎), [1/2,1/1](left half X) , [2/2,1/1](right half H), [1/2,1/1](left half 😀) , [2/2,1/1](right half 😀) ``` It is also possible with this technique to print out mathematical expressions and multi-level formulas (*monospace* textual documents with formulas, CJK, wide emoji and so on - are the Unicode problems that outside terminal world). ###### Line Wrap ``` ... ``` ###### Side effects ``` ... ``` ##### 4.2.2 Capturing ``` ... ``` ### 5. Applications #### 5.1 Cost of Initial Implementation ``` ... ``` ### 6. Existed Infrastructure Compatibility ``` ... ``` ### 7. Security Issues #### 7.1 Unicode Security Considerations [Unicode Technical Report #36](http://unicode.org/reports/tr36/) This section describes some of the security considerations that programmers, system analysts, standards developers, and users should take into account. For example, consider visual spoofing, where a similarity in visual appearance fools a user and causes him or her to take unsafe actions. ``` ... ``` </p></details>

claunia commented

@jerch commented on GitHub (Jan 31, 2020):

@o-sdn-o Can you open an issue in terminal-wg? This way more terminals devs will see it and it can be dicussed more in detail.

Some early remarks from my side:
As @egmontkob already pointed out - we have several issues with newer unicode rules in general in the terminal. While up to unicode 8 the simple wcwidth approach worked for most things (minus BiDi), the newer unicode versions introduced several new rules, that are not quite terminal environment friendly:

grapheme clustering: currently not handled correctly by any common terminal or cmdline lib - this should be changed
wcwidth is kinda broken / cannot deal with complicated clustering rules anymore
different screen rendering possibilities given by unicode (like emojis can have a text/picogram repr, compounds can have different "run-widths") needs to be dealt with
different unicode versions imply different rules, needs some kind of version "handshaking"

Among these issues the question whether a n-wide user percieved character can be split into n parts seems to be a minor detail. Until we have not fixed the basics it is hard to think about further extensions or alternate behavior.

Currently all terminals follow a common behavior here for n=2 (wider clusters were no issue until cluster rules took place) at the end of the row: if DECAWM (auto wrap) is set, the wide char would reflow to the next line early, otherwise it gets not printed (stays blank). Erasing an east asian wide char would actually clear two cells in most terminals.

Also note that your suggestion might not work with every terminal/render engine (not every terminal does the rendering on its own, technical level). Also to me it feels weird to be able to split those chars up (semantical level).

@jerch commented on GitHub (Jan 31, 2020): @o-sdn-o Can you open an issue in terminal-wg? This way more terminals devs will see it and it can be dicussed more in detail. Some early remarks from my side: As @egmontkob already pointed out - we have several issues with newer unicode rules in general in the terminal. While up to unicode 8 the simple `wcwidth` approach worked for most things (minus BiDi), the newer unicode versions introduced several new rules, that are not quite terminal environment friendly: - grapheme clustering: currently not handled correctly by any common terminal or cmdline lib - this should be changed - wcwidth is kinda broken / cannot deal with complicated clustering rules anymore - different screen rendering possibilities given by unicode (like emojis can have a text/picogram repr, compounds can have different "run-widths") needs to be dealt with - different unicode versions imply different rules, needs some kind of version "handshaking" Among these issues the question whether a n-wide user percieved character can be split into n parts seems to be a minor detail. Until we have not fixed the basics it is hard to think about further extensions or alternate behavior. Currently all terminals follow a common behavior here for n=2 (wider clusters were no issue until cluster rules took place) at the end of the row: if DECAWM (auto wrap) is set, the wide char would reflow to the next line early, otherwise it gets not printed (stays blank). Erasing an east asian wide char would actually clear two cells in most terminals. Also note that your suggestion might not work with every terminal/render engine (not every terminal does the rendering on its own, technical level). Also to me it feels weird to be able to split those chars up (semantical level).

claunia commented