UTF-8 line-drawing characters do not render correctly in Terminal vs. console #3130

@DHowett-MSFT commented on GitHub (Aug 5, 2019):

It looks like this is in tmux or screen. Can you share the value of $TERM inside and outside your multiplexer?

@DHowett-MSFT commented on GitHub (Aug 5, 2019): It looks like this is in tmux or screen. Can you share the value of `$TERM` inside and outside your multiplexer?

claunia commented

@cerebrate commented on GitHub (Aug 5, 2019):

Outside, it’s xterm-256color. Inside, it’s screen-256color.

I should add, though, that I also thought it might be tmux-related, so I tried the same experiment without it running. Same results.

@cerebrate commented on GitHub (Aug 5, 2019): Outside, it’s xterm-256color. Inside, it’s screen-256color. I should add, though, that I also thought it might be tmux-related, so I tried the same experiment without it running. Same results.

claunia commented

@zadjii-msft commented on GitHub (Aug 5, 2019):

@cerebrate What font are you using? Is en-us.UTF-8 important? I'm working with C.UTF-8 over here, and that seems just fine.

Curiously also, in the conhost.exe window on the right, you have LANG=C.UTF-8.

@zadjii-msft commented on GitHub (Aug 5, 2019): @cerebrate What font are you using? Is `en-us.UTF-8` important? I'm working with `C.UTF-8` over here, and that seems just fine. Curiously also, in the conhost.exe window on the right, you have `LANG=C.UTF-8`.

claunia commented

@DHowett-MSFT commented on GitHub (Aug 5, 2019):

Hm. Can you run

pstree -G
pstree -A
pstree -U

in both conhost and Terminal?

@DHowett-MSFT commented on GitHub (Aug 5, 2019): Hm. Can you run ``` pstree -G pstree -A pstree -U ``` in both conhost and Terminal?

claunia commented

@cerebrate commented on GitHub (Aug 6, 2019):

@zadjii-msft Consolas.

And I'm using en-US.UTF-8 rather than C.UTF-8 primarily for collation reasons; I do a fair amount of work with non-ASCII characters, which en-US.UTF-8 sorts correctly and C.UTF-8 does not.

@cerebrate commented on GitHub (Aug 6, 2019): @zadjii-msft Consolas. And I'm using `en-US.UTF-8` rather than `C.UTF-8` primarily for collation reasons; I do a fair amount of work with non-ASCII characters, which `en-US.UTF-8` sorts correctly and `C.UTF-8` does not.

claunia commented

@cerebrate commented on GitHub (Aug 6, 2019):

@DHowett-MSFT Well, this is awkward.

Since reporting this issue yesterday, and without me intentionally changing anything (font, locale, etc.), it is now working - i.e., rendering with line-drawing characters in Terminal, as well as in conhost. Behavior on running the three commands in either is as expected: -G and -U render with line-drawing characters, -A with ASCII art, and no underlines are to be seen.

And so far, I can't seem to break it again. Gah. I hate heisenbugs.

Leaving this open for now, and I'll try tomorrow to repro it on another machine still running on installation defaults.

@cerebrate commented on GitHub (Aug 6, 2019): @DHowett-MSFT Well, this is awkward. Since reporting this issue yesterday, and without me intentionally changing anything (font, locale, etc.), it is now working - i.e., rendering with line-drawing characters in Terminal, as well as in conhost. Behavior on running the three commands in either is as expected: -G and -U render with line-drawing characters, -A with ASCII art, and no underlines are to be seen. And so far, I can't seem to break it again. Gah. I _hate_ heisenbugs. Leaving this open for now, and I'll try tomorrow to repro it on another machine still running on installation defaults.

claunia commented

@nicm commented on GitHub (Aug 6, 2019):

tmux will draw all UTF-8 characters as underscores if it thinks the terminal does not support UTF-8, perhaps that is the case here. Are you sure at least one of LANG, LC_ALL or LC_CTYPE is set correctly outside tmux? You can also use -u to tell tmux explicitly that the terminal supports UTF-8.

@nicm commented on GitHub (Aug 6, 2019): tmux will draw all UTF-8 characters as underscores if it thinks the terminal does not support UTF-8, perhaps that is the case here. Are you sure at least one of LANG, LC_ALL or LC_CTYPE is set correctly *outside* tmux? You can also use -u to tell tmux explicitly that the terminal supports UTF-8.

claunia commented

@DHowett-MSFT commented on GitHub (Aug 8, 2019):

I'll close this one out in light of "tmux ... underscores ...". Thanks!

@DHowett-MSFT commented on GitHub (Aug 8, 2019): I'll close this one out in light of "tmux ... underscores ...". Thanks!

claunia commented

@cerebrate commented on GitHub (Aug 8, 2019):

@nicm Trouble is, the same thing was happening outside tmux, per comment above: https://github.com/microsoft/terminal/issues/2243#issuecomment-518049785 ; just bypassing all my normal .zprofile / .zcshrc and starting up in bash alone.

On the other hand, since I can't get the issue to reproduce, yeah, might as well go ahead and close it/leave it closed.

@cerebrate commented on GitHub (Aug 8, 2019): @nicm Trouble is, the same thing was happening outside tmux, per comment above: https://github.com/microsoft/terminal/issues/2243#issuecomment-518049785 ; just bypassing all my normal .zprofile / .zcshrc and starting up in bash alone. On the other hand, since I can't get the issue to reproduce, yeah, might as well go ahead and close it/leave it closed.

claunia commented

@iamkentleom commented on GitHub (May 26, 2020):

I had the same issue. Probably there's something wrong with the rendering/screen because I've tried making the font bigger and everything came back to normal. Probably the font is small enough that the rendered line can't be seen.

@iamkentleom commented on GitHub (May 26, 2020): I had the same issue. Probably there's something wrong with the rendering/screen because I've tried making the font bigger and everything came back to normal. Probably the font is small enough that the rendered line can't be seen.

claunia commented

@ivho commented on GitHub (Oct 12, 2021):

I had the same issue. I'm running tmux on a server that I'm not root on so I can't change my default shell from tcsh to bash. If I start tmux directly from tcsh I get the underscores, if I detach, start bash and attach to tmux again it rerenders the output and it looks fine, i.e I don't even have to rerun pstree. $TERM is xterm-256color in both bash and tcsh.

edit: tmux v3.0

@ivho commented on GitHub (Oct 12, 2021): I had the same issue. I'm running tmux on a server that I'm not root on so I can't change my default shell from tcsh to bash. If I start tmux directly from tcsh I get the underscores, if I detach, start bash and attach to tmux again it rerenders the output and it looks fine, i.e I don't even have to rerun pstree. $TERM is xterm-256color in both bash and tcsh. edit: tmux v3.0

claunia commented

@DHowett commented on GitHub (Oct 12, 2021):

Curious! @ivho what's the output of locale? (From both shells)

@DHowett commented on GitHub (Oct 12, 2021): Curious! @ivho what's the output of `locale`? (From both shells)

claunia commented

@ivho commented on GitHub (Oct 12, 2021):

 [17:50] [/home/xxx] -> locale
LANG=en_US
LC_CTYPE="en_US"
LC_NUMERIC=sv_SE.UTF-8
LC_TIME=C
LC_COLLATE=C
LC_MONETARY=sv_SE.UTF-8
LC_MESSAGES=C
LC_PAPER=sv_SE.UTF-8
LC_NAME=sv_SE.UTF-8
LC_ADDRESS=sv_SE.UTF-8
LC_TELEPHONE=sv_SE.UTF-8
LC_MEASUREMENT=sv_SE.UTF-8
LC_IDENTIFICATION=sv_SE.UTF-8
LC_ALL=
 [17:50] [xxx] -> bash
nn@xx:~$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

@ivho commented on GitHub (Oct 12, 2021): ``` [17:50] [/home/xxx] -> locale LANG=en_US LC_CTYPE="en_US" LC_NUMERIC=sv_SE.UTF-8 LC_TIME=C LC_COLLATE=C LC_MONETARY=sv_SE.UTF-8 LC_MESSAGES=C LC_PAPER=sv_SE.UTF-8 LC_NAME=sv_SE.UTF-8 LC_ADDRESS=sv_SE.UTF-8 LC_TELEPHONE=sv_SE.UTF-8 LC_MEASUREMENT=sv_SE.UTF-8 LC_IDENTIFICATION=sv_SE.UTF-8 LC_ALL= [17:50] [xxx] -> bash nn@xx:~$ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=en_US.UTF-8 ```

claunia commented

@DHowett commented on GitHub (Oct 12, 2021):

I suspect that the missing .UTF-8 on LANG (and perhaps the empty LC_ALL?) are contributing to tmux's decision to remove the UTF-8 characters from its output. 😄

@DHowett commented on GitHub (Oct 12, 2021): I suspect that the missing `.UTF-8` on `LANG` (and perhaps the empty `LC_ALL`?) are contributing to tmux's decision to remove the UTF-8 characters from its output. :smile:

claunia commented