starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-09-24 16:05:21 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

Null character in the Windows Terminal causes subsequent characters to be dropped #2749

New Issue
Closed
opened 2026-01-30 23:04:05 +00:00 by claunia · 4 comments
claunia commented 2026-01-30 23:04:05 +00:00
Owner
Copy Link

Originally created by @j4james on GitHub (Jul 13, 2019).

Environment

Windows build number: 10.0.18362.175
Windows Terminal version (if applicable): Locally built from commit 3377f06

Steps to reproduce

  1. Build the Windows Terminal locally, and run Windows Terminal (Dev build) from the Start menu.

  2. When the default PowerShell tab opens, type the following command:

     echo "BEFORE `0 AFTER"

Expected behavior

I'd expect to see it output something like this:

BEFORE   AFTER

Actual behavior

The output ends at the null character, and the AFTER part of the string is not displayed.

image

Sometimes I don't even see the following command prompt - I assume it depends on how much content is buffered before it's output.

I can't reproduce this in the released version of Windows Terminal, though - it only happens for me with a locally built version. But looking at the code in the debugger, I can see why it is failing, so I'm really confused why it would ever work.

If you look at the TermControl::_InitializeTerminal method, there's a lambda expression which takes the terminal output, in the form of an hstring, and passes it on to the Terminal::Write method, which expects a wstring_view.

3377f06e52/src/cascadia/TerminalControl/TermControl.cpp (L418-L420)

But because it's using the c_str method on the hstring, it's losing the length of the data, so the wstring_view is forced to guess the length from the null terminator (thus losing half the characters). If that hstring had been passed directly to the Write method, it would actually have worked, because hstring has a cast operator which automatically converts to a wstring_view, and retains the real length.

Btw, I have actually built a version of the code with the c_str() call removed, and confirmed that it does fix the problem for me.

Originally created by @j4james on GitHub (Jul 13, 2019). # Environment Windows build number: 10.0.18362.175 Windows Terminal version (if applicable): Locally built from commit [3377f06](https://github.com/microsoft/terminal/commit/3377f06e52914fbe5b536f74b234c5e7173dfaf2) # Steps to reproduce 1. Build the Windows Terminal locally, and run _Windows Terminal (Dev build)_ from the _Start_ menu. 2. When the default PowerShell tab opens, type the following command: echo "BEFORE `0 AFTER" # Expected behavior I'd expect to see it output something like this: BEFORE AFTER # Actual behavior The output ends at the null character, and the `AFTER` part of the string is not displayed. ![image](https://user-images.githubusercontent.com/4181424/61172353-f36b4f00-a57a-11e9-9312-cda81a3ebb44.png) Sometimes I don't even see the following command prompt - I assume it depends on how much content is buffered before it's output. I can't reproduce this in the released version of Windows Terminal, though - it only happens for me with a locally built version. But looking at the code in the debugger, I can see why it is failing, so I'm really confused why it would ever work. If you look at the `TermControl::_InitializeTerminal` method, there's a lambda expression which takes the terminal output, in the form of an `hstring`, and passes it on to the `Terminal::Write` method, which expects a `wstring_view`. https://github.com/microsoft/terminal/blob/3377f06e52914fbe5b536f74b234c5e7173dfaf2/src/cascadia/TerminalControl/TermControl.cpp#L418-L420 But because it's using the `c_str` method on the `hstring`, it's losing the length of the data, so the `wstring_view` is forced to guess the length from the null terminator (thus losing half the characters). If that `hstring` had been passed directly to the `Write` method, it would actually have worked, because `hstring` has a cast operator which automatically converts to a `wstring_view`, and retains the real length. Btw, I have actually built a version of the code with the `c_str()` call removed, and confirmed that it does fix the problem for me.
claunia commented 2026-01-30 23:04:06 +00:00
Author
Owner
Copy Link

@DHowett-MSFT commented on GitHub (Jul 14, 2019):

Yeah, that's definitely wrong. We also need to be way better about buffer handling in the control<->connection layer. We are almost certainly making more copies than we should, and I think eventually I want to switch us to a Read-style interface or, at least, pass more byte buffers and stay in UTF-8 byte streams and out of strings as long as possible.

@DHowett-MSFT commented on GitHub (Jul 14, 2019): Yeah, that's definitely wrong. We also need to be way better about buffer handling in the control<->connection layer. We are almost certainly making more copies than we should, and I think eventually I want to switch us to a Read-style interface or, at least, pass more byte buffers and stay in UTF-8 byte streams and out of strings as long as possible.
claunia commented 2026-01-30 23:04:06 +00:00
Author
Owner
Copy Link

@DHowett-MSFT commented on GitHub (Jul 14, 2019):

(we also use c_str and to_string and to_hstring way more than we really should.)

@DHowett-MSFT commented on GitHub (Jul 14, 2019): (we also use c_str and to_string and to_hstring way more than we really should.)
claunia commented 2026-01-30 23:04:06 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Jul 14, 2019):

I just realised why this only fails in my local build - it's because I'm to blame for breaking it! This came up when I was testing my fix for issue #166, and it turns out that my fix ended up changing the behaviour of the null character.

There's a special-case check in WriteCharsLegacy that maps the null character to a space, but after I fixed the GetStringType test, the null started being matched as a C1_CNTRL, so no longer fell through to its previous special-case handling. You guys were clearly justified in being hesitant to mess with the WriteCharsLegacy code!

In short, this is possibly not a real bug. Technically the code may be incorrect, but it may not matter in practice if you never get nulls in those strings.

@j4james commented on GitHub (Jul 14, 2019): I just realised why this only fails in my local build - it's because I'm to blame for breaking it! This came up when I was testing my fix for issue #166, and it turns out that my fix ended up changing the behaviour of the null character. There's a special-case check in `WriteCharsLegacy` that [maps the null character to a space](https://github.com/microsoft/terminal/blob/3377f06e52914fbe5b536f74b234c5e7173dfaf2/src/host/_stream.cpp#L527-L530), but after I fixed the `GetStringType` test, the null started being matched as a `C1_CNTRL`, so no longer fell through to its previous special-case handling. You guys were clearly justified in being hesitant to mess with the `WriteCharsLegacy` code! In short, this is possibly not a real bug. Technically the code may be incorrect, but it may not matter in practice if you never get nulls in those strings.
claunia commented 2026-01-30 23:04:06 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Jan 26, 2020):

I found another way to reproduce this issue. You can write a null into the screen buffer with FillConsoleOutputCharacter, and that null then gets propagated over the conpty pipe blocking any further output that follows it. A simple example that can cause all sorts of chaos is:

HANDLE handle = GetStdHandle(STD_OUTPUT_HANDLE);
FillConsoleOutputCharacterA(handle, 0, 1, COORD{ 0, 0 }, NULL);

I thought this was good incentive to submit a PR getting rid of some of these unnecessary c_str calls, including the one that is causing this bug.

That said, this is actually a sign of a bigger problem - you can get any control character into the screen buffer this way, and it'll be interpreted by conpty as a control when it should be a glyph. I'll raise that as a separate issue though.

@j4james commented on GitHub (Jan 26, 2020): I found another way to reproduce this issue. You can write a null into the screen buffer with `FillConsoleOutputCharacter`, and that null then gets propagated over the conpty pipe blocking any further output that follows it. A simple example that can cause all sorts of chaos is: ```cpp HANDLE handle = GetStdHandle(STD_OUTPUT_HANDLE); FillConsoleOutputCharacterA(handle, 0, 1, COORD{ 0, 0 }, NULL); ``` I thought this was good incentive to submit a PR getting rid of some of these unnecessary `c_str` calls, including the one that is causing this bug. That said, this is actually a sign of a bigger problem - you can get any control character into the screen buffer this way, and it'll be interpreted by conpty as a control when it should be a glyph. I'll raise that as a separate issue though.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label Area-TerminalControl Issue-Bug Needs-Tag-Fix Product-Terminal Resolution-Fix-Committed
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#2749
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?