starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

ConPTY flush unknown CSI sequences too early #19758

New Issue
Closed
opened 2026-01-31 06:52:46 +00:00 by claunia · 9 comments
claunia commented 2026-01-31 06:52:46 +00:00
Owner
Copy Link

Originally created by @Mervap on GitHub (Apr 24, 2023).

Windows Terminal version

Not installed

Windows build number

10.0.19045.2846

Other Software

.NET 7.0 x64
PowerShell 7.3.4

Steps to reproduce

  1. Create ConPTY from dotnet (minimized version below, full working code here, based on samples/GUIConsole.ConPTY)
  2. Run powershell.exe with next ps1 script:
[Console]::Write("Begin")
[Console]::Write("`e[1q")
[Console]::Write("IN_BETWEEN_1")
[Console]::Write("`e[1Q")
[Console]::Write("`e[2q")
[Console]::Write("IN_BETWEEN_2")
[Console]::Write("`e[2Q")
[Console]::Write("`e[3q")
[Console]::Write("IN_BETWEEN_3")
[Console]::Write("`e[3Q")
[Console]::Write("Some long string at the end of output: aaaaaaaaaaaaaaa")
  1. Read charachters from PTY output
SafeFileHandle inputRead, inputWrite, outputRead, outputWrite;

CreatePseudoConsole(new COORD { X = 40, Y = 20 }, inputRead, outputWrite, 0, out IntPtr hPC);
ProcessFactory.Start("powershell.exe -File a.ps1", 0x00020016, hPC);

var reader = new StreamReader(new FileStream(outputRead, FileAccess.Read));

// Read output
var bytesRead = reader.ReadBlock(buf, 0, 140);
var output = new string(buf.Take(bytesRead).ToArray());
Console.Out.Write(output);

Expected Behavior

The entries in the output are in the same order, in which they were written

Begin←[1qIN_BETWEEN_1←[1Q←[2qIN_BETWEEN_2←[2Q←[3qIN_BETWEEN_3←[3Q

Actual Behavior

CSI sequences are printed before the text output

←[1q←[1Q←[2q←[2Q←[3q←[3QBeginIN_BETWEEN_1IN_BETWEEN_2IN_BETWEEN_3

I think it happens here
Don't you think that it's betteer to flush text buffer before flushing escape sequences?
And can you suggest some workaround? Can I flush text buffer manually from script?
Thanks

Originally created by @Mervap on GitHub (Apr 24, 2023). ### Windows Terminal version Not installed ### Windows build number 10.0.19045.2846 ### Other Software .NET 7.0 x64 PowerShell 7.3.4 ### Steps to reproduce 1) Create ConPTY from dotnet (minimized version below, full working code [here](https://github.com/microsoft/terminal/files/11309418/Wrap.zip), based on `samples/GUIConsole.ConPTY`) 2) Run `powershell.exe` with next `ps1` script: ```ps1 [Console]::Write("Begin") [Console]::Write("`e[1q") [Console]::Write("IN_BETWEEN_1") [Console]::Write("`e[1Q") [Console]::Write("`e[2q") [Console]::Write("IN_BETWEEN_2") [Console]::Write("`e[2Q") [Console]::Write("`e[3q") [Console]::Write("IN_BETWEEN_3") [Console]::Write("`e[3Q") [Console]::Write("Some long string at the end of output: aaaaaaaaaaaaaaa") ``` 3) Read charachters from PTY output ```c# SafeFileHandle inputRead, inputWrite, outputRead, outputWrite; CreatePseudoConsole(new COORD { X = 40, Y = 20 }, inputRead, outputWrite, 0, out IntPtr hPC); ProcessFactory.Start("powershell.exe -File a.ps1", 0x00020016, hPC); var reader = new StreamReader(new FileStream(outputRead, FileAccess.Read)); // Read output var bytesRead = reader.ReadBlock(buf, 0, 140); var output = new string(buf.Take(bytesRead).ToArray()); Console.Out.Write(output); ``` ### Expected Behavior The entries in the output are in the same order, in which they were written `Begin←[1qIN_BETWEEN_1←[1Q←[2qIN_BETWEEN_2←[2Q←[3qIN_BETWEEN_3←[3Q` ### Actual Behavior CSI sequences are printed before the text output `←[1q←[1Q←[2q←[2Q←[3q←[3QBeginIN_BETWEEN_1IN_BETWEEN_2IN_BETWEEN_3` I think it happens [here](https://github.com/microsoft/terminal/blob/62448969b30325a75f35201427a052de0e4dbb7d/src/terminal/parser/OutputStateMachineEngine.cpp#L732) Don't you think that it's betteer to flush text buffer before flushing escape sequences? And can you suggest some workaround? Can I flush text buffer manually from script? Thanks
claunia added the Issue-BugResolution-Duplicate labels 2026-01-31 06:52:46 +00:00
claunia commented 2026-01-31 06:52:47 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Apr 24, 2023):

This looks a lot like what I was trying to fix over in #13710

@zadjii-msft commented on GitHub (Apr 24, 2023): This looks a _lot_ like what I was trying to fix over in #13710
claunia commented 2026-01-31 06:52:47 +00:00
Author
Owner
Copy Link

@Mervap commented on GitHub (Apr 24, 2023):

@zadjii-msft seems https://github.com/microsoft/terminal/issues/13710 relates to specific escape sequences, but my issue relates to any unknown (or not implemented) sequence

Let me please ask the question again: can I force buffer flushing somehow? To be sure that my output is synchronized. Probably it would be enough to solve my issue

@Mervap commented on GitHub (Apr 24, 2023): @zadjii-msft seems https://github.com/microsoft/terminal/issues/13710 relates to specific escape sequences, but my issue relates to any unknown (or not implemented) sequence Let me please ask the question again: can I force buffer flushing somehow? To be sure that my output is synchronized. Probably it would be enough to solve my issue
claunia commented 2026-01-31 06:52:47 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Apr 25, 2023):

Ah, sorry, I linked the issue but not the PR. #14677 should fix this by flushing the frame when we encounter an unexpected sequence, then write the sequence, then start a new frame and continue. I just need to do the diligence on my part to re-check out that branch and double check your repro is fixed too.

There was also another thread that felt similar. #13462? Hey maybe I can close that out, since #14677 feels better to me

@zadjii-msft commented on GitHub (Apr 25, 2023): Ah, sorry, I linked the issue but not the PR. #14677 should fix this by flushing the frame when we encounter an unexpected sequence, then write the sequence, then start a new frame and continue. I just need to do the diligence on my part to re-check out that branch and double check your repro is fixed too. There was also another thread that felt similar. #13462? Hey maybe I can close that out, since #14677 _feels_ better to me
claunia commented 2026-01-31 06:52:48 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Apr 25, 2023):

FYI, I've tested PR #14677 with my test case from issue #8698, and it hasn't done anything to fix the order of operations there. The only thing I can see different when looking at the debug tap is that the DECTCEM (cursor show/hide) sequence appears in a slightly different position.

That said, from a visual standpoint it seems to be an improvement (at least for that particular test case), since I'm not seeing the red flash anymore. I'm not sure whether that's just a matter of lucky timing though. I suspect the position of the DECTCEM may be triggering a redraw in the renderer at a more appropriate time.

And with the test case in this issue, the ordering just seems random, both with and without PR #14677 applied. That suggests it's just dependent on when the render thread happens to trigger a flush.

@j4james commented on GitHub (Apr 25, 2023): FYI, I've tested PR #14677 with my test case from issue #8698, and it hasn't done anything to fix the order of operations there. The only thing I can see different when looking at the debug tap is that the `DECTCEM` (cursor show/hide) sequence appears in a slightly different position. That said, from a visual standpoint it seems to be an improvement (at least for that particular test case), since I'm not seeing the red flash anymore. I'm not sure whether that's just a matter of lucky timing though. I suspect the position of the `DECTCEM` may be triggering a redraw in the renderer at a more appropriate time. And with the test case in this issue, the ordering just seems random, both with and without PR #14677 applied. That suggests it's just dependent on when the render thread happens to trigger a flush.
claunia commented 2026-01-31 06:52:48 +00:00
Author
Owner
Copy Link

@Mervap commented on GitHub (Apr 25, 2023):

You better know whether to close the issue or not 🙂
I just wanted to achieve 2 goals:

  1. Notify about the problem
  2. Find out if there is a workaround right now because I can't wait for the fix (and it's not a fact that it will be in <22000 at all)

Since you have not answered the 2nd question, I dare to assume that there is no workaround. Can you please confirm this?

@Mervap commented on GitHub (Apr 25, 2023): You better know whether to close the issue or not 🙂 I just wanted to achieve 2 goals: 1) Notify about the problem 2) Find out if there is a workaround right now because I can't wait for the fix (and it's not a fact that it will be in <22000 at all) Since you have not answered the 2nd question, I dare to assume that there is no workaround. Can you please confirm this?
claunia commented 2026-01-31 06:52:48 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Apr 25, 2023):

@Mervap I'm not personally aware of any workaround. It's also worth mentioning that passing through unknown sequences will potentially end up failing anyway, even if they were in the correct order. It depends on what your sequences are intended to do, but anything that modifies the buffer is at risk of being overwritten by conpty at a later point in time.

@j4james commented on GitHub (Apr 25, 2023): @Mervap I'm not personally aware of any workaround. It's also worth mentioning that passing through unknown sequences will potentially end up failing anyway, even if they were in the correct order. It depends on what your sequences are intended to do, but anything that modifies the buffer is at risk of being overwritten by conpty at a later point in time.
claunia commented 2026-01-31 06:52:49 +00:00
Author
Owner
Copy Link

@Mervap commented on GitHub (Apr 25, 2023):

I understand that unknown sequences may behave really differently
In my case, it would be enough to force text buffer flushing before the escape sequence emitting (on pwsh script side)
Like

[Console]::Flush()
[Console]::Write("`e[1q")
@Mervap commented on GitHub (Apr 25, 2023): I understand that unknown sequences may behave really differently In my case, it would be enough to force text buffer flushing before the escape sequence emitting (on pwsh script side) Like ```pwsh [Console]::Flush() [Console]::Write("`e[1q") ```
claunia commented 2026-01-31 06:52:49 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Apr 25, 2023):

Okay yea, after caching this whole space back in, I'm gonna call this a /dupe of #8698. That's the more correct root issue here. Thanks again @j4james for the dilligence teasing them apart.

Right now, I'm not sure there's anything you could do to flush it manually on your side. maybe doing a `e[?1049h`e[?1049l to go into and out of the alt buffer, which would technically force a pair of frames, but also definitely horribly flicker. Yea definitely don't do that.

If you're building something on top of ConPTY, I might recommend upvoting #15065. Thats the thread we're using for "allow folks to use the most up-to-date conpty without relying on the OS version".

@zadjii-msft commented on GitHub (Apr 25, 2023): Okay yea, after caching this whole space back in, I'm gonna call this a /dupe of #8698. That's the more correct root issue here. Thanks again @j4james for the dilligence teasing them apart. Right now, I'm not sure there's anything you could do to flush it manually on your side. ~_maybe_ doing a ``` `e[?1049h`e[?1049l ``` to go into and out of the alt buffer, which would technically force a pair of frames, but also definitely horribly flicker.~ Yea definitely don't do that. <hr> If you're building something on top of ConPTY, I might recommend upvoting #15065. Thats the thread we're using for "allow folks to use the most up-to-date conpty without relying on the OS version".
claunia commented 2026-01-31 06:52:49 +00:00
Author
Owner
Copy Link

@microsoft-github-policy-service[bot] commented on GitHub (Apr 25, 2023):

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

@microsoft-github-policy-service[bot] commented on GitHub (Apr 25, 2023): Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label Issue-Bug Resolution-Duplicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#19758
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?