starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

[PR #17510] A minor ConPTY refactoring: Goodbye VtEngine Edition #31251

New Issue
Open
opened 2026-01-31 09:46:05 +00:00 by claunia · 0 comments
claunia commented 2026-01-31 09:46:05 +00:00
Owner
Copy Link

Original Pull Request: https://github.com/microsoft/terminal/pull/17510

State: closed
Merged: Yes

The idea is that we can translate Console API calls directly to VT at
least as well as the current VtEngine setup can. For instance, a call
to SetConsoleCursorPosition clearly translates directly to a CUP
escape sequence. Effectively, instead of translating output
asynchronously in the renderer thread, we'll do it synchronously
right during the Console API call.

Most importantly, the this means that any VT output that an
application generates will now be given to the terminal unmodified.

Aside from reducing our project's complexity quite a bit and opening
the path towards various interesting work like sixels, Device Control
Strings, buffer snapshotting, synchronized updates, and more, it also
improves performance for mixed text output like enwik8.txt in conhost
to 1.3-2x and in Windows Terminal via ConPTY to roughly 20x.

This adds support for overlapped IO, because now that output cannot
be "skipped" anymore (VtEngine worked like a renderer after all)
it's become crucial to block conhost as little as possible.

⚠️ Intentionally unresolved changes/quirks:

  • To force a delayed EOL wrap to wrap, WriteCharsLegacy emits a
    \r\n if necessary. This breaks text reflow on window resize.
    We cannot emit \r the way readline does it, because this would
    overwrite the first column in the next row with a whitespace.
    The alternative is to read back the affected cell from the buffer
    and emit that character and its attributes followed by a \r.
    I chose to not do that, because buffer read-back is lossy (= UCS2).
    Unless the window is resized, the difference is unnoticeable
    and historically, conhost had no support for buffer reflow anyway.
  • If ENABLE_VIRTUAL_TERMINAL_PROCESSING is set while
    DISABLE_NEWLINE_AUTO_RETURN is reset, we'll blindly replace all
    LF with CRLF. This may hypothetically break DCS sequences, but it's
    the only way to do this without parsing the given VT string and
    thus the only way we can achieve passthrough mode in the future.
  • ENABLE_WRAP_AT_EOL_OUTPUT is translated to DECAWM.
    Between Windows XP and Windows 11 21H2, ENABLE_WRAP_AT_EOL_OUTPUT
    being reset would cause the cursor position to reset to wherever
    a write started, if the write, including expanded control chars,
    was less than 100 characters long. If it was longer than that,
    the cursor position would end up in an effectively random position.
    After lengthy research I believe that this is a bug introduced in
    Windows XP and that the original intention was for this mode to be
    equivalent to DECAWM. This is compounded by MSDN's description
    (emphasis mine):

    If this mode is disabled, the last character in the row is
    overwritten with any subsequent characters.

⚠️ Unresolved issues/quirks:

  • Focus/Unfocus events are injected into the output stream without
    checking whether the VT output is currently in a ground state.
    This may break whatever VT sequence is currently ongoing.
    This is an existing issue.
  • VtIo::Writer::WriteInfos should properly verify the width of
    each individual character.
  • Using SetConsoleActiveScreenBuffer destroys surrogate pairs
    and extended (VT) attributes. It could be translated to VT pages
    in the long term.
  • Similarly, ScrollConsoleScreenBuffer results in the same and
    could be translated to DECCRA and DECFRA in the near term.
    This is important because otherwise vim output may loose
    its extended attributes during scrolling.
  • Reflowing a long line until it wraps results in the cooked read
    prompt to be misaligned vertically.
  • SCREEN_INFORMATION::s_RemoveScreenBuffer should trigger a
    buffer switch similar to SetConsoleActiveScreenBuffer.
  • Translation of COMMON_LVB_GRID_HORIZONTAL to SGR 53 was dropped
    and may be reintroduced alongside UNDERSCORE = SGR 4.
  • Move the OSC 0 ; P t BEL sequence to WriteWindowTitle
    and swap the BEL with the ST (ESC \).
  • PowerShell on Windows 10 ships with PSReadLine 2.0.0-beta2
    which emits SGR 37/40 instead of 39/49. This results in black
    spaces when typing and there's no good way to fix that.
  • A test is missing that ensures that FillConsoleOutputCharacterW
    results in a CSI n J during the PowerShell shim.
  • A test is missing that ensures that PtySignal::ClearBuffer
    does not result in any VT being generated.

Closes #262
Closes #1173
Closes #3016
Closes #4129
Closes #5228
Closes #8698
Closes #12336
Closes #15014
Closes #15888
Closes #16461
Closes #16911
Closes #17151
Closes #17313

**Original Pull Request:** https://github.com/microsoft/terminal/pull/17510 **State:** closed **Merged:** Yes --- The idea is that we can translate Console API calls directly to VT at least as well as the current VtEngine setup can. For instance, a call to `SetConsoleCursorPosition` clearly translates directly to a `CUP` escape sequence. Effectively, instead of translating output asynchronously in the renderer thread, we'll do it synchronously right during the Console API call. Most importantly, the this means that any VT output that an application generates will now be given to the terminal unmodified. Aside from reducing our project's complexity quite a bit and opening the path towards various interesting work like sixels, Device Control Strings, buffer snapshotting, synchronized updates, and more, it also improves performance for mixed text output like enwik8.txt in conhost to 1.3-2x and in Windows Terminal via ConPTY to roughly 20x. This adds support for overlapped IO, because now that output cannot be "skipped" anymore (VtEngine worked like a renderer after all) it's become crucial to block conhost as little as possible. ⚠️ Intentionally unresolved changes/quirks: * To force a delayed EOL wrap to wrap, `WriteCharsLegacy` emits a `\r\n` if necessary. This breaks text reflow on window resize. We cannot emit ` \r` the way readline does it, because this would overwrite the first column in the next row with a whitespace. The alternative is to read back the affected cell from the buffer and emit that character and its attributes followed by a `\r`. I chose to not do that, because buffer read-back is lossy (= UCS2). Unless the window is resized, the difference is unnoticeable and historically, conhost had no support for buffer reflow anyway. * If `ENABLE_VIRTUAL_TERMINAL_PROCESSING` is set while `DISABLE_NEWLINE_AUTO_RETURN` is reset, we'll blindly replace all LF with CRLF. This may hypothetically break DCS sequences, but it's the only way to do this without parsing the given VT string and thus the only way we can achieve passthrough mode in the future. * `ENABLE_WRAP_AT_EOL_OUTPUT` is translated to `DECAWM`. Between Windows XP and Windows 11 21H2, `ENABLE_WRAP_AT_EOL_OUTPUT` being reset would cause the cursor position to reset to wherever a write started, _if_ the write, including expanded control chars, was less than 100 characters long. If it was longer than that, the cursor position would end up in an effectively random position. After lengthy research I believe that this is a bug introduced in Windows XP and that the original intention was for this mode to be equivalent to `DECAWM`. This is compounded by MSDN's description (emphasis mine): > If this mode is disabled, the **last character** in the row is > overwritten with any subsequent characters. ⚠️ Unresolved issues/quirks: * Focus/Unfocus events are injected into the output stream without checking whether the VT output is currently in a ground state. This may break whatever VT sequence is currently ongoing. This is an existing issue. * `VtIo::Writer::WriteInfos` should properly verify the width of each individual character. * Using `SetConsoleActiveScreenBuffer` destroys surrogate pairs and extended (VT) attributes. It could be translated to VT pages in the long term. * Similarly, `ScrollConsoleScreenBuffer` results in the same and could be translated to `DECCRA` and `DECFRA` in the near term. This is important because otherwise `vim` output may loose its extended attributes during scrolling. * Reflowing a long line until it wraps results in the cooked read prompt to be misaligned vertically. * `SCREEN_INFORMATION::s_RemoveScreenBuffer` should trigger a buffer switch similar to `SetConsoleActiveScreenBuffer`. * Translation of `COMMON_LVB_GRID_HORIZONTAL` to `SGR 53` was dropped and may be reintroduced alongside `UNDERSCORE` = `SGR 4`. * Move the `OSC 0 ; P t BEL` sequence to `WriteWindowTitle` and swap the `BEL` with the `ST` (`ESC \`). * PowerShell on Windows 10 ships with PSReadLine 2.0.0-beta2 which emits SGR 37/40 instead of 39/49. This results in black spaces when typing and there's no good way to fix that. * A test is missing that ensures that `FillConsoleOutputCharacterW` results in a `CSI n J` during the PowerShell shim. * A test is missing that ensures that `PtySignal::ClearBuffer` does not result in any VT being generated. Closes #262 Closes #1173 Closes #3016 Closes #4129 Closes #5228 Closes #8698 Closes #12336 Closes #15014 Closes #15888 Closes #16461 Closes #16911 Closes #17151 Closes #17313
claunia added the pull-request label 2026-01-31 09:46:05 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#31251
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?