starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

Cursor flicker in Cygwin irssi #22610

New Issue
Closed
opened 2026-01-31 08:18:24 +00:00 by claunia · 18 comments
claunia commented 2026-01-31 08:18:24 +00:00
Owner
Copy Link

Originally created by @pragma- on GitHub (Nov 30, 2024).

Originally assigned to: @DHowett on GitHub.

Windows Terminal version

1.22.3232.0

Windows build number

10.0.19045.0

Other Software

Cygwin

Steps to reproduce

  1. Open Windows Terminal
  2. Open Task Manager with Page Faults displayed (Details tab, right-click columns, select Page fault delta column)
  3. Note zero page faults in OpenConsole.exe
  4. Run Cygwin.bat
  5. Note 800-2000 page faults per second in OpenConsole.exe
  6. Exit Cygwin
  7. Note page faults returns to 0 in OpenConsole.exe

Expected Behavior

Page faults to remain at 0 or low in OpenConsole.exe while Cygwin is running.

Actual Behavior

800-2000 page faults per second in OpenConsole.exe while Cygwin is running. See video below:

https://github.com/user-attachments/assets/81b4fe68-a71b-4c96-95d8-e093a2d8f494

Originally created by @pragma- on GitHub (Nov 30, 2024). Originally assigned to: @DHowett on GitHub. ### Windows Terminal version 1.22.3232.0 ### Windows build number 10.0.19045.0 ### Other Software Cygwin ### Steps to reproduce 1. Open Windows Terminal 2. Open Task Manager with Page Faults displayed (Details tab, right-click columns, select Page fault delta column) 3. Note zero page faults in OpenConsole.exe 4. Run Cygwin.bat 5. Note 800-2000 page faults per second in OpenConsole.exe 6. Exit Cygwin 7. Note page faults returns to 0 in OpenConsole.exe ### Expected Behavior Page faults to remain at 0 or low in OpenConsole.exe while Cygwin is running. ### Actual Behavior 800-2000 page faults per second in OpenConsole.exe while Cygwin is running. See video below: https://github.com/user-attachments/assets/81b4fe68-a71b-4c96-95d8-e093a2d8f494
claunia added the Issue-BugResolution-Duplicate labels 2026-01-31 08:18:24 +00:00
claunia commented 2026-01-31 08:18:25 +00:00
Author
Owner
Copy Link

@pragma- commented on GitHub (Nov 30, 2024):

Page faults are 0 when starting Cygwin in either Windows' cmd.exe or Cygwin's mintty.exe.

@pragma- commented on GitHub (Nov 30, 2024): Page faults are 0 when starting Cygwin in either Windows' cmd.exe or Cygwin's mintty.exe.
claunia commented 2026-01-31 08:18:26 +00:00
Author
Owner
Copy Link

@DHowett commented on GitHub (Dec 2, 2024):

Cygwin is calling console APIs in its steady state.

It looks like it's calling GetConsoleProcessList in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface.

I don't think there's anything we can do about that, other than stopping them from doing so

Is there an issue you're troubleshooting here other than a raw increased count of page faults/?

@DHowett commented on GitHub (Dec 2, 2024): Cygwin is calling console APIs in its steady state. It looks like it's calling `GetConsoleProcessList` in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface. I don't think there's anything we can do about that, other than stopping them from doing so Is there an issue you're troubleshooting here other than a raw increased count of page faults/?
claunia commented 2026-01-31 08:18:26 +00:00
Author
Owner
Copy Link

@pragma- commented on GitHub (Dec 2, 2024):

I am experiencing stutters and visible cursor movement during terminal screen redraws that I do not experience when using cmd.exe or mintty.exe. I will attempt to bring this issue to the Cygwin team's awareness.

@pragma- commented on GitHub (Dec 2, 2024): I am experiencing stutters and visible cursor movement during terminal screen redraws that I do not experience when using cmd.exe or mintty.exe. I will attempt to bring this issue to the Cygwin team's awareness.
claunia commented 2026-01-31 08:18:26 +00:00
Author
Owner
Copy Link

@DHowett commented on GitHub (Dec 2, 2024):

during terminal screen redraws

You're experiencing a graphical issue using the new Windows Terminal which uses the GPU to render things.

I doubt your issue has anything to do with the console server and its soft page faults. 🙂

@lhecker What are our usual troubleshooting steps for things like this?

@DHowett commented on GitHub (Dec 2, 2024): > during terminal screen redraws You're experiencing a graphical issue using the new Windows Terminal which uses the GPU to render things. I doubt your issue has anything to do with the console server and its soft page faults. 🙂 @lhecker What are our usual troubleshooting steps for things like this?
claunia commented 2026-01-31 08:18:27 +00:00
Author
Owner
Copy Link

@BrianInglis commented on GitHub (Dec 2, 2024):

The only page thrasher of that magnitude on my system at the moment appears to be MS Windows Defender! ;^>
I've used Cygwin for decades and never used or even looked at Cygwin.bat which seems to be causing these problems.
That command script Cygwin.bat seems just to cd to Cygwin "root" dir using MS Windows path, and last line runs bash as an interactive login shell, so what is the difference from running under cmd?
I expect bash threads to hang for console input and subprocess termination, or run shell functions or builtin commands, and I see 3-6 (Xwindows is 6) for each instance at the moment.

@BrianInglis commented on GitHub (Dec 2, 2024): The only page thrasher of that magnitude on my system at the moment appears to be *MS Windows Defender*! ;^> I've used *Cygwin* for decades and never used or even looked at `Cygwin.bat` which seems to be causing these problems. That command script `Cygwin.bat` seems just to `cd` to *Cygwin* "root" dir using *MS Windows* path, and last line runs `bash` as an interactive login shell, so what is the difference from running under `cmd`? I expect `bash` threads to hang for console input and subprocess termination, or run shell functions or builtin commands, and I see 3-6 (Xwindows is 6) for each instance at the moment.
claunia commented 2026-01-31 08:18:27 +00:00
Author
Owner
Copy Link

@lhecker commented on GitHub (Dec 2, 2024):

It looks like it's calling GetConsoleProcessList in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface.

It would be great if there was a different solution for that. Waiting on a handle seems better than polling to me.

@lhecker What are our usual troubleshooting steps for things like this?

I think a video of the issue would be a good start. Ideally a screen recording. If a video doesn't clear up what exactly is lagging and how and when, I'd need a significantly more detailed description of the issue.

@lhecker commented on GitHub (Dec 2, 2024): > It looks like it's calling `GetConsoleProcessList` in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface. It would be great if there was a different solution for that. Waiting on a handle seems better than polling to me. > @lhecker What are our usual troubleshooting steps for things like this? I think a video of the issue would be a good start. Ideally a screen recording. If a video doesn't clear up what exactly is lagging and how and when, I'd need a significantly more detailed description of the issue.
claunia commented 2026-01-31 08:18:27 +00:00
Author
Owner
Copy Link

@pragma- commented on GitHub (Dec 2, 2024):

Here is a video of the cursor flickering. On the left side is Windows Terminal without Cygwin. On the right side is Windows Terminal with Cygwin. Both terminals are running a SSH session into my Linux server where I have reattached to a TMUX session that is running WeeChat, an IRC client. Please note the annoyingly visible cursor flickering between the clock and the input location in the right-side Terminal that is running Cygwin. This cursor flickering is not present when Cygwin is running in cmd.exe or mintty.exe.

https://github.com/user-attachments/assets/f2e42c08-0066-47a8-abe5-115cfb630866

@pragma- commented on GitHub (Dec 2, 2024): Here is a video of the cursor flickering. On the left side is Windows Terminal without Cygwin. On the right side is Windows Terminal with Cygwin. Both terminals are running a SSH session into my Linux server where I have reattached to a TMUX session that is running WeeChat, an IRC client. Please note the annoyingly visible cursor flickering between the clock and the input location in the right-side Terminal that is running Cygwin. This cursor flickering is not present when Cygwin is running in cmd.exe or mintty.exe. https://github.com/user-attachments/assets/f2e42c08-0066-47a8-abe5-115cfb630866
claunia commented 2026-01-31 08:18:28 +00:00
Author
Owner
Copy Link

@lhecker commented on GitHub (Dec 3, 2024):

The cursor flickering issue is over here: #6217
I have different theories on why this may be happening. The most likely one at this time is that applications simply aren't controlling their stdout very well. If they don't atomically redraw the entire UI in a single stdout call, then we can randomly render the terminal contents in between two frames.

In the past this was not obvious because after we saw a write we would wait 16ms until we flush our internal buffers. That's also the reason why wezterm regressed: It updated to a newer ConPTY version where that 16ms was rightfully removed.

One solution is to add back a slight delay, but I would prefer solving the issue in another way.

@lhecker commented on GitHub (Dec 3, 2024): The cursor flickering issue is over here: #6217 I have different theories on why this may be happening. The most likely one at this time is that applications simply aren't controlling their stdout very well. If they don't atomically redraw the entire UI in a single stdout call, then we can randomly render the terminal contents in between two frames. In the past this was not obvious because after we saw a write we would wait 16ms until we flush our internal buffers. That's also the reason why wezterm regressed: It updated to a newer ConPTY version where that 16ms was rightfully removed. One solution is to add back a slight delay, but I would prefer solving the issue in another way.
claunia commented 2026-01-31 08:18:28 +00:00
Author
Owner
Copy Link

@tyan0 commented on GitHub (Dec 3, 2024):

Cygwin is calling console APIs in its steady state.

It looks like it's calling GetConsoleProcessList in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface.

I don't think there's anything we can do about that, other than stopping them from doing so

Do you mean we cannot call Windows API (GetConsoleProcessList()) frequently? Then, how often can we call it?
Since this issue does not happen with Windows Terminal 1.21.3231.0, it sounds like a regression of Windows Terminal for me.

I also wonder why "the allocation of a new buffer" is necessary. The buffer LPDWORD lpdwProcessList is provided by caller side.

@tyan0 commented on GitHub (Dec 3, 2024): > Cygwin is calling console APIs in its steady state. > > It looks like it's calling `GetConsoleProcessList` in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface. > > I don't think there's anything we can do about that, other than stopping them from doing so Do you mean we cannot call Windows API (GetConsoleProcessList()) frequently? Then, how often can we call it? Since this issue does not happen with Windows Terminal 1.21.3231.0, it sounds like a regression of Windows Terminal for me. I also wonder why "the allocation of a new buffer" is necessary. The buffer LPDWORD lpdwProcessList is provided by caller side.
claunia commented 2026-01-31 08:18:28 +00:00
Author
Owner
Copy Link

@tyan0 commented on GitHub (Dec 3, 2024):

It looks like it's calling GetConsoleProcessList in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface.

I doubt that this is the real cause.
The following code does not cause the issue even with version 1.22.3232.0

#include <windows.h>

int main()
{
	DWORD buf[65536];
	for (;;) GetConsoleProcessList(buf, 65536);
	return 0;
}
@tyan0 commented on GitHub (Dec 3, 2024): > It looks like it's calling `GetConsoleProcessList` in a tight loop, which results in the allocation of a new buffer that is returned to their process via condrv's ioctl interface. I doubt that this is the real cause. The following code does not cause the issue even with version 1.22.3232.0 ``` #include <windows.h> int main() { DWORD buf[65536]; for (;;) GetConsoleProcessList(buf, 65536); return 0; } ````
claunia commented 2026-01-31 08:18:29 +00:00
Author
Owner
Copy Link

@DHowett commented on GitHub (Dec 3, 2024):

The buffer LPDWORD lpdwProcessList is provided by caller side.

It's provided by the caller in another process. The console host does not share memory with or map memory from the caller.

FWIW, GetConsoleProcessList is the only thing Cygwin is doing repeatedly when idle. It turns out that tracking page faults is difficult (? or impossible?) without a kernel debugger. Thanks for testing.

Regardless, neither page faults nor this API are the reason @pragma-'s cursor is jumping between irssi's status line (updated every second) and their input line. That's just because we render the intermediate cursor position. The cursor moves to the status line when irssi overwrites it, because that's how overwriting a region of text works.

Usually, it's hidden when it does that. That's almost certainly the actual root cause: somebody isn't hiding it (whether it's irssi, Cygwin's emulated TTY, or us).

@pragma-, can you echo $TERM in the working and broken sessions?

(thanks for sharing the video!)

@DHowett commented on GitHub (Dec 3, 2024): > The buffer LPDWORD lpdwProcessList is provided by caller side. It's provided by the caller _in another process_. The console host does not share memory with or map memory from the caller. FWIW, `GetConsoleProcessList` is the only thing Cygwin is doing repeatedly when idle. It turns out that tracking _page faults_ is difficult (? or impossible?) without a kernel debugger. Thanks for testing. Regardless, neither page faults _nor_ this API are the reason @pragma-'s cursor is jumping between irssi's status line (updated every second) and their input line. That's just because we render the intermediate cursor position. The cursor moves to the status line when irssi overwrites it, because that's how overwriting a region of text works. Usually, it's hidden when it does that. That's almost certainly the actual root cause: somebody isn't hiding it (whether it's irssi, Cygwin's emulated TTY, or us). @pragma-, can you `echo $TERM` in the working and broken sessions? (thanks for sharing the video!)
claunia commented 2026-01-31 08:18:29 +00:00
Author
Owner
Copy Link

@tyan0 commented on GitHub (Dec 3, 2024):

I could have reproduced the page fault with the following simple test case. Regardless of the frequency of API calls, 64 page-fault occurs every loop. Unlike cygwin case, the page fault happens also with version 1.21.3231.0.

#include <windows.h>

int main()
{
	DWORD buf[65536];
	INPUT_RECORD rec;
	DWORD n;
	for (;;) {
		GetConsoleProcessList(buf, 65536);
		Sleep(1000);
		PeekConsoleInput(GetStdHandle(STD_INPUT_HANDLE), &rec, 1, &n);
		Sleep(1000);
	}
	return 0;
}
@tyan0 commented on GitHub (Dec 3, 2024): I could have reproduced the page fault with the following simple test case. Regardless of the frequency of API calls, 64 page-fault occurs every loop. Unlike cygwin case, the page fault happens also with version 1.21.3231.0. ``` #include <windows.h> int main() { DWORD buf[65536]; INPUT_RECORD rec; DWORD n; for (;;) { GetConsoleProcessList(buf, 65536); Sleep(1000); PeekConsoleInput(GetStdHandle(STD_INPUT_HANDLE), &rec, 1, &n); Sleep(1000); } return 0; } ```
claunia commented 2026-01-31 08:18:29 +00:00
Author
Owner
Copy Link

@pragma- commented on GitHub (Dec 3, 2024):

@DHowett The $TERM is screen-256color in both terminals. By the way, it's weechat rather than irssi. I've customized my weechat theme to look like irssi! :-)

@pragma- commented on GitHub (Dec 3, 2024): @DHowett The $TERM is screen-256color in both terminals. By the way, it's weechat rather than irssi. I've customized my weechat theme to look like irssi! :-)
claunia commented 2026-01-31 08:18:29 +00:00
Author
Owner
Copy Link

@DHowett commented on GitHub (Dec 3, 2024):

customized

Oh nice!

screen

Oh, uh, what is TERM outside of screen in both cases?

@DHowett commented on GitHub (Dec 3, 2024): > customized Oh nice! > screen Oh, uh, what is `TERM` outside of screen in both cases?
claunia commented 2026-01-31 08:18:29 +00:00
Author
Owner
Copy Link

@pragma- commented on GitHub (Dec 3, 2024):

xterm-256color

@pragma- commented on GitHub (Dec 3, 2024): xterm-256color
claunia commented 2026-01-31 08:18:30 +00:00
Author
Owner
Copy Link

@lhecker commented on GitHub (Dec 3, 2024):

I figured out why this happens: When we read from the console pipe, we currently need to provide a buffer where the message is copied into, similar to ReadFile. In order to slightly amortize the cost of allocating a buffer we reuse the previous allocation unless the previous one was >16KiB and the next one is less than half the size. This avoids issues where a huge 100MB buffer never gets freed.
This however does not work well for this usage of the API, because the GetConsoleProcessList call needs 128KiB and the PeekConsoleInput call only needs 20 (?) bytes. This causes the buffer to be repeatedly freed and reallocated.

IMO there's 3 things we should do:

  • cygwin should avoid making "large" data requests if there's no reason for that. That DWORD buf[65536]; buffer is 128KiB large and in practice maybe 16 byte of that will ever be used. It could improve the code by only increasing the request size if the return value is equal to the buffer size.
  • We should increase the cutoff size from 16KiB to 128KiB, because that's the size of the cat chunk size anyway.
  • We should switch to a cheap allocator for processing console messages in the future, because malloc is a poor fit for it. But this doesn't fix the issue for large allocations, since we still need to be conservative about our resident set size. This means that even with a custom allocator, cygwin's usage pattern may still be suboptimal.
@lhecker commented on GitHub (Dec 3, 2024): I figured out why this happens: When we read from the console pipe, we currently need to provide a buffer where the message is copied into, similar to `ReadFile`. In order to slightly amortize the cost of allocating a buffer we reuse the previous allocation unless the previous one was >16KiB and the next one is less than half the size. This avoids issues where a huge 100MB buffer never gets freed. This however does not work well for this usage of the API, because the `GetConsoleProcessList` call needs 128KiB and the `PeekConsoleInput` call only needs 20 (?) bytes. This causes the buffer to be repeatedly freed and reallocated. IMO there's 3 things we should do: * cygwin should avoid making "large" data requests if there's no reason for that. That `DWORD buf[65536];` buffer is 128KiB large and in practice maybe 16 byte of that will ever be used. It could improve the code by only increasing the request size if the return value is equal to the buffer size. * We should increase the cutoff size from 16KiB to 128KiB, because that's the size of the `cat` chunk size anyway. * We should switch to a cheap allocator for processing console messages in the future, because malloc is a poor fit for it. But this doesn't fix the issue for large allocations, since we still need to be conservative about our resident set size. This means that even with a custom allocator, cygwin's usage pattern may still be suboptimal.
claunia commented 2026-01-31 08:18:30 +00:00
Author
Owner
Copy Link

@carlos-zamora commented on GitHub (Jan 13, 2025):

Now that the first two remediations are complete, we're going to mark this bug as a /dup of #6217. Thanks!

@carlos-zamora commented on GitHub (Jan 13, 2025): Now that the first two remediations are complete, we're going to mark this bug as a /dup of #6217. Thanks!
claunia commented 2026-01-31 08:18:30 +00:00
Author
Owner
Copy Link

@microsoft-github-policy-service[bot] commented on GitHub (Jan 13, 2025):

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

@microsoft-github-policy-service[bot] commented on GitHub (Jan 13, 2025): Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report! <!-- Policy app identification https://img.shields.io/static/v1?label=PullRequestIssueManagement. -->
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label Issue-Bug Resolution-Duplicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#22610
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?