starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

Add SGR-Pixels (mouse mode 1016) #22942

New Issue
Open
opened 2026-01-31 08:28:01 +00:00 by claunia · 5 comments
claunia commented 2026-01-31 08:28:01 +00:00
Owner
Copy Link

Originally created by @salt-die on GitHub (Feb 19, 2025).

Description of the new feature

Now that sixel is implemented, SGR-Pixels mouse mode would be a nice feature to have. Also helps to implement features such as, e.g., smooth scrolling.

I've searched through issues and only found two references to this feature (below), so hopefully this isn't a duplicate:

https://github.com/microsoft/terminal/issues/448#issuecomment-1010263526 with response from @j4james :

I wasn't considering 1016, but I do have a POC of the DEC locator mode. There's no point in doing either of them until we have sixel, though, because the pixel coordinates will need to be tightly coupled to the sixel resolution.

And listed as not supported here https://github.com/microsoft/terminal/issues/10531#issuecomment-872398721 from @DHowett

Would prefer 1016 over DEC Locator since it seems to be more widely used?
https://github.com/dankamongmen/notcurses/issues/2326
https://github.com/wezterm/wezterm/issues/1457
16d27b6f41/README.md (L26C1-L26C70)
https://github.com/Aetf/libtsm/pull/29
https://github.com/xtermjs/xterm.js/issues/4092

Proposed technical implementation details

No response

Originally created by @salt-die on GitHub (Feb 19, 2025). ### Description of the new feature Now that sixel is implemented, [SGR-Pixels](https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#:~:text=improvement%20over%201006.-,sgr-pixels%20(1016),-Use%20the%20same) mouse mode would be a nice feature to have. Also helps to implement features such as, e.g., [smooth scrolling](https://textual.textualize.io/blog/2025/02/16/smoother-scrolling-in-the-terminal-mdash-a-feature-decades-in-the-making/). I've searched through issues and only found two references to this feature (below), so hopefully this isn't a duplicate: https://github.com/microsoft/terminal/issues/448#issuecomment-1010263526 with response from @j4james : > I wasn't considering 1016, but I do have a POC of the DEC locator mode. There's no point in doing either of them until we have sixel, though, because the pixel coordinates will need to be tightly coupled to the sixel resolution. And listed as not supported here https://github.com/microsoft/terminal/issues/10531#issuecomment-872398721 from @DHowett Would prefer 1016 over DEC Locator since it seems to be more widely used? https://github.com/dankamongmen/notcurses/issues/2326 https://github.com/wezterm/wezterm/issues/1457 https://github.com/mahmoudimus/jexer/blob/16d27b6f4182e3c2cfecd7673934ebe09782861a/README.md?plain=1#L26C1-L26C70 https://github.com/Aetf/libtsm/pull/29 https://github.com/xtermjs/xterm.js/issues/4092 ### Proposed technical implementation details _No response_
claunia added the Issue-FeatureHelp WantedArea-InputArea-VTProduct-Terminal labels 2026-01-31 08:28:02 +00:00
claunia commented 2026-01-31 08:28:03 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Feb 20, 2025):

Also helps to implement features such as, e.g., smooth scrolling.

That's neat, but it's disappointing that they felt the need to resort to proprietary sequences. Here's a similar demo running in OpenConsole using DEC sequences from the 1980's. They could have achieved this 30+ years ago if terminal devs weren't constantly reinventing the wheel.

Smooth scrolling demonstration

This is using my POC of DEC Locator mode btw. I never got around to turning it into a PR, because it required significant rewrites to the mouse handling, and I suspect there are things that got broken in the process. Implementing mode 1016 will almost certainly have the same problem.

But I'm not opposed to mode 1016. I just prefer the DEC Locator mode because it's more powerful, and has been around for 30 more years.

@j4james commented on GitHub (Feb 20, 2025): > Also helps to implement features such as, e.g., [smooth scrolling](https://textual.textualize.io/blog/2025/02/16/smoother-scrolling-in-the-terminal-mdash-a-feature-decades-in-the-making/). That's neat, but it's disappointing that they felt the need to resort to proprietary sequences. Here's a similar demo running in OpenConsole using DEC sequences from the 1980's. They could have achieved this 30+ years ago if terminal devs weren't constantly reinventing the wheel. ![Smooth scrolling demonstration](https://github.com/user-attachments/assets/a8e5526b-757f-4ee0-a300-f4f7fc4acf75) This is using my POC of DEC Locator mode btw. I never got around to turning it into a PR, because it required significant rewrites to the mouse handling, and I suspect there are things that got broken in the process. Implementing mode 1016 will almost certainly have the same problem. But I'm not opposed to mode 1016. I just prefer the DEC Locator mode because it's more powerful, and has been around for 30 more years.
claunia commented 2026-01-31 08:28:03 +00:00
Author
Owner
Copy Link

@schrmh commented on GitHub (Mar 3, 2025):

@j4james Interesting, that's the first time I heard that it even is more powerful. I was told SGR is more useful (https://github.com/wezterm/wezterm/issues/217#issuecomment-1156710458) but given your comment I would rather lean towards supporting the DEC way as well (and then just trying to do positive propaganda for implementing that or PRs. I think the SGR is not really used by many applications for mouse location tracking anyways since I did not stumple upon other than the mention stuff so far even when actively looking for it mid 2022).
Btw. which terminal emulator (that would need to run on GNU/Linux since I don't use Windows, sorry) would you recommend specifically for trying out stuff with DEC Locator mode? E.g. some are a bit glitchy or require a full-screened terminal. Ideally I would like to have a SIXEL-using application to respond to mouse clicks at certain locations (e.g. https://github.com/schrmh/pdfgrepSIXEL to do something with the clicked on slide)...

@schrmh commented on GitHub (Mar 3, 2025): @j4james Interesting, that's the first time I heard that it even is more powerful. I was told SGR is more useful (https://github.com/wezterm/wezterm/issues/217#issuecomment-1156710458) but given your comment I would rather lean towards supporting the DEC way as well (and then just trying to do positive propaganda for implementing that or PRs. I think the SGR is not really used by many applications for mouse location tracking anyways since I did not stumple upon other than the mention stuff so far even when actively looking for it mid 2022). Btw. which terminal emulator (that would need to run on GNU/Linux since I don't use Windows, sorry) would you recommend specifically for trying out stuff with DEC Locator mode? E.g. some are a bit glitchy or require a full-screened terminal. Ideally I would like to have a SIXEL-using application to respond to mouse clicks at certain locations (e.g. https://github.com/schrmh/pdfgrepSIXEL to do something with the clicked on slide)...
claunia commented 2026-01-31 08:28:03 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Mar 3, 2025):

@schrmh It's been a few years since I last tested, but the only Linux terminals I'm aware of that have DEC locator support are mlterm and xterm, and it's possible you may have had to compile xterm with a specific build flag (--enable-dec-locator) to get that functionality.

Neither implementation was perfect (and I think the DEC documentation itself had errors), but if you just need basic click reporting it shouldn't be too complicated. However, I'd recommend testing with more than one terminal, just to make sure you aren't relying on behavior specific to one implementation.

@j4james commented on GitHub (Mar 3, 2025): @schrmh It's been a few years since I last tested, but the only Linux terminals I'm aware of that have DEC locator support are mlterm and xterm, and it's possible you may have had to compile xterm with a specific build flag (`--enable-dec-locator`) to get that functionality. Neither implementation was perfect (and I think the DEC documentation itself had errors), but if you just need basic click reporting it shouldn't be too complicated. However, I'd recommend testing with more than one terminal, just to make sure you aren't relying on behavior specific to one implementation.
claunia commented 2026-01-31 08:28:04 +00:00
Author
Owner
Copy Link

@schrmh commented on GitHub (Mar 4, 2025):

Yeah, sadly saitoha's drawing example did not work in mlterm at all when I tried it years ago. The full-screen thing I mentioned was with xterm regarding that example.

@schrmh commented on GitHub (Mar 4, 2025): Yeah, sadly saitoha's drawing example did not work in mlterm at all when I tried it years ago. The full-screen thing I mentioned was with xterm regarding that example.
claunia commented 2026-01-31 08:28:05 +00:00
Author
Owner
Copy Link

@j4james commented on GitHub (Mar 4, 2025):

saitoha's drawing example did not work in mlterm at all when I tried it years ago.

Yeah, I've just looked at my notes on mlterm, and it seems that it doesn't (or at least didn't) handle DECRQLP requests when a mouse button is held down. And saitoha's drawing example is entirely dependent on DECRQLP polling, so I can see why that wouldn't work. But if you just need to detect click events, I think mlterm should be fine.

@j4james commented on GitHub (Mar 4, 2025): > saitoha's drawing example did not work in mlterm at all when I tried it years ago. Yeah, I've just looked at my notes on mlterm, and it seems that it doesn't (or at least didn't) handle `DECRQLP` requests when a mouse button is held down. And saitoha's drawing example is entirely dependent on `DECRQLP` polling, so I can see why that wouldn't work. But if you just need to detect click events, I think mlterm should be fine.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label Area-Input Area-VT Help Wanted Issue-Feature Product-Terminal
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#22942
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?