Introduce abstraction layer between TermControl and TerminalCore (aka Interactivity) #9513

Closed
opened 2026-01-31 01:56:21 +00:00 by claunia · 3 comments
Owner

Originally created by @carlos-zamora on GitHub (Jul 9, 2020).

Originally assigned to: @zadjii-msft on GitHub.

Description of the new feature/enhancement

We have some redundant code between the WPF Control and the UWP TermControl. This indicates a missing layer of abstraction between the TerminalCore and the TermControl. Some benefits that we could/would get from introducing an "interactivity" layer includes...

  • better parity between both controls
  • less duplicated code
  • better separating scrolling in both layers

Proposed technical implementation details (optional)

Specific interactions that could be abstracted better include (but are not limited to)...

  • Selection
  • Scrolling
  • VT Mouse Mode (sorta)

\cc @DHowett

Originally created by @carlos-zamora on GitHub (Jul 9, 2020). Originally assigned to: @zadjii-msft on GitHub. # Description of the new feature/enhancement We have some redundant code between the WPF Control and the UWP TermControl. This indicates a missing layer of abstraction between the TerminalCore and the TermControl. Some benefits that we could/would get from introducing an "interactivity" layer includes... - better parity between both controls - less duplicated code - better separating scrolling in both layers # Proposed technical implementation details (optional) Specific interactions that could be abstracted better include (but are not limited to)... - Selection - Scrolling - VT Mouse Mode (sorta) \cc @DHowett
Author
Owner

@zadjii-msft commented on GitHub (Mar 31, 2021):

Alright, as of 2ac83180f, this is fundamentally pulled into 3 files, but there's still a lot of work to do. I'm using this comment to track that.

  • Move ControlCore and ControlInteractivity to Microsoft.Terminal.Core so I can unittest them
    • didn't exactly do that, because that was WAY crazier. But we got tests!
  • Mousewheel transparency is busted
  • Acrylic actually doesn't work at all
  • Manually print hyperlink to console
  • auto-detecting hyperlinks doesn't work
    • oh good this actually has a TODO!
  • Trying to make a second selection while one is active is weird
  • Actually making a second selection while one is active is impossible? (OH BUT IT'S BUSTED IN 1.8.892.0 AND 1.8.921.0 - fixed in #9727)
  • multi-click selection does seem to work
  • The scrollbar never moves!
  • Scrolling with the mouse is fine. With the trackpad though: Scrolling up seems to work, but scrolling down is very unsmooth
  • OH GOD just typing dir (presumably causing a scroll) HANGS THE TERMINAL
  • wheel scrolling of the font size seems to work
  • keybinding changing of the font size seems to work
  • changing DPI seems fine
  • Find dialog opens, works
  • Closing a tab with right-click close tab reliably crashes the Terminal
    • some crash in ITerminalConnection)->remove_TerminalOutput(impl::bind_in(token))); when exiting with VS debugger attached
    • The _connection is already null in ControlCore::Close
  • Remove Windows::UI::Input::PointerPoint
    • PointerPressed
    • PointerMoved
    • PointerReleased
    • MouseWheel
    • Touched
    • TouchMoved
    • TouchReleased
  • single click = no selection
  • single click and drag = selection starting from first point
  • single click in unfocused pane and drag = focus pane, selection starting from first point
  • double-click = selects a whole word
  • triple-click = selects a whole line
  • double-click and drag = selects a whole word, drag selects whole words
  • triple-click and drag = selects a whole line, drag selects whole lines
  • Shift single-click = defines start point
  • second Shift single-click = defines end point
  • Shift double-click = selects entire word
  • Shift triple-click = selects entire line
  • Shift double-click and drag = selects entire word, drag selects whole words
  • With existing selection: single-click outside the selection and drag = establishes a new selection starting from the click point
  • toggling renderer settings at runtime (without a shader set unset<->retro)
  • toggling renderer settings at runtime (with a shader set unset<->shader)
  • Hot reloading settings
    • Font
    • Antialiasing OH NO - seemingly a DPI change fixes this though. So something in the stack there is wrong. Not triggering a redraw, but triggering a font change?
      • Actually, this NEVER hot reloaded! We need to force the _d2dDeviceContext to the new setting
    • Background color
    • BG image
    • retro effect
    • Acrylic opacity
    • Scheme
    • CopyOnSelect
  • Hot reloading the settings seemed to make text with a background color disappear. But it didn't happen in both windows. ?
    • this never reprod again. Must've been something else.
    • I saw this come back after merging #8392. Might have actually been caused by that PR, need to investigate.
    • The two repros I saw were:
      • Change the BG to have an image - some of the text was temporarily invisible. not all of it, and not only the text with a BG color.
      • Mouse wheel the opacity - again once triggered the invisibility, but might have been a settings reload that hadn't yet propagated.
  • Right-click copy w/o CopyOnSelect
  • CopyOnSelect
  • read-only mode doesn't work at all currently.
    • keyboard input
    • mouse input in VT mode
  • The taskbar is fucky #9743
    • Setting the taskbar progress to indeterminate changes the taskbar, but not the tab icon
    • Set the taskbar progress to indeterminate in one tab, and switch to another tab. The taskbar still shows indeterminate. Neither tab has the spinner icon.
    • Set the taskbar progress to indeterminate in one pane, and switch to another pane. The taskbar still shows indeterminate. Neither pane gets the spinner icon.

Post- 5dc16327e6

  • The font weight doesn't load right initially

Other scenarios to check:

  • Click+drag out of the window to autoscroll the buffer
  • snap to grid - does that still work?
    • Seems good on 96 DPI
    • Might not work on high DPI
      • nevermind, this is behaving the same as it did before
    • Is the window supposed to resize when mouse scrolling the font size with snap to grid? because it doesn't currently.
      • It is not supposed to. So that's a good sign.
  • Mouse mode: Shift single-click = defines start point
  • Mouse mode: second Shift single-click = defines end point
  • Mouse mode: Shift double-click = selects entire word
  • Mouse mode: Shift triple-click = selects entire line
  • Mouse mode: Shift double-click and drag = selects entire word, drag selects whole words
  • Does actual mouse mode still work?
  • Prevent mouse dragging from dismissing existing selection #9790
  • more...

Tests to write:

  • Changing opacity with mouse wheel
  • Closing the ControlCore, then dtoring it
  • Scrolling with the mouse
  • Scrolling by panning with touch
  • scroll initiated by the terminal
  • scrollbar initiated scrolling
  • single click = no selection
  • single click and drag = selection starting from first point
  • With existing selection: single-click outside the selection and drag = establishes a new selection starting from the click point
  • Prevent mouse dragging from dismissing existing selection #9790
@zadjii-msft commented on GitHub (Mar 31, 2021): Alright, as of 2ac83180f, this is fundamentally pulled into 3 files, but there's still a lot of work to do. I'm using this comment to track that. * [x] Move `ControlCore` and `ControlInteractivity` to `Microsoft.Terminal.Core` so I can unittest them - didn't exactly do that, because that was WAY crazier. But we got tests! * [x] Mousewheel transparency is busted * [x] Acrylic actually doesn't work at all * [x] Manually print hyperlink to console * [x] auto-detecting hyperlinks doesn't work - oh good this actually has a `TODO!` * [x] Trying to make a second selection while one is active is weird * [x] Actually making a second selection while one is active is impossible? (OH BUT IT'S BUSTED IN `1.8.892.0` AND `1.8.921.0` - fixed in #9727) * [x] multi-click selection does seem to work * [x] The scrollbar _never moves_! * [x] Scrolling with the mouse is fine. With the trackpad though: Scrolling up seems to work, but scrolling down is _very_ unsmooth * [x] OH GOD just typing `dir` (presumably causing a scroll) HANGS THE TERMINAL * [x] wheel scrolling of the font size seems to work * [x] keybinding changing of the font size seems to work * [x] changing DPI seems fine * [x] Find dialog opens, works * [x] Closing a tab with right-click close tab reliably crashes the Terminal - some crash in `ITerminalConnection)->remove_TerminalOutput(impl::bind_in(token)));` when exiting with VS debugger attached - The `_connection` is already null in `ControlCore::Close` * [x] Remove `Windows::UI::Input::PointerPoint` - [x] `PointerPressed` - [x] `PointerMoved` - [x] `PointerReleased` - [x] `MouseWheel` - [x] `Touched` - [x] `TouchMoved` - [x] `TouchReleased` * [x] single click = no selection * [x] single click and drag = selection starting from first point * [x] single click in unfocused pane and drag = focus pane, selection starting from first point * [x] double-click = selects a whole word * [x] triple-click = selects a whole line * [x] double-click and drag = selects a whole word, drag selects whole words * [x] triple-click and drag = selects a whole line, drag selects whole lines * [x] Shift single-click = defines start point * [x] second Shift single-click = defines end point * [x] Shift double-click = selects entire word * [x] Shift triple-click = selects entire line * [x] Shift double-click and drag = selects entire word, drag selects whole words * [x] With existing selection: single-click outside the selection and drag = establishes a new selection starting from the click point * [x] toggling renderer settings at runtime (without a shader set unset\<->retro) * [x] toggling renderer settings at runtime (with a shader set unset\<->shader) * [ ] Hot reloading settings - [x] Font - [ ] Antialiasing ❌ OH NO - seemingly a DPI change fixes this though. So something in the stack there is wrong. Not triggering a redraw, but triggering a font change? - Actually, this NEVER hot reloaded! We need to force the `_d2dDeviceContext` to the new setting - [x] Background color - [x] BG image - [x] retro effect - [x] Acrylic opacity - [x] Scheme - [x] CopyOnSelect * [x] Hot reloading the settings seemed to make text with a background color disappear. But it didn't happen in both windows. ? - this never reprod again. Must've been something else. - I saw this come back after merging #8392. Might have actually been caused by that PR, need to investigate. - The two repros I saw were: - Change the BG to have an image - some of the text was temporarily invisible. _not_ all of it, and not only the text with a BG color. - Mouse wheel the opacity - again once triggered the invisibility, but might have been a settings reload that hadn't yet propagated. * [x] Right-click copy w/o CopyOnSelect * [x] CopyOnSelect * [x] read-only mode doesn't work at all currently. * [x] keyboard input * [x] mouse input in VT mode * [x] The taskbar is fucky #9743 * [x] Setting the taskbar progress to indeterminate changes the taskbar, but not the tab icon * [x] Set the taskbar progress to indeterminate in one tab, and switch to another tab. The taskbar still shows indeterminate. Neither tab has the spinner icon. * [x] Set the taskbar progress to indeterminate in one pane, and switch to another pane. The taskbar still shows indeterminate. Neither pane gets the spinner icon. Post- 5dc16327e60df92e984bfa887f3d318d96775c47 * [x] The font weight doesn't load right initially Other scenarios to check: * [x] Click+drag out of the window to autoscroll the buffer * [x] snap to grid - does that still work? * [x] Seems good on 96 DPI * [x] Might not work on high DPI - nevermind, this is behaving the same as it did before * [x] Is the window supposed to resize when mouse scrolling the font size with snap to grid? because it doesn't currently. - It is _not_ supposed to. So that's a good sign. * [x] Mouse mode: Shift single-click = defines start point * [x] Mouse mode: second Shift single-click = defines end point * [x] Mouse mode: Shift double-click = selects entire word * [x] Mouse mode: Shift triple-click = selects entire line * [x] Mouse mode: Shift double-click and drag = selects entire word, drag selects whole words * [x] Does actual mouse mode still work? * [ ] Prevent mouse dragging from dismissing existing selection #9790 * [ ] more... Tests to write: * [x] Changing opacity with mouse wheel * [x] Closing the `ControlCore`, then dtoring it * [ ] Scrolling with the mouse * [ ] Scrolling by panning with touch * [ ] scroll initiated by the terminal * [ ] scrollbar initiated scrolling * [x] single click = no selection * [x] single click and drag = selection starting from first point * [x] With existing selection: single-click outside the selection and drag = establishes a new selection starting from the click point * [ ] Prevent mouse dragging from dismissing existing selection #9790
Author
Owner

@DHowett commented on GitHub (Apr 22, 2021):

  • moving between DPIs does not introduce any issues in VT mouse events, rendering, scrolling, dragging, etc
@DHowett commented on GitHub (Apr 22, 2021): * [ ] moving between DPIs does not introduce any issues in VT mouse events, rendering, scrolling, dragging, etc
Author
Owner

@ghost commented on GitHub (May 25, 2021):

:tada:This issue was addressed in #9820, which has now been successfully released as Windows Terminal Preview v1.9.1445.0.🎉

Handy links:

@ghost commented on GitHub (May 25, 2021): :tada:This issue was addressed in #9820, which has now been successfully released as `Windows Terminal Preview v1.9.1445.0`.:tada: Handy links: * [Release Notes](https://github.com/microsoft/terminal/releases/tag/v1.9.1445.0) * [Store Download](https://www.microsoft.com/store/apps/9n8g5rfz9xk3?cid=storebadge&ocid=badge)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#9513