Interactivity bugs post-#9820 #13633

Closed
opened 2026-01-31 03:47:51 +00:00 by claunia · 8 comments
Owner

Originally created by @zadjii-msft on GitHub (Apr 26, 2021).

Originally assigned to: @zadjii-msft on GitHub.

This is a tracking thread for things I probably broke in #9820. I'd rather not file 100 bugs for these

Mysterious DX crash

I experienced a crash when I killed DWM, and it took particularly long to recover

crash was on:

            RETURN_IF_FAILED(_dxgiFactory2->CreateSwapChainForComposition(_d3dDevice.Get(),
                                                                          &_swapChainDesc,
                                                                          nullptr,
                                                                          &_dxgiSwapChain));

in

0:000> k
 # Child-SP          RetAddr               Call Site
00 00000043`cef8ec80 00007ff9`3a4ac5a1     Microsoft_Terminal_Control!Microsoft::Console::Render::DxEngine::_CreateDeviceResources+0x47b [E:\BA\242\s\src\renderer\dx\DxRenderer.cpp @ 630] 
01 (Inline Function) --------`--------     Microsoft_Terminal_Control!Microsoft::Console::Render::DxEngine::GetSwapChain+0x14 [E:\BA\242\s\src\renderer\dx\DxRenderer.cpp @ 1010] 
02 00000043`cef8ed50 00007ff9`3a55667d     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::ControlCore::GetSwapChain+0x21 [E:\BA\242\s\src\cascadia\TerminalControl\ControlCore.cpp @ 1183] 
03 00000043`cef8ed80 00007ff9`3a4e1f71     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::TermControl::RenderEngineSwapChainChanged$_ResumeCoro$1+0x2bd [E:\BA\242\s\src\cascadia\TerminalControl\TermControl.cpp @ 582] 
04 (Inline Function) --------`--------     Microsoft_Terminal_Control!std::experimental::coroutine_handle<void>::resume+0x9 [C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\experimental\coroutine @ 107] 
05 (Inline Function) --------`--------     Microsoft_Terminal_Control!std::experimental::coroutine_handle<void>::operator()+0x9 [C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\experimental\coroutine @ 99] 
06 00000043`cef8ee50 00007ff9`5e2623a6     Microsoft_Terminal_Control!winrt::impl::resume_apartment_callback+0x11 [E:\BA\242\s\src\cascadia\TerminalControl\Generated Files\winrt\windows.ui.core.h @ 1088] 
07 00000043`cef8ee80 00007ff9`5e2af93f     Windows_UI!<lambda_59517943c03487243f9bea31c6c1a784>::operator()+0x84 [onecoreuap\windows\advcore\winrt\onecoreiwindow\corewindow\common\dispatcher.cpp @ 908] 
08 00000043`cef8eeb0 00007ff9`7de6a440     Windows_UI!Microsoft::WRL::Details::DelegateArgTraits<long (__cdecl Windows::System::IDispatcherQueueHandler::*)(void)>::DelegateInvokeHelper<Microsoft::WRL::Implements<Microsoft::WRL::RuntimeClassFlags<2>,Windows::System::IDispatcherQueueHandler,Microsoft::WRL::FtmBase>,<lambda_59517943c03487243f9bea31c6c1a784>,-1>::Invoke+0xf [onecore\external\sdk\inc\wrl\event.h @ 354] 
09 00000043`cef8eee0 00007ff9`7de4229c     CoreMessaging!Windows::System::DispatcherQueue::DeferInvokeCallback+0x20 [mincore\coreui\dev\dispatcherqueue\wrtdispatcherqueue.cpp @ 903] 
0a (Inline Function) --------`--------     CoreMessaging!Microsoft::CoreUI::ActionCallback::ImportAdapter$::__l2::<lambda_a81ff790741c2a62f2197c2561f5fe49>::operator()+0x1f [mincore\CoreUI\Dev\System\Api\IExportMessageSession.cs @ 22] 

the _dxgiFactory2 was null. That's unexpected. It shouldn't be null.

Originally created by @zadjii-msft on GitHub (Apr 26, 2021). Originally assigned to: @zadjii-msft on GitHub. This is a tracking thread for things I probably broke in #9820. I'd rather not file 100 bugs for these * [x] (**A**) Mousedown, select, SCROLL does a weird thing with endpoints that doesn't happen in stable - fixed in 4f4df01 - fixed in #9980 * [x] (**B**) Trackpad scrolling _down_ with _small_ increments seems buggy. But scrolling with a big increment goes fast? And scrolling with the mouse is fine. And trackpad scrolling in mouse mode is fine. - fixed in #9980 * [x] (**C**) Looks like there's a selection issue when you click and drag too quickly. Seems like the start endpoint doesn't always get set. - fixed in 9d33e89 - fixed in #9980 * [x] (**D**) Mysterious DX crash - _this never reprod, so we're assuming it's unrelated_ <details> <summary>Mysterious DX crash</summary> I experienced a crash when I killed DWM, and it took particularly long to recover crash was on: ```c++ RETURN_IF_FAILED(_dxgiFactory2->CreateSwapChainForComposition(_d3dDevice.Get(), &_swapChainDesc, nullptr, &_dxgiSwapChain)); ``` in ``` 0:000> k # Child-SP RetAddr Call Site 00 00000043`cef8ec80 00007ff9`3a4ac5a1 Microsoft_Terminal_Control!Microsoft::Console::Render::DxEngine::_CreateDeviceResources+0x47b [E:\BA\242\s\src\renderer\dx\DxRenderer.cpp @ 630] 01 (Inline Function) --------`-------- Microsoft_Terminal_Control!Microsoft::Console::Render::DxEngine::GetSwapChain+0x14 [E:\BA\242\s\src\renderer\dx\DxRenderer.cpp @ 1010] 02 00000043`cef8ed50 00007ff9`3a55667d Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::ControlCore::GetSwapChain+0x21 [E:\BA\242\s\src\cascadia\TerminalControl\ControlCore.cpp @ 1183] 03 00000043`cef8ed80 00007ff9`3a4e1f71 Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::TermControl::RenderEngineSwapChainChanged$_ResumeCoro$1+0x2bd [E:\BA\242\s\src\cascadia\TerminalControl\TermControl.cpp @ 582] 04 (Inline Function) --------`-------- Microsoft_Terminal_Control!std::experimental::coroutine_handle<void>::resume+0x9 [C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\experimental\coroutine @ 107] 05 (Inline Function) --------`-------- Microsoft_Terminal_Control!std::experimental::coroutine_handle<void>::operator()+0x9 [C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\experimental\coroutine @ 99] 06 00000043`cef8ee50 00007ff9`5e2623a6 Microsoft_Terminal_Control!winrt::impl::resume_apartment_callback+0x11 [E:\BA\242\s\src\cascadia\TerminalControl\Generated Files\winrt\windows.ui.core.h @ 1088] 07 00000043`cef8ee80 00007ff9`5e2af93f Windows_UI!<lambda_59517943c03487243f9bea31c6c1a784>::operator()+0x84 [onecoreuap\windows\advcore\winrt\onecoreiwindow\corewindow\common\dispatcher.cpp @ 908] 08 00000043`cef8eeb0 00007ff9`7de6a440 Windows_UI!Microsoft::WRL::Details::DelegateArgTraits<long (__cdecl Windows::System::IDispatcherQueueHandler::*)(void)>::DelegateInvokeHelper<Microsoft::WRL::Implements<Microsoft::WRL::RuntimeClassFlags<2>,Windows::System::IDispatcherQueueHandler,Microsoft::WRL::FtmBase>,<lambda_59517943c03487243f9bea31c6c1a784>,-1>::Invoke+0xf [onecore\external\sdk\inc\wrl\event.h @ 354] 09 00000043`cef8eee0 00007ff9`7de4229c CoreMessaging!Windows::System::DispatcherQueue::DeferInvokeCallback+0x20 [mincore\coreui\dev\dispatcherqueue\wrtdispatcherqueue.cpp @ 903] 0a (Inline Function) --------`-------- CoreMessaging!Microsoft::CoreUI::ActionCallback::ImportAdapter$::__l2::<lambda_a81ff790741c2a62f2197c2561f5fe49>::operator()+0x1f [mincore\CoreUI\Dev\System\Api\IExportMessageSession.cs @ 22] ``` the `_dxgiFactory2` was null. That's unexpected. It shouldn't be null. </details> * [x] (**E**) It doesn't seem like we're calling this when we create the automation peer -- we are not guaranteed to always call UpdateSettings after creating a peer, so the first peer may have no padding. See https://github.com/microsoft/terminal/pull/10051#discussion_r672443301. (fixed by #10874)
Author
Owner

@DHowett commented on GitHub (Apr 28, 2021):

Looks like there's a selection issue when you click and drag too quickly.

(Sorry for the terse body; I have to post this using powershell because of a new MSIT rule about personal devices :|)

@DHowett commented on GitHub (Apr 28, 2021): Looks like there's a selection issue when you click and drag too quickly. (Sorry for the terse body; I have to post this using powershell because of a new MSIT rule about personal devices :|)
Author
Owner

@zadjii-msft commented on GitHub (Apr 28, 2021):

Looks like there's a selection issue when you click and drag too quickly.

There sure is, just hit like 4 times this morning. Added to the list ☺️

@zadjii-msft commented on GitHub (Apr 28, 2021): > Looks like there's a selection issue when you click and drag too quickly. There sure is, just hit like 4 times this morning. Added to the list ☺️
Author
Owner

@zadjii-msft commented on GitHub (Apr 28, 2021):

haha, found it

if (_singleClickTouchdownPos)
{
    // Figure out if the user's moved a quarter of a cell's smaller axis away from the clickdown point
    auto& touchdownPoint{ *_singleClickTouchdownPos };
    auto distance{ std::sqrtf(std::powf(cursorPosition.X - touchdownPoint.X, 2) + std::powf(cursorPosition.Y - touchdownPoint.Y, 2)) };
    const til::size fontSize{ _actualFont.GetSize() };

    const auto fontSizeInDips = fontSize.scale(til::math::rounding, 1.0f / _renderEngine->GetScaling());
    if (distance >= (std::min(fontSizeInDips.width(), fontSizeInDips.height()) / 4.f))
    {
        _terminal->SetSelectionAnchor(_GetTerminalPosition(touchdownPoint));
        // stop tracking the touchdown point
        _singleClickTouchdownPos = std::nullopt;
    }
}
if (_singleClickTouchdownPos)
{
    // Figure out if the user's moved a quarter of a cell's smaller axis away from the clickdown point
    auto& touchdownPoint{ *_singleClickTouchdownPos };
    float dx = ::base::saturated_cast<float>(pixelPosition.x() - touchdownPoint.x());
    float dy = ::base::saturated_cast<float>(pixelPosition.y() - touchdownPoint.y());
    auto distance{ std::sqrtf(std::powf(dx, 2) +
                              std::powf(dy, 2)) };

    const auto fontSizeInDips{ _core->FontSizeInDips() };
    if (distance >= (std::min(fontSizeInDips.width(), fontSizeInDips.height()) / 4.f))
    {
        _core->SetSelectionAnchor(terminalPosition);
        // stop tracking the touchdown point
        _singleClickTouchdownPos = std::nullopt;
    }
}
        _terminal->SetSelectionAnchor(_GetTerminalPosition(touchdownPoint));

vs

        _core->SetSelectionAnchor(terminalPosition);

We're now using the location of the drag event as the selection anchor, instead of the location that the user initially clicked. Oops. Now to write a test.

@zadjii-msft commented on GitHub (Apr 28, 2021): haha, found it <table> <tr> <td> ```c++ if (_singleClickTouchdownPos) { // Figure out if the user's moved a quarter of a cell's smaller axis away from the clickdown point auto& touchdownPoint{ *_singleClickTouchdownPos }; auto distance{ std::sqrtf(std::powf(cursorPosition.X - touchdownPoint.X, 2) + std::powf(cursorPosition.Y - touchdownPoint.Y, 2)) }; const til::size fontSize{ _actualFont.GetSize() }; const auto fontSizeInDips = fontSize.scale(til::math::rounding, 1.0f / _renderEngine->GetScaling()); if (distance >= (std::min(fontSizeInDips.width(), fontSizeInDips.height()) / 4.f)) { _terminal->SetSelectionAnchor(_GetTerminalPosition(touchdownPoint)); // stop tracking the touchdown point _singleClickTouchdownPos = std::nullopt; } } ``` </td> <td> ```c++ if (_singleClickTouchdownPos) { // Figure out if the user's moved a quarter of a cell's smaller axis away from the clickdown point auto& touchdownPoint{ *_singleClickTouchdownPos }; float dx = ::base::saturated_cast<float>(pixelPosition.x() - touchdownPoint.x()); float dy = ::base::saturated_cast<float>(pixelPosition.y() - touchdownPoint.y()); auto distance{ std::sqrtf(std::powf(dx, 2) + std::powf(dy, 2)) }; const auto fontSizeInDips{ _core->FontSizeInDips() }; if (distance >= (std::min(fontSizeInDips.width(), fontSizeInDips.height()) / 4.f)) { _core->SetSelectionAnchor(terminalPosition); // stop tracking the touchdown point _singleClickTouchdownPos = std::nullopt; } } ``` </td> </tr> </table> ```c++ _terminal->SetSelectionAnchor(_GetTerminalPosition(touchdownPoint)); ``` vs ```c++ _core->SetSelectionAnchor(terminalPosition); ``` We're now using the location of the drag event as the selection anchor, instead of the location that the user initially clicked. Oops. Now to write a test.
Author
Owner

@zadjii-msft commented on GitHub (Apr 28, 2021):

Oof. B is hard, I think. The touchpad sends very many small scroll deltas, less than one row at a time. The control scrollbar can store a double, so small deltas can accumulate. Originally, these would accumulate in the scrollbar, and we'd only read that out as an int in the scrollbar updater, which is throttled.

In the interactivity split, there's no place for us to store that double. We immediately narrow to an int for ControlInteractivity::_updateScrollbar.

Theoretically, we could store a double in Interactivity that's a fake scrollbar position, to accumulate to. Then, ControlInteractivity would need to listen to scroll events from the core, unsure how easy that is right now...

@zadjii-msft commented on GitHub (Apr 28, 2021): Oof. B is hard, I think. The touchpad sends very many small scroll deltas, less than one row at a time. The control scrollbar can store a `double`, so small deltas can accumulate. Originally, these would accumulate in the scrollbar, and we'd only read that out as an `int` in the scrollbar updater, which is throttled. In the interactivity split, there's no place for us to store that double. We immediately narrow to an `int` for `ControlInteractivity::_updateScrollbar`. Theoretically, we could store a `double` in `Interactivity` that's a fake scrollbar position, to accumulate to. Then, `ControlInteractivity` would need to listen to scroll events from the core, unsure how easy that is right now...
Author
Owner

@DHowett commented on GitHub (Apr 28, 2021):

Oh yes, that is frightening for sure.

This is a wackadoo idea, but... what if the scrollbar's size was WHEEL_DELTA * BufferHeight and we just ... let the deltas accumulate and only scroll every WHEEL_DELTA units

@DHowett commented on GitHub (Apr 28, 2021): Oh yes, that is frightening for sure. This is a wackadoo idea, but... what if the scrollbar's size was `WHEEL_DELTA * BufferHeight` and we just ... let the deltas accumulate and only scroll every WHEEL_DELTA units
Author
Owner

@DHowett commented on GitHub (Apr 28, 2021):

No no it would probably be absolutely wack for the UI to display that (or Accessibility would say "line 300 of 1991661")

@DHowett commented on GitHub (Apr 28, 2021): No no it would probably be absolutely wack for the UI to display that (or Accessibility would say "line 300 of 1991661")
Author
Owner

@DHowett commented on GitHub (Jul 19, 2021):

Point [E] may cause an accessibility regression in the 1.11 timeframe.

@DHowett commented on GitHub (Jul 19, 2021): Point [E] may cause an accessibility regression in the 1.11 timeframe.
Author
Owner

@ghost commented on GitHub (Aug 31, 2021):

:tada:This issue was addressed in #10874, which has now been successfully released as Windows Terminal Preview v1.11.2421.0.🎉

Handy links:

@ghost commented on GitHub (Aug 31, 2021): :tada:This issue was addressed in #10874, which has now been successfully released as `Windows Terminal Preview v1.11.2421.0`.:tada: Handy links: * [Release Notes](https://github.com/microsoft/terminal/releases/tag/v1.11.2421.0) * [Store Download](https://www.microsoft.com/store/apps/9n8g5rfz9xk3?cid=storebadge&ocid=badge)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#13633