[PR #9092] Work around a (likely) optimizer issue with SetPixelShaderPath #27418

Open
opened 2026-01-31 09:21:51 +00:00 by claunia · 0 comments
Owner

Original Pull Request: https://github.com/microsoft/terminal/pull/9092

State: closed
Merged: Yes


It appears as though the optimizer is generating a sequence of
instructions on x64 that results in a nonsense std::wstring_view being
passed to SetPixelShaderPath when it's converted from a winrt::hstring.

Initially, I suspected that the issue was in us caching _settings
before we broke off the coroutine to update settings on the UI thread. I
implemented a quick fix for this (applying values off the new settings
object while also storing it in the control instance), but it didn't
actually lead anywhere. I do think it's the right thing to do for code
health's sake. Pankaj already changed how this works in 1.7: we no
longer (ever) re-seat the _settings reference... we only ever change
its parentage. Whether this is right or wrong is not for this paragraph
to discuss.

Eventually, I started looking more closely at the time travel traces. It
seriously looks like the wstring_view is generated wrong to begin with.
The debugger points directly at return { L"", 0 }; (which is correct),
but the values we get immediately on the other side of the call are
something like { 0x7FFFFFFF, 0 } or { 0x0, 0x48454C4C }.

I moved just the call to SetPixelShaderPath into a separate function.
The bug miraculously disappeared when I marked it noinline. It
reappeared when the function was fully inlined.

To avoid any future issues, I moved the whole UI thread body of
UpdateSettings out into its own function, to be called only while on the
UI thread. This fixes the bug.

Closes #8723.
Closes #9064.

Validation Steps Performed

I found a repro (update the settings file every 0.5 seconds and resize
the terminal wildly while it's doing so) that would trigger the bug
within ~10 seconds. It stopped doing so.

**Original Pull Request:** https://github.com/microsoft/terminal/pull/9092 **State:** closed **Merged:** Yes --- It appears as though the optimizer is generating a sequence of instructions on x64 that results in a nonsense std::wstring_view being passed to SetPixelShaderPath when it's converted from a winrt::hstring. Initially, I suspected that the issue was in us caching `_settings` before we broke off the coroutine to update settings on the UI thread. I implemented a quick fix for this (applying values off the new settings object while also storing it in the control instance), but it didn't actually lead anywhere. I do think it's the right thing to do for code health's sake. Pankaj already changed how this works in 1.7: we no longer (ever) re-seat the `_settings` reference... we only ever change its parentage. Whether this is right or wrong is not for this paragraph to discuss. Eventually, I started looking more closely at the time travel traces. It seriously looks like the wstring_view is generated wrong to begin with. The debugger points directly at `return { L"", 0 };` (which is correct), but the values we get immediately on the other side of the call are something like `{ 0x7FFFFFFF, 0 }` or `{ 0x0, 0x48454C4C }`. I moved _just_ the call to SetPixelShaderPath into a separate function. The bug miraculously disappeared when I marked it **noinline**. It reappeared when the function was fully inlined. To avoid any future issues, I moved the whole UI thread body of UpdateSettings out into its own function, to be called only while on the UI thread. This fixes the bug. Closes #8723. Closes #9064. ## Validation Steps Performed I found a repro (update the settings file every 0.5 seconds and resize the terminal wildly while it's doing so) that would trigger the bug within ~10 seconds. It stopped doing so.
claunia added the pull-request label 2026-01-31 09:21:51 +00:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#27418