[PR #9249] [MERGED] PICK: Work around an optimizer issue with SetPixelShaderPath #27494

Open
opened 2026-01-31 09:22:17 +00:00 by claunia · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/microsoft/terminal/pull/9249
Author: @DHowett
Created: 2/22/2021
Status: Merged
Merged: 2/22/2021
Merged by: @DHowett

Base: mainHead: dev/duhowett/17opt


📝 Commits (1)

  • 7bcdb67 Work around a (likely) optimizer issue with SetPixelShaderPath (#9092)

📊 Changes

2 files changed (+74 additions, -53 deletions)

View changed files

📝 src/cascadia/TerminalControl/TermControl.cpp (+72 -52)
📝 src/cascadia/TerminalControl/TermControl.h (+2 -1)

📄 Description

It appears as though the optimizer is generating a sequence of
instructions on x64 that results in a nonsense std::wstring_view being
passed to SetPixelShaderPath when it's converted from a winrt::hstring.

Initially, I suspected that the issue was in us caching _settings
before we broke off the coroutine to update settings on the UI thread. I
implemented a quick fix for this (applying values off the new settings
object while also storing it in the control instance), but it didn't
actually lead anywhere. I do think it's the right thing to do for code
health's sake. Pankaj already changed how this works in 1.7: we no
longer (ever) re-seat the _settings reference... we only ever change
its parentage. Whether this is right or wrong is not for this paragraph
to discuss.

Eventually, I started looking more closely at the time travel traces. It
seriously looks like the wstring_view is generated wrong to begin with.
The debugger points directly at return { L"", 0 }; (which is correct),
but the values we get immediately on the other side of the call are
something like { 0x7FFFFFFF, 0 } or { 0x0, 0x48454C4C }.

I moved just the call to SetPixelShaderPath into a separate function.
The bug miraculously disappeared when I marked it noinline. It
reappeared when the function was fully inlined.

To avoid any future issues, I moved the whole UI thread body of
UpdateSettings out into its own function, to be called only while on the
UI thread. This fixes the bug.

Closes #8723.
Closes #9064.

I found a repro (update the settings file every 0.5 seconds and resize
the terminal wildly while it's doing so) that would trigger the bug
within ~10 seconds. It stopped doing so.

(cherry picked from commit 91b867102c)


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/microsoft/terminal/pull/9249 **Author:** [@DHowett](https://github.com/DHowett) **Created:** 2/22/2021 **Status:** ✅ Merged **Merged:** 2/22/2021 **Merged by:** [@DHowett](https://github.com/DHowett) **Base:** `main` ← **Head:** `dev/duhowett/17opt` --- ### 📝 Commits (1) - [`7bcdb67`](https://github.com/microsoft/terminal/commit/7bcdb67065a53475488ce7097968fa68b854ce2c) Work around a (likely) optimizer issue with SetPixelShaderPath (#9092) ### 📊 Changes **2 files changed** (+74 additions, -53 deletions) <details> <summary>View changed files</summary> 📝 `src/cascadia/TerminalControl/TermControl.cpp` (+72 -52) 📝 `src/cascadia/TerminalControl/TermControl.h` (+2 -1) </details> ### 📄 Description It appears as though the optimizer is generating a sequence of instructions on x64 that results in a nonsense std::wstring_view being passed to SetPixelShaderPath when it's converted from a winrt::hstring. Initially, I suspected that the issue was in us caching `_settings` before we broke off the coroutine to update settings on the UI thread. I implemented a quick fix for this (applying values off the new settings object while also storing it in the control instance), but it didn't actually lead anywhere. I do think it's the right thing to do for code health's sake. Pankaj already changed how this works in 1.7: we no longer (ever) re-seat the `_settings` reference... we only ever change its parentage. Whether this is right or wrong is not for this paragraph to discuss. Eventually, I started looking more closely at the time travel traces. It seriously looks like the wstring_view is generated wrong to begin with. The debugger points directly at `return { L"", 0 };` (which is correct), but the values we get immediately on the other side of the call are something like `{ 0x7FFFFFFF, 0 }` or `{ 0x0, 0x48454C4C }`. I moved _just_ the call to SetPixelShaderPath into a separate function. The bug miraculously disappeared when I marked it **noinline**. It reappeared when the function was fully inlined. To avoid any future issues, I moved the whole UI thread body of UpdateSettings out into its own function, to be called only while on the UI thread. This fixes the bug. Closes #8723. Closes #9064. I found a repro (update the settings file every 0.5 seconds and resize the terminal wildly while it's doing so) that would trigger the bug within ~10 seconds. It stopped doing so. (cherry picked from commit 91b867102c0fb9522a4fa3100f207cfa72aac3cc) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-31 09:22:17 +00:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#27494