When the renderer requires too many retries, try gracefully surviving #5600

Closed
opened 2026-01-31 00:17:04 +00:00 by claunia · 6 comments
Owner

Originally created by @zadjii-msft on GitHub (Dec 17, 2019).

Today I had DWM or some graphics driver or something take a real long trip upstate. Fortunately, the Terminal didn't immediately crash. However, when the rendering did return, the terminal eventually did crash, because the renderer required too many tries to paint the frame.

while (tries > 0)
{
    const auto hr = _PaintFrameForEngine(pEngine);
    if (E_PENDING == hr)
    {
        if (--tries == 0)
        {
            FAIL_FAST_HR_MSG(E_UNEXPECTED, "A rendering engine required too many retries.");
        }
        continue;
    }
    LOG_IF_FAILED(hr);
    break;
}

@miniksa ca correct me if I'm wrong here, but I think this should maybe just gracefully do nothing, maybe log a message.

I have no idea how to repro this manually, maybe taskkill /f /im dwm.exe would work, but I don't really want to try again.

Originally created by @zadjii-msft on GitHub (Dec 17, 2019). Today I had DWM or some graphics driver or something take a real _long_ trip upstate. Fortunately, the Terminal didn't immediately crash. However, when the rendering did return, the terminal eventually did crash, because the renderer required too many tries to paint the frame. ```c++ while (tries > 0) { const auto hr = _PaintFrameForEngine(pEngine); if (E_PENDING == hr) { if (--tries == 0) { FAIL_FAST_HR_MSG(E_UNEXPECTED, "A rendering engine required too many retries."); } continue; } LOG_IF_FAILED(hr); break; } ``` @miniksa ca correct me if I'm wrong here, but I think this should maybe just gracefully _do nothing_, maybe log a message. I have no idea how to repro this manually, maybe `taskkill /f /im dwm.exe` would work, but I don't really want to try again.
Author
Owner

@miniksa commented on GitHub (Jan 3, 2020):

@miniksa Michael Niksa FTE ca correct me if I'm wrong here, but I think this should maybe just gracefully do nothing, maybe log a message.

I have no idea how to repro this manually, maybe taskkill /f /im dwm.exe would work, but I don't really want to try again.

I think maybe we need to identify the reason behind the failure and determine which ones are worth retrying and which ones are unrecoverable.

Also, I'd want to look back in time and check the blame on the file to find the commit/PR/discussion that added that retry loop to see if we can gain some insight on whether the retry loop is mitigating some other usually recoverable issue or whether we must fail for some specific reason.

@miniksa commented on GitHub (Jan 3, 2020): > @miniksa Michael Niksa FTE ca correct me if I'm wrong here, but I think this should maybe just gracefully _do nothing_, maybe log a message. > > I have no idea how to repro this manually, maybe `taskkill /f /im dwm.exe` would work, but I don't really want to try again. I think maybe we need to identify the reason behind the failure and determine which ones are worth retrying and which ones are unrecoverable. Also, I'd want to look back in time and check the blame on the file to find the commit/PR/discussion that added that retry loop to see if we can gain some insight on whether the retry loop is mitigating some other usually recoverable issue or whether we must fail for some specific reason.
Author
Owner

@DHowett-MSFT commented on GitHub (Jan 3, 2020):

@miniksa so, that was me. We were failing when we lost dcomp/dwm and the swap chain flip method failed with "E_PLEASE_RETRY_MY_DEVICE_WENT_AWAY".

@DHowett-MSFT commented on GitHub (Jan 3, 2020): @miniksa so, that was me. We were failing when we lost dcomp/dwm and the swap chain flip method failed with "E_PLEASE_RETRY_MY_DEVICE_WENT_AWAY".
Author
Owner

@miniksa commented on GitHub (Jan 3, 2020):

I still feel like there's a maximum number of times to retry when your graphics pipeline soils the bed. Just like when you have bigger problems when your entire machine is out of available virtual memory.

@miniksa commented on GitHub (Jan 3, 2020): I still feel like there's a maximum number of times to retry when your graphics pipeline soils the bed. Just like when you have bigger problems when your entire machine is out of available virtual memory.
Author
Owner

@zadjii-msft commented on GitHub (Jan 3, 2020):

@miniksa that's probably true, but I still thought it was peculiar that the Terminal was the only app that beef'd it when the display crashed like that. Even Teams survived. Maybe there's some other event we're missing, like SwapChainPanel::OnDisplayBeefedIt, that we can use to disable/re-enable the render engine.

@zadjii-msft commented on GitHub (Jan 3, 2020): @miniksa that's probably true, but I still thought it was peculiar that the Terminal was the only app that beef'd it when the display crashed like that. Even Teams survived. Maybe there's some other event we're missing, like `SwapChainPanel::OnDisplayBeefedIt`, that we can use to disable/re-enable the render engine.
Author
Owner

@miniksa commented on GitHub (Jan 3, 2020):

SwapChainPanel::OnDisplayBeefedIt

Yes.

@miniksa commented on GitHub (Jan 3, 2020): > `SwapChainPanel::OnDisplayBeefedIt` Yes.
Author
Owner

@DHowett-MSFT commented on GitHub (May 12, 2020):

It turns out that we did this for v1 in #5353!

@DHowett-MSFT commented on GitHub (May 12, 2020): It turns out that we did this for v1 in #5353!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#5600