Repeated Bell (eg. end of line in Vim) Causes Reproducible Crash #14883

@DHowett commented on GitHub (Aug 17, 2021):

@comerford you may, as a workaround, want to switch to visual belling only. It's at the bottom of the profile settings' first page. If you're on Preview, you can set it for all profiles at once in the "Defaults" section. That is, unless you really like having audible bells. 😄

@DHowett commented on GitHub (Aug 17, 2021): @comerford you may, as a workaround, want to switch to visual belling only. It's at the bottom of the profile settings' first page. If you're on Preview, you can set it for all profiles at once in the "Defaults" section. That is, unless you really like having audible bells. 😄

claunia commented

@comerford commented on GitHub (Aug 17, 2021):

You know, I recall that I previously had the sound disabled, but that may have gotten reset previously, and certainly once I deleted and reinstalled.

I played with some of the combinations, and enabling a screen flash in addition to the audible bell still caused the crash to occur but it took a lot longer to happen. It's as if the flash slows things down enough to make the crash less likely (the flash causes the audible bells to be slower, I think).

If I enable just the window flash there seems to be no crash, though I will have to do some extended testing to be sure. Sometimes the audible bell can take some time to cause a crash too.

@comerford commented on GitHub (Aug 17, 2021): You know, I recall that I previously had the sound disabled, but that may have gotten reset previously, and certainly once I deleted and reinstalled. I played with some of the combinations, and enabling a screen flash in addition to the audible bell still caused the crash to occur but it took a lot longer to happen. It's as if the flash slows things down enough to make the crash less likely (the flash causes the audible bells to be slower, I think). If I enable just the window flash there seems to be no crash, though I will have to do some extended testing to be sure. Sometimes the audible bell can take some time to cause a crash too.

claunia commented

@j4james commented on GitHub (Aug 17, 2021):

This may be a long shot, but have a look at the settings for your audio output device, and see if you've got any advanced options enabled (e.g. something like surround sound, or bass boost) and try turning those off. I remember having a problem in the past when triggering a lot of PlaySound operations, and it ended up being the result of a bug in the audio driver, which was resolved by disabling some or other option.

@j4james commented on GitHub (Aug 17, 2021): This may be a long shot, but have a look at the settings for your audio output device, and see if you've got any advanced options enabled (e.g. something like surround sound, or bass boost) and try turning those off. I remember having a problem in the past when triggering a lot of `PlaySound` operations, and it ended up being the result of a bug in the audio driver, which was resolved by disabling some or other option.

claunia commented

@comerford commented on GitHub (Aug 18, 2021):

I have several audio output devices but the one that was in use when I reproduced the crash (and the one I use most often) does not have any such effects enabled (it's an external USB DAC for headphones).

I have successfully reproduced the crash with the window flash and the sound enabled (using vim, same as the GIF above), but have yet to see a problem once the sound setting is disabled (have been testing window flash setting all day without a crash) and I cannot force a crash with the sound disabled, so that definitely seems to be the root of the issue.

@comerford commented on GitHub (Aug 18, 2021): I have several audio output devices but the one that was in use when I reproduced the crash (and the one I use most often) does not have any such effects enabled (it's an external USB DAC for headphones). I have successfully reproduced the crash with the window flash and the sound enabled (using vim, same as the GIF above), but have yet to see a problem once the sound setting is disabled (have been testing window flash setting all day without a crash) and I cannot force a crash with the sound disabled, so that definitely seems to be the root of the issue.

claunia commented

@miniksa commented on GitHub (Aug 18, 2021):

I've contacted the team that owns the MmDevApi.dll internally. I found an entire class of crashes reported to our crash analysis with similar stacks. I browsed the code for this particular callback DLL and it has a lot of locking and threading going on... so some sort of race isn't totally out of the question especially if we've changed our behavior in how we're calling it recently. Hopefully the owning team gets back to us soon and gives us the assist in understanding how we can fix the issue in Windows and work around it here to make it less impactful.

@miniksa commented on GitHub (Aug 18, 2021): I've contacted the team that owns the MmDevApi.dll internally. I found an entire class of crashes reported to our crash analysis with similar stacks. I browsed the code for this particular callback DLL and it has a lot of locking and threading going on... so some sort of race isn't totally out of the question especially if we've changed our behavior in how we're calling it recently. Hopefully the owning team gets back to us soon and gives us the assist in understanding how we can fix the issue in Windows and work around it here to make it less impactful.

claunia commented

@miniksa commented on GitHub (Aug 19, 2021):

@comerford can you tell me some of the hardware specifications of your machine?

What CPU do you have?
What graphics card do you have?
What audio devices do you have (please include all of them including classic audio jack, bluetooth endpoints, and HDMI audio endpoints that may be a part of your graphics card)?
Be as specific as possible with make and model numbers and potentially driver versions where that feels relevant.

(You may be able to shorthand all of those questions with dxdiag.exe and presenting the log. If you feel it's too much information to post for the general internet... you can privately hit me up via email by checking out my GitHub username and appending the hopefully obvious domain name for my company.)

And can you reproduce the issue no matter which audio endpoint is chosen? Like is it only with your external DAC? Or does it also happen coming out of a possible onboard audio jack and/or an HDMI audio connection (if your monitor is equipped as such)?

Thank you for your patience. This entire class of crashes has been a fascinating journey for me today.

@miniksa commented on GitHub (Aug 19, 2021): @comerford can you tell me some of the hardware specifications of your machine? What CPU do you have? What graphics card do you have? What audio devices do you have (please include all of them including classic audio jack, bluetooth endpoints, and HDMI audio endpoints that may be a part of your graphics card)? Be as specific as possible with make and model numbers and potentially driver versions where that feels relevant. (You may be able to shorthand all of those questions with `dxdiag.exe` and presenting the log. If you feel it's too much information to post for the general internet... you can privately hit me up via email by checking out my GitHub username and appending the hopefully obvious domain name for my company.) And can you reproduce the issue no matter which audio endpoint is chosen? Like is it only with your external DAC? Or does it also happen coming out of a possible onboard audio jack and/or an HDMI audio connection (if your monitor is equipped as such)? Thank you for your patience. This entire class of crashes has been a fascinating journey for me today.

claunia commented

@comerford commented on GitHub (Aug 19, 2021):

I will experiment with the different sound outputs (I have 7 permanently connected, not including bluetooth) and let you know how it goes.

What CPU do you have?
AMD Ryzen 9 3900X 12-Core Processor (24 CPUs), ~3.8GHz

What graphics card do you have?
ASUS Dual GeForce RTX 2070 Super EVO OC Edition 8GB GDDR6

What audio devices do you have (please include all of them)
DELL UP2716D (NVIDIA High Definition Audio)
Echo Cancelling Speakerphone (Jabra SPEAK 410 USB)
Headphones (4- Arctis Pro Wireless Game)
Headset Earphone (4- Arctis Pro Wireless Chat)
ROG XG279Q (NVIDIA High Definition Audio)
Realtek Digital Output (2- Realtek(R) Audio)
SPDIF Interface (3- FX-AUDIO) <--- primary device, what has been used for most, if not all, testing

Be as specific as possible with make and model numbers and potentially driver versions where that feels relevant.
The primary device, the FX-Audio is specifically the FX Audio DAC-X6 MKII. It looks like this in dxdiag:

            Description: SPDIF Interface (3- FX-AUDIO)
 Default Sound Playback: Yes
 Default Voice Playback: No
            Hardware ID: USB\VID_0D8C&PID_0158&REV_0008&MI_00
        Manufacturer ID: N/A
             Product ID: N/A
                   Type: N/A
            Driver Name: USBAUDIO.sys
         Driver Version: 10.0.19041.1081 (English)
      Driver Attributes: Final Retail
            WHQL Logo'd: Yes
          Date and Size: 08/06/2021 01:00:00, 201728 bytes
            Other Files: 
        Driver Provider: Microsoft
         HW Accel Level: Emulation Only
              Cap Flags: 0xF1F
    Min/Max Sample Rate: 100, 200000
Static/Strm HW Mix Bufs: 1, 0
 Static/Strm HW 3D Bufs: 0, 0
              HW Memory: 0
       Voice Management: No
 EAX(tm) 2.0 Listen/Src: No, No
   I3DL2(tm) Listen/Src: No, No
Sensaura(tm) ZoomFX(tm): No

@comerford commented on GitHub (Aug 19, 2021): I will experiment with the different sound outputs (I have 7 permanently connected, not including bluetooth) and let you know how it goes. > What CPU do you have? AMD Ryzen 9 3900X 12-Core Processor (24 CPUs), ~3.8GHz > What graphics card do you have? ASUS Dual GeForce RTX 2070 Super EVO OC Edition 8GB GDDR6 > What audio devices do you have (please include all of them) DELL UP2716D (NVIDIA High Definition Audio) Echo Cancelling Speakerphone (Jabra SPEAK 410 USB) Headphones (4- Arctis Pro Wireless Game) Headset Earphone (4- Arctis Pro Wireless Chat) ROG XG279Q (NVIDIA High Definition Audio) Realtek Digital Output (2- Realtek(R) Audio) SPDIF Interface (3- FX-AUDIO) <--- primary device, what has been used for most, if not all, testing > Be as specific as possible with make and model numbers and potentially driver versions where that feels relevant. The primary device, the FX-Audio is specifically the FX Audio DAC-X6 MKII. It looks like this in dxdiag: ``` Description: SPDIF Interface (3- FX-AUDIO) Default Sound Playback: Yes Default Voice Playback: No Hardware ID: USB\VID_0D8C&PID_0158&REV_0008&MI_00 Manufacturer ID: N/A Product ID: N/A Type: N/A Driver Name: USBAUDIO.sys Driver Version: 10.0.19041.1081 (English) Driver Attributes: Final Retail WHQL Logo'd: Yes Date and Size: 08/06/2021 01:00:00, 201728 bytes Other Files: Driver Provider: Microsoft HW Accel Level: Emulation Only Cap Flags: 0xF1F Min/Max Sample Rate: 100, 200000 Static/Strm HW Mix Bufs: 1, 0 Static/Strm HW 3D Bufs: 0, 0 HW Memory: 0 Voice Management: No EAX(tm) 2.0 Listen/Src: No, No I3DL2(tm) Listen/Src: No, No Sensaura(tm) ZoomFX(tm): No ```

claunia commented

@comerford commented on GitHub (Aug 19, 2021):

OK, interesting. I got the crash on the ROG XG279Q (NVIDIA High Definition Audio) but not on the Echo Cancelling Speakerphone (Jabra SPEAK 410 USB). However, if I did a lot of bell sounds on a non-crashing sound output and then changed output, the change of output would then trigger the crash.

It doesn't seem to matter what I swap to, though I haven't gone through all the combinations, but the swap to a different output is definitely causing it to crash, same DLL too.

Here's a demo:

@comerford commented on GitHub (Aug 19, 2021): OK, interesting. I got the crash on the `ROG XG279Q (NVIDIA High Definition Audio)` but not on the `Echo Cancelling Speakerphone (Jabra SPEAK 410 USB)`. However, if I did a lot of bell sounds on a non-crashing sound output and then changed output, the change of output would then trigger the crash. It doesn't seem to matter what I swap to, though I haven't gone through all the combinations, but the swap to a different output is definitely causing it to crash, same DLL too. Here's a demo: ![terminal-crash-bell-swap](https://user-images.githubusercontent.com/445308/130090736-92660581-4755-4207-acf9-919f0fa6d8c0.gif)

claunia commented

@shanselman commented on GitHub (Aug 19, 2021):

I'm can repro this 100%. Let me know how I can help (I already gave you dumps)

@shanselman commented on GitHub (Aug 19, 2021): I'm can repro this 100%. Let me know how I can help (I already gave you dumps)

claunia commented

@miniksa commented on GitHub (Aug 20, 2021):

@comerford, I got an answer from MmDevApi.dll team. Do you have A-Volute Sonic Studio 3 installed?

@miniksa commented on GitHub (Aug 20, 2021): @comerford, I got an answer from MmDevApi.dll team. Do you have A-Volute Sonic Studio 3 installed?

claunia commented

@comerford commented on GitHub (Aug 20, 2021):

I am away for my 10th wedding anniversary so I can't check but that does sound familiar based on random software updates (I have never actually used it). I will confirm when I am back in a couple of days.

@comerford commented on GitHub (Aug 20, 2021): I am away for my 10th wedding anniversary so I can't check but that does sound familiar based on random software updates (I have never actually used it). I will confirm when I am back in a couple of days.

claunia commented

@miniksa commented on GitHub (Aug 20, 2021):

No problem. Enjoy your anniversary!

The sound team is going to reach out to A-Volute too to inform them of the issue as well.
I'm also looking for ways to improve resiliency on our side.

@miniksa commented on GitHub (Aug 20, 2021): No problem. Enjoy your anniversary! The sound team is going to reach out to A-Volute too to inform them of the issue as well. I'm also looking for ways to improve resiliency on our side.

claunia commented

@comerford commented on GitHub (Aug 22, 2021):

Can confirm - I had Sonic Studio 3 and Sonic Radar 3, must have been bundled with something, because it is not software that I actually use. Once I removed both pieces of software, no more crash in Windows Terminal with the bell sound enabled. Many thanks for all the work on this!

@comerford commented on GitHub (Aug 22, 2021): Can confirm - I had Sonic Studio 3 and Sonic Radar 3, must have been bundled with something, because it is not software that I actually use. Once I removed both pieces of software, no more crash in Windows Terminal with the bell sound enabled. Many thanks for all the work on this!

claunia commented

@zadjii-msft commented on GitHub (Aug 23, 2021):

Thanks for filing this! This whole bug has been quite the journey to watch get investigated. I'm gonna leave it open while we file up with the MmDevApi team to see if there's anything we can do on our side to be resilient to these types of crashes. That being said, I'm gonna also punt to the backlog, since this seems to have a root cause identified externally, and a workaround.

@zadjii-msft commented on GitHub (Aug 23, 2021): Thanks for filing this! This whole bug has been quite the journey to watch get investigated. I'm gonna leave it open while we file up with the MmDevApi team to see if there's anything we can do on our side to be resilient to these types of crashes. That being said, I'm gonna also punt to the backlog, since this seems to have a root cause identified externally, and a workaround.

claunia commented

@comerford commented on GitHub (Aug 23, 2021):

Just in case anyone is wondering where Sonic Studio 3 comes from - it is often bundled with Realtek Audio drivers (which I updated recently) and that is how it may come back in future if you uninstall it manually.

@comerford commented on GitHub (Aug 23, 2021): Just in case anyone is wondering where Sonic Studio 3 comes from - it is often bundled with Realtek Audio drivers (which I updated recently) and that is how it may come back in future if you uninstall it manually.

claunia commented