[PR #10415] Persist inbox conhost; delegate control activities to it via a pipe #28043

Closed
opened 2026-01-31 09:25:56 +00:00 by claunia · 0 comments
Owner

Original Pull Request: https://github.com/microsoft/terminal/pull/10415

State: closed
Merged: Yes


Persist inbox conhost; delegate control activities to it via a pipe

PR Checklist

  • Closes #10194 - WSL Debug Tap doesn't work
  • Closes #10134 - WSL Parameter is Incorrect
  • Closes #10413 - Ctrl+C not passed to client
  • Closes #10414 - Leftover processes on abrupt termination
  • Might help #10251 - Win+X Powershell sometimes fails to attach
  • I work here
  • Manually tested with assorted launch scenarios

Detailed Description of the Pull Request / Additional comments

It turns out that there's a bit of ownership that goes on with the original inbox conhost.exe and the operating system/driver. The PID of that original conhost.exe is stowed when the initial connection is established and it is verified for several activities. This means that the plan of letting it go completely away and having the OpenConsole.exe take over all of its activities must be slightly revised.

I have tested the following two alternatives to keeping conhost.exe around and they do not work:

  1. Replacing the original owner conhost.exe with OpenConsole.exe - A.) The driver does not allow this. Once the owner is registered, it cannot be replaced. B.) There's no way of updating this information inside the client process space and it is kept there too in the kernelbase/conclnt data from its initial connection.
  2. Attempting to pick up the first packet (to determine headed/headless and other initial connection information that we use to determine whether handoff is appropriate or not) prior to registering any owner at all. - The driver doesn't allow this either. The owner must be registered prior to a packet coming through.

Put this mental model in your head:
CMD --> Conhost (inbox) --> OpenConsole (WT Package) --> Terminal (WT Package)

So since the conhost.exe needs to stick around, here's what I'm doing in this PR:

  • conhost.exe in the OS will receive back the OpenConsole.exe process handle on a successful handoff and is expected to remain alive until the OpenConsole.exe exits. It's now waiting on that before it terminates itself.
  • conhost.exe in the OS will establish a signal channel pipe and listen for control commands from OpenConsole.exe in a very similar fashion to how the ConPTY signal pipe operates between the Terminal and the PTY (provided by OpenConsole.exe in this particular example.) When OpenConsole.exe needs to do something that would be verified by the OS and rejected... it will instead signal the original conhost.exe to do that thing and it will go through.
  • conhost.exe will give its own handle through to OpenConsole.exe so it can monitor its lifetime and cleanup. If the owner is gone, the session should end.
  • Assorted handle cleanup that was leading to improper exits. I was confused between .reset() and .release() for some of the wil::unique_any<T> handling and it lead to leaked handles. The leaked handles meant that threads weren't aware of the other sides collapsing and wouldn't cleanup/terminate appropriately.

How does this fix things?

  • For the WSL cases... WSL was specifically looking up the owner PID of the console session from the driver. That was the conhost.exe PID. If it exits, that PID isn't valid and is recycled. Thus the parameter is incorrect or other inappropriate WSL setup behaviors.
  • Ctrl+C not passed... this is a signal the operating system rejects from a PID that is not the owner. This is now relayed through the original owner and it works.
  • Leftover processes... I believe I explained this was both not-enough-monitoring of each others' process lifetimes coupled with mishandling of release/resetting handles and leaking them.
  • Powershell sometimes fails to attach... my theory on this one is that it's a race that became upset when the conhost.exe disappeared while something about Powershell/.NET was still starting, much like the WSL one. I believe now that it is sticking around, it will be fine.

Also, this WILL require an OS update to complete improvement of functionality and I have revised the interface ID. This is considered an acceptable breaking change with no mitigation because we said this feature was an alpha preview.

Validation Steps Performed

  • Launched WSL with defapp set, it works
  • Launched WSL with defapp set and the debug tap on, it works and opens in two tabs
  • Launched CMD, ran ping, did Ctrl+C, it now receives it
  • Launched Win+X powershell a ton of times. It seems fine now
  • Launched cmd, powershell, wsl, etc. Killed assorted processes in the chain (client/conhost/openconsole/windowsterminal) and observed in Process Explorer (with a long delta timer so I could see it) that they all successfully tear down now without leftovers.
**Original Pull Request:** https://github.com/microsoft/terminal/pull/10415 **State:** closed **Merged:** Yes --- Persist inbox conhost; delegate control activities to it via a pipe ## PR Checklist * [x] Closes #10194 - WSL Debug Tap doesn't work * [x] Closes #10134 - WSL Parameter is Incorrect * [x] Closes #10413 - Ctrl+C not passed to client * [x] Closes #10414 - Leftover processes on abrupt termination * [x] Might help #10251 - Win+X Powershell sometimes fails to attach * [x] I work here * [x] Manually tested with assorted launch scenarios ## Detailed Description of the Pull Request / Additional comments It turns out that there's a bit of ownership that goes on with the original inbox `conhost.exe` and the operating system/driver. The PID of that original `conhost.exe` is stowed when the initial connection is established and it is verified for several activities. This means that the plan of letting it go completely away and having the `OpenConsole.exe` take over all of its activities must be slightly revised. I have tested the following two alternatives to keeping `conhost.exe` around and they do not work: 1. Replacing the original owner `conhost.exe` with `OpenConsole.exe` - A.) The driver does not allow this. Once the owner is registered, it cannot be replaced. B.) There's no way of updating this information inside the client process space and it is kept there too in the `kernelbase`/`conclnt` data from its initial connection. 2. Attempting to pick up the first packet (to determine headed/headless and other initial connection information that we use to determine whether handoff is appropriate or not) prior to registering any owner at all. - The driver doesn't allow this either. The owner must be registered prior to a packet coming through. Put this mental model in your head: CMD --> Conhost (inbox) --> OpenConsole (WT Package) --> Terminal (WT Package) So since the `conhost.exe` needs to stick around, here's what I'm doing in this PR: - `conhost.exe` in the OS will receive back the `OpenConsole.exe` process handle on a successful handoff and is expected to remain alive until the `OpenConsole.exe` exits. It's now waiting on that before it terminates itself. - `conhost.exe` in the OS will establish a signal channel pipe and listen for control commands from `OpenConsole.exe` in a very similar fashion to how the `ConPTY` signal pipe operates between the Terminal and the PTY (provided by `OpenConsole.exe` in this particular example.) When `OpenConsole.exe` needs to do something that would be verified by the OS and rejected... it will instead signal the original `conhost.exe` to do that thing and it will go through. - `conhost.exe` will give its own handle through to `OpenConsole.exe` so it can monitor its lifetime and cleanup. If the owner is gone, the session should end. - Assorted handle cleanup that was leading to improper exits. I was confused between `.reset()` and `.release()` for some of the `wil::unique_any<T>` handling and it lead to leaked handles. The leaked handles meant that threads weren't aware of the other sides collapsing and wouldn't cleanup/terminate appropriately. How does this fix things? - For the WSL cases... WSL was specifically looking up the owner PID of the console session from the driver. That was the `conhost.exe` PID. If it exits, that PID isn't valid and is recycled. Thus the parameter is incorrect or other inappropriate WSL setup behaviors. - Ctrl+C not passed... this is a signal the operating system rejects from a PID that is not the owner. This is now relayed through the original owner and it works. - Leftover processes... I believe I explained this was both not-enough-monitoring of each others' process lifetimes coupled with mishandling of release/resetting handles and leaking them. - Powershell sometimes fails to attach... my theory on this one is that it's a race that became upset when the `conhost.exe` disappeared while something about Powershell/.NET was still starting, much like the WSL one. I believe now that it is sticking around, it will be fine. Also, this WILL require an OS update to complete improvement of functionality and I have revised the interface ID. This is considered an acceptable breaking change with no mitigation because we said this feature was an alpha preview. ## Validation Steps Performed - Launched WSL with defapp set, it works - Launched WSL with defapp set and the debug tap on, it works and opens in two tabs - Launched CMD, ran ping, did Ctrl+C, it now receives it - Launched Win+X powershell a ton of times. It seems fine now - Launched cmd, powershell, wsl, etc. Killed assorted processes in the chain (client/conhost/openconsole/windowsterminal) and observed in Process Explorer (with a long delta timer so I could see it) that they all successfully tear down now without leftovers.
claunia added the pull-request label 2026-01-31 09:25:56 +00:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#28043