starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

How to use Windows ConPTY API from a process whose output has been redirected? #15274

New Issue
Closed
opened 2026-01-31 04:33:24 +00:00 by claunia · 6 comments
claunia commented 2026-01-31 04:33:24 +00:00
Owner
Copy Link

Originally created by @ForNeVeR on GitHub (Sep 19, 2021).

Originally assigned to: @DHowett on GitHub.

I am working on an application which creates a Windows pseudoconsole (using the ConPTY API, according to the official manual which is really good).

It starts a PowerShell session (for test), and shows everything ConPTY sends from that session via the pseudoconsole, and everything works really well. Until I redirect the parent program output to anywhere.

Here's my full code listing for reference.

#include <string>
#include <stdexcept>
#include <iostream>
#include <vector>

#include <Windows.h>

using namespace std::string_literals;

void throwLastError(int number)
{
	throw std::runtime_error("Error "s + std::to_string(number) + ": "s + std::to_string(GetLastError()));
}

STARTUPINFOEXW prepareStartupInformation(HPCON hPCon)
{
	STARTUPINFOEXW startupInfo{sizeof(STARTUPINFOEXW)};
	SIZE_T bytesRequired = 0;
	if (InitializeProcThreadAttributeList(nullptr, 1, 0, &bytesRequired))
		throw std::runtime_error("InitializeProcThreadAttributeList wasn't expected to succeed at that time.");

	const auto threadAttributeList = static_cast<LPPROC_THREAD_ATTRIBUTE_LIST>(calloc(bytesRequired, 1));
	startupInfo.lpAttributeList = threadAttributeList;
	
	if (!InitializeProcThreadAttributeList(threadAttributeList, 1, 0, &bytesRequired))
		throwLastError(4);

	if (!UpdateProcThreadAttribute(
			threadAttributeList,
			0,
			PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE,
			hPCon,
			sizeof(HPCON),
			nullptr,
			nullptr)
	)
		throwLastError(5);

	return startupInfo;
}

PROCESS_INFORMATION startProcess(STARTUPINFOEXW startupInfo, const std::wstring &commandLine)
{
	std::vector<wchar_t> cmdLineBuffer;
	for (auto x : commandLine)
		cmdLineBuffer.push_back(x);
	cmdLineBuffer.push_back(L'\0');
	
	PROCESS_INFORMATION processInfo{nullptr};
	if (!CreateProcessW(
		nullptr,
		cmdLineBuffer.data(),
		nullptr,
		nullptr,
		FALSE,
		EXTENDED_STARTUPINFO_PRESENT,
		nullptr,
		nullptr,
		&startupInfo.StartupInfo,
		&processInfo)
	)
		throwLastError(6);

	return processInfo;
}

int main()
{
	auto h = GetStdHandle(STD_OUTPUT_HANDLE);
	auto type = GetFileType(h);
	std::cout << "STD_OUTPUT_HANDLE " << h << " " << type << std::endl;
	
	try
	{
		HANDLE inPipeRead = nullptr, inPipeWrite = nullptr;
		if (!CreatePipe(&inPipeRead, &inPipeWrite, nullptr, 0))
			throwLastError(1);
	
		HANDLE outPipeRead = nullptr, outPipeWrite = nullptr;
		if (!CreatePipe(&outPipeRead, &outPipeWrite, nullptr, 0))
			throwLastError(2);
	
		HPCON hPCon = nullptr;
		if (CreatePseudoConsole(COORD { 80, 25 }, inPipeRead, outPipeWrite, 0, &hPCon) != S_OK)
			throwLastError(3);

		CloseHandle(inPipeRead);
		CloseHandle(outPipeWrite);
	
		auto startupInfo = prepareStartupInformation(hPCon);

		auto processInfo = startProcess(startupInfo, L"powershell.exe");
	
		std::cout << "PID: " << processInfo.dwProcessId << "\nTID: " << processInfo.dwThreadId << "\n";

		char buffer[1024];
		DWORD readBytes = 0;
		while (ReadFile(
			outPipeRead,
			buffer,
			sizeof buffer - 1,
			&readBytes,
			nullptr
		))
		{
			buffer[readBytes] = 0;
			std::cout << "Read bytes: " << readBytes << "\n";
			std::cout << buffer << "\n";
		}
	}
	catch (const std::runtime_error &x)
	{
		std::cerr << x.what() << "\n";
	}
}

So, if I start this program from terminal or from an IDE, it works (and tells how much data it has read from the PTY pipe). But if I start it with output redirected (e.g. myprogram.exe > C:\Temp\somefile.txt, or even myprogram.exe | out-host in pwsh), then it stops working: PowerShell then somehow inherits the stdin and stdout, doesn't write anything to the PTY pipe and uses the stdout/stdin instead.

For diagnostics, I've added output of GetFileType(GetStdHandle(STD_OUTPUT_HANDLE)), which helps to detect if the output has been redirected (since it's not always obvious in practice). For pristine stdout, it should write 2, and other numbers for other output modes.

How can I overcome this issue? Is it possible for my program to work even if its own stdout/stderr were redirected somewhere? I thought one of the points of PTY was to create a separate isolated environment, which won't interfere with the parent one.

Originally created by @ForNeVeR on GitHub (Sep 19, 2021). Originally assigned to: @DHowett on GitHub. I am working on an application which creates a Windows pseudoconsole (using [the ConPTY API, according to the official manual](https://docs.microsoft.com/en-us/windows/console/creating-a-pseudoconsole-session) which is really good). It starts a PowerShell session (for test), and shows everything ConPTY sends from that session via the pseudoconsole, and everything works really well. Until I redirect the parent program output to anywhere. Here's my full code listing for reference. ```cpp #include <string> #include <stdexcept> #include <iostream> #include <vector> #include <Windows.h> using namespace std::string_literals; void throwLastError(int number) { throw std::runtime_error("Error "s + std::to_string(number) + ": "s + std::to_string(GetLastError())); } STARTUPINFOEXW prepareStartupInformation(HPCON hPCon) { STARTUPINFOEXW startupInfo{sizeof(STARTUPINFOEXW)}; SIZE_T bytesRequired = 0; if (InitializeProcThreadAttributeList(nullptr, 1, 0, &bytesRequired)) throw std::runtime_error("InitializeProcThreadAttributeList wasn't expected to succeed at that time."); const auto threadAttributeList = static_cast<LPPROC_THREAD_ATTRIBUTE_LIST>(calloc(bytesRequired, 1)); startupInfo.lpAttributeList = threadAttributeList; if (!InitializeProcThreadAttributeList(threadAttributeList, 1, 0, &bytesRequired)) throwLastError(4); if (!UpdateProcThreadAttribute( threadAttributeList, 0, PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE, hPCon, sizeof(HPCON), nullptr, nullptr) ) throwLastError(5); return startupInfo; } PROCESS_INFORMATION startProcess(STARTUPINFOEXW startupInfo, const std::wstring &commandLine) { std::vector<wchar_t> cmdLineBuffer; for (auto x : commandLine) cmdLineBuffer.push_back(x); cmdLineBuffer.push_back(L'\0'); PROCESS_INFORMATION processInfo{nullptr}; if (!CreateProcessW( nullptr, cmdLineBuffer.data(), nullptr, nullptr, FALSE, EXTENDED_STARTUPINFO_PRESENT, nullptr, nullptr, &startupInfo.StartupInfo, &processInfo) ) throwLastError(6); return processInfo; } int main() { auto h = GetStdHandle(STD_OUTPUT_HANDLE); auto type = GetFileType(h); std::cout << "STD_OUTPUT_HANDLE " << h << " " << type << std::endl; try { HANDLE inPipeRead = nullptr, inPipeWrite = nullptr; if (!CreatePipe(&inPipeRead, &inPipeWrite, nullptr, 0)) throwLastError(1); HANDLE outPipeRead = nullptr, outPipeWrite = nullptr; if (!CreatePipe(&outPipeRead, &outPipeWrite, nullptr, 0)) throwLastError(2); HPCON hPCon = nullptr; if (CreatePseudoConsole(COORD { 80, 25 }, inPipeRead, outPipeWrite, 0, &hPCon) != S_OK) throwLastError(3); CloseHandle(inPipeRead); CloseHandle(outPipeWrite); auto startupInfo = prepareStartupInformation(hPCon); auto processInfo = startProcess(startupInfo, L"powershell.exe"); std::cout << "PID: " << processInfo.dwProcessId << "\nTID: " << processInfo.dwThreadId << "\n"; char buffer[1024]; DWORD readBytes = 0; while (ReadFile( outPipeRead, buffer, sizeof buffer - 1, &readBytes, nullptr )) { buffer[readBytes] = 0; std::cout << "Read bytes: " << readBytes << "\n"; std::cout << buffer << "\n"; } } catch (const std::runtime_error &x) { std::cerr << x.what() << "\n"; } } ``` So, if I start this program from terminal or from an IDE, it works (and tells how much data it has read from the PTY pipe). But if I start it with output redirected (e.g. `myprogram.exe > C:\Temp\somefile.txt`, or even `myprogram.exe | out-host` in `pwsh`), then it stops working: PowerShell then somehow inherits the stdin and stdout, doesn't write anything to the PTY pipe and uses the stdout/stdin instead. For diagnostics, I've added output of `GetFileType(GetStdHandle(STD_OUTPUT_HANDLE))`, which helps to detect if the output has been redirected (since it's not always obvious in practice). For pristine stdout, it should write `2`, and other numbers for other output modes. How can I overcome this issue? Is it possible for my program to work even if its own stdout/stderr were redirected somewhere? I thought one of the points of PTY was to create a separate isolated environment, which won't interfere with the parent one.
claunia added the Needs-TriageNeeds-Tag-Fix labels 2026-01-31 04:33:24 +00:00
claunia commented 2026-01-31 04:33:25 +00:00
Author
Owner
Copy Link

@ForNeVeR commented on GitHub (Sep 19, 2021):

I've been able to reproduce the same issue even on the test C# GUI application. Steps to reproduce:

  1. Open GUIConsole.WPF.csproj from this repo, and replace <OutputType>WinExe</OutputType> with <OutputType>Exe</OutputType>, to make it a console application with redirectable output (aka /SUBSYSTEM:CONSOLE):
    0a7310dee4/samples/ConPTY/GUIConsole/GUIConsole.WPF/GUIConsole.WPF.csproj (L8)
  2. Compile it and start GUIConsole.WPF.exe, observe that it works well.
  3. Start GUIConsole.WPF.exe > C:\Temp\file.txt, observe how it breaks (no console output, the window is empty).
    image
@ForNeVeR commented on GitHub (Sep 19, 2021): I've been able to reproduce the same issue even on the test C# GUI application. Steps to reproduce: 1. Open `GUIConsole.WPF.csproj` from this repo, and replace `<OutputType>WinExe</OutputType>` with `<OutputType>Exe</OutputType>`, to make it a console application with redirectable output (aka `/SUBSYSTEM:CONSOLE`): https://github.com/microsoft/terminal/blob/0a7310dee4c36f1c423c58b958ffbdbb164ce140/samples/ConPTY/GUIConsole/GUIConsole.WPF/GUIConsole.WPF.csproj#L8 2. Compile it and start `GUIConsole.WPF.exe`, observe that it works well. 3. Start `GUIConsole.WPF.exe > C:\Temp\file.txt`, observe how it breaks (no console output, the window is empty). ![image](https://user-images.githubusercontent.com/92793/133935409-429c69fe-6cc2-4915-9f3d-98a3f8193a0d.png)
claunia commented 2026-01-31 04:33:25 +00:00
Author
Owner
Copy Link

@DHowett commented on GitHub (Sep 20, 2021):

Thanks for expressing appreciation for that document! You know, this is tickling a memory of handle inheritance causing us some significant trouble in the past.

While I go look for the change we made on the OS intended to make this better, could you try setting startupInfo.StartupInfo.dwFlags |= STARTF_USESTDHANDLES?

Enabling the use of hStdIn... et al but leaving them set to null should force the new process to not inherit the redirected default handles.

@DHowett commented on GitHub (Sep 20, 2021): Thanks for expressing appreciation for that document! You know, this is tickling a memory of handle inheritance causing us some significant trouble in the past. While I go look for the change we made on the OS intended to make this better, could you try setting `startupInfo.StartupInfo.dwFlags |= STARTF_USESTDHANDLES`? Enabling the use of `hStdIn...` et al but leaving them set to null should force the new process to *not* inherit the redirected default handles.
claunia commented 2026-01-31 04:33:25 +00:00
Author
Owner
Copy Link

@DHowett commented on GitHub (Sep 20, 2021):

A-ha! We did do some work to improve handle inheritance, but only if the inherited handles were console handles (so: the spawned application would connect to its parent's stdin/stdout even if it was supposed to be using a pseudoconsole)

    //
    // If we are establishing a Pseudoterminal Connection, close
    // any in/out/err handles that may have been copied from the parent.
    // We will re-make new connections to the given PTY instead of
    // using the automatically inherited handles.
    //
        
    if ((ProcessParameters->ConsoleFlags & CONSOLE_USING_PTY_REFERENCE) != 0)
    {
        ConsoleCloseIfConsoleHandle(&ProcessParameters->StandardInput);
        ConsoleCloseIfConsoleHandle(&ProcessParameters->StandardOutput);
        ConsoleCloseIfConsoleHandle(&ProcessParameters->StandardError);
    }

We didn't do the same for all other inherited handles, so... I'm going to file a bug on the docs to make sure we document the need for STARTF_USESTDHANDLES.

@DHowett commented on GitHub (Sep 20, 2021): A-ha! We did do some work to improve handle inheritance, but **only** if the inherited handles were console handles (so: the spawned application would connect to its _parent's_ stdin/stdout even if it was supposed to be using a pseudoconsole) ```c++ // // If we are establishing a Pseudoterminal Connection, close // any in/out/err handles that may have been copied from the parent. // We will re-make new connections to the given PTY instead of // using the automatically inherited handles. // if ((ProcessParameters->ConsoleFlags & CONSOLE_USING_PTY_REFERENCE) != 0) { ConsoleCloseIfConsoleHandle(&ProcessParameters->StandardInput); ConsoleCloseIfConsoleHandle(&ProcessParameters->StandardOutput); ConsoleCloseIfConsoleHandle(&ProcessParameters->StandardError); } ``` We didn't do the same for all other inherited handles, so... I'm going to file a bug on the docs to make sure we document the need for `STARTF_USESTDHANDLES`.
claunia commented 2026-01-31 04:33:26 +00:00
Author
Owner
Copy Link

@eryksun commented on GitHub (Sep 20, 2021):

We didn't do the same for all other inherited handles, so... I'm going to file a bug on the docs to make sure we document the need for STARTF_USESTDHANDLES.

It's not handle inheritance. CreateProcess is called with bInheritHandles as FALSE. What happens is that the standard handles get duplicated to the child, not inherited (i.e. the handle values are not the same). It's an old hack to make spawning console applications magically work when the parent's console is inherited but handle inheritance is disabled, and the standard handles aren't explicitly set. Specifically, this occurs in the following case:

  • the target executable is flagged as a console application
  • inheritance is disabled
  • the startup info doesn't include the flag STARTF_USESTDHANDLES
  • the creation flags do not specify CREATE_NEW_CONSOLE, CREATE_NO_WINDOW, or DETACHED_PROCESS

Maybe I'm missing something, but it seems to me that the pseudoconsole case is like the latter three scenarios (i.e. a new console or no console). If so, then the implementation of NtCreateUserProcess in the kernel should be updated to skip duplicating the parent's standard handles to the child, and to set the standard handle values in the child's ProcessParameters to NULL.

@eryksun commented on GitHub (Sep 20, 2021): > We didn't do the same for all other inherited handles, so... I'm going to file a bug on the docs to make sure we document the need for STARTF_USESTDHANDLES. It's not handle inheritance. `CreateProcess` is called with `bInheritHandles` as `FALSE`. What happens is that the standard handles get duplicated to the child, not inherited (i.e. the handle values are not the same). It's an old hack to make spawning console applications magically work when the parent's console is inherited but handle inheritance is disabled, and the standard handles aren't explicitly set. Specifically, this occurs in the following case: * the target executable is flagged as a console application * inheritance is disabled * the startup info doesn't include the flag `STARTF_USESTDHANDLES` * the creation flags do not specify `CREATE_NEW_CONSOLE`, `CREATE_NO_WINDOW`, or `DETACHED_PROCESS` Maybe I'm missing something, but it seems to me that the pseudoconsole case is like the latter three scenarios (i.e. a new console or no console). If so, then the implementation of `NtCreateUserProcess` in the kernel should be updated to skip duplicating the parent's standard handles to the child, and to set the standard handle values in the child's `ProcessParameters` to `NULL`.
claunia commented 2026-01-31 04:33:26 +00:00
Author
Owner
Copy Link

@ForNeVeR commented on GitHub (Sep 21, 2021):

@DHowett, thanks for your answer!

Yes, your suggestion seems to work. I'll perform some experiments this week to see if it covers my use case, and let you know if there are any additional questions.

For now, I guess we could close the issue? Or would you like for it to stay open until the documentation changes are made?

@ForNeVeR commented on GitHub (Sep 21, 2021): @DHowett, thanks for your answer! Yes, your suggestion seems to work. I'll perform some experiments this week to see if it covers my use case, and let you know if there are any additional questions. For now, I guess we could close the issue? Or would you like for it to stay open until the documentation changes are made?
claunia commented 2026-01-31 04:33:26 +00:00
Author
Owner
Copy Link

@eryksun commented on GitHub (Sep 24, 2021):

Or would you like for it to stay open until the documentation changes are made?

That's a disappointing resolution. If a child process isn't getting spawned with the same console as the parent, the system simply shouldn't duplicate the parent's standard I/O handles to the child. It should work the same way as CREATE_NEW_CONSOLE, CREATE_NO_WINDOW, and DETACHED_PROCESS.

@eryksun commented on GitHub (Sep 24, 2021): > Or would you like for it to stay open until the documentation changes are made? That's a disappointing resolution. If a child process isn't getting spawned with the same console as the parent, the system simply shouldn't duplicate the parent's standard I/O handles to the child. It should work the same way as `CREATE_NEW_CONSOLE`, `CREATE_NO_WINDOW`, and `DETACHED_PROCESS`.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label Needs-Tag-Fix Needs-Triage
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#15274
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?