How to use Windows ConPTY API from a process whose output has been redirected? #15271

Open
opened 2026-01-31 04:33:21 +00:00 by claunia · 0 comments
Owner

Originally created by @ForNeVeR on GitHub (Sep 19, 2021).

Originally assigned to: @DHowett on GitHub.

I am working on an application which creates a Windows pseudoconsole (using the ConPTY API, according to the official manual which is really good).

It starts a PowerShell session (for test), and shows everything ConPTY sends from that session via the pseudoconsole, and everything works really well. Until I redirect the parent program output to anywhere.

Here's my full code listing for reference.

#include <string>
#include <stdexcept>
#include <iostream>
#include <vector>

#include <Windows.h>

using namespace std::string_literals;

void throwLastError(int number)
{
	throw std::runtime_error("Error "s + std::to_string(number) + ": "s + std::to_string(GetLastError()));
}

STARTUPINFOEXW prepareStartupInformation(HPCON hPCon)
{
	STARTUPINFOEXW startupInfo{sizeof(STARTUPINFOEXW)};
	SIZE_T bytesRequired = 0;
	if (InitializeProcThreadAttributeList(nullptr, 1, 0, &bytesRequired))
		throw std::runtime_error("InitializeProcThreadAttributeList wasn't expected to succeed at that time.");

	const auto threadAttributeList = static_cast<LPPROC_THREAD_ATTRIBUTE_LIST>(calloc(bytesRequired, 1));
	startupInfo.lpAttributeList = threadAttributeList;
	
	if (!InitializeProcThreadAttributeList(threadAttributeList, 1, 0, &bytesRequired))
		throwLastError(4);

	if (!UpdateProcThreadAttribute(
			threadAttributeList,
			0,
			PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE,
			hPCon,
			sizeof(HPCON),
			nullptr,
			nullptr)
	)
		throwLastError(5);

	return startupInfo;
}

PROCESS_INFORMATION startProcess(STARTUPINFOEXW startupInfo, const std::wstring &commandLine)
{
	std::vector<wchar_t> cmdLineBuffer;
	for (auto x : commandLine)
		cmdLineBuffer.push_back(x);
	cmdLineBuffer.push_back(L'\0');
	
	PROCESS_INFORMATION processInfo{nullptr};
	if (!CreateProcessW(
		nullptr,
		cmdLineBuffer.data(),
		nullptr,
		nullptr,
		FALSE,
		EXTENDED_STARTUPINFO_PRESENT,
		nullptr,
		nullptr,
		&startupInfo.StartupInfo,
		&processInfo)
	)
		throwLastError(6);

	return processInfo;
}

int main()
{
	auto h = GetStdHandle(STD_OUTPUT_HANDLE);
	auto type = GetFileType(h);
	std::cout << "STD_OUTPUT_HANDLE " << h << " " << type << std::endl;
	
	try
	{
		HANDLE inPipeRead = nullptr, inPipeWrite = nullptr;
		if (!CreatePipe(&inPipeRead, &inPipeWrite, nullptr, 0))
			throwLastError(1);
	
		HANDLE outPipeRead = nullptr, outPipeWrite = nullptr;
		if (!CreatePipe(&outPipeRead, &outPipeWrite, nullptr, 0))
			throwLastError(2);
	
		HPCON hPCon = nullptr;
		if (CreatePseudoConsole(COORD { 80, 25 }, inPipeRead, outPipeWrite, 0, &hPCon) != S_OK)
			throwLastError(3);

		CloseHandle(inPipeRead);
		CloseHandle(outPipeWrite);
	
		auto startupInfo = prepareStartupInformation(hPCon);

		auto processInfo = startProcess(startupInfo, L"powershell.exe");
	
		std::cout << "PID: " << processInfo.dwProcessId << "\nTID: " << processInfo.dwThreadId << "\n";

		char buffer[1024];
		DWORD readBytes = 0;
		while (ReadFile(
			outPipeRead,
			buffer,
			sizeof buffer - 1,
			&readBytes,
			nullptr
		))
		{
			buffer[readBytes] = 0;
			std::cout << "Read bytes: " << readBytes << "\n";
			std::cout << buffer << "\n";
		}
	}
	catch (const std::runtime_error &x)
	{
		std::cerr << x.what() << "\n";
	}
}

So, if I start this program from terminal or from an IDE, it works (and tells how much data it has read from the PTY pipe). But if I start it with output redirected (e.g. myprogram.exe > C:\Temp\somefile.txt, or even myprogram.exe | out-host in pwsh), then it stops working: PowerShell then somehow inherits the stdin and stdout, doesn't write anything to the PTY pipe and uses the stdout/stdin instead.

For diagnostics, I've added output of GetFileType(GetStdHandle(STD_OUTPUT_HANDLE)), which helps to detect if the output has been redirected (since it's not always obvious in practice). For pristine stdout, it should write 2, and other numbers for other output modes.

How can I overcome this issue? Is it possible for my program to work even if its own stdout/stderr were redirected somewhere? I thought one of the points of PTY was to create a separate isolated environment, which won't interfere with the parent one.

Originally created by @ForNeVeR on GitHub (Sep 19, 2021). Originally assigned to: @DHowett on GitHub. I am working on an application which creates a Windows pseudoconsole (using [the ConPTY API, according to the official manual](https://docs.microsoft.com/en-us/windows/console/creating-a-pseudoconsole-session) which is really good). It starts a PowerShell session (for test), and shows everything ConPTY sends from that session via the pseudoconsole, and everything works really well. Until I redirect the parent program output to anywhere. Here's my full code listing for reference. ```cpp #include <string> #include <stdexcept> #include <iostream> #include <vector> #include <Windows.h> using namespace std::string_literals; void throwLastError(int number) { throw std::runtime_error("Error "s + std::to_string(number) + ": "s + std::to_string(GetLastError())); } STARTUPINFOEXW prepareStartupInformation(HPCON hPCon) { STARTUPINFOEXW startupInfo{sizeof(STARTUPINFOEXW)}; SIZE_T bytesRequired = 0; if (InitializeProcThreadAttributeList(nullptr, 1, 0, &bytesRequired)) throw std::runtime_error("InitializeProcThreadAttributeList wasn't expected to succeed at that time."); const auto threadAttributeList = static_cast<LPPROC_THREAD_ATTRIBUTE_LIST>(calloc(bytesRequired, 1)); startupInfo.lpAttributeList = threadAttributeList; if (!InitializeProcThreadAttributeList(threadAttributeList, 1, 0, &bytesRequired)) throwLastError(4); if (!UpdateProcThreadAttribute( threadAttributeList, 0, PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE, hPCon, sizeof(HPCON), nullptr, nullptr) ) throwLastError(5); return startupInfo; } PROCESS_INFORMATION startProcess(STARTUPINFOEXW startupInfo, const std::wstring &commandLine) { std::vector<wchar_t> cmdLineBuffer; for (auto x : commandLine) cmdLineBuffer.push_back(x); cmdLineBuffer.push_back(L'\0'); PROCESS_INFORMATION processInfo{nullptr}; if (!CreateProcessW( nullptr, cmdLineBuffer.data(), nullptr, nullptr, FALSE, EXTENDED_STARTUPINFO_PRESENT, nullptr, nullptr, &startupInfo.StartupInfo, &processInfo) ) throwLastError(6); return processInfo; } int main() { auto h = GetStdHandle(STD_OUTPUT_HANDLE); auto type = GetFileType(h); std::cout << "STD_OUTPUT_HANDLE " << h << " " << type << std::endl; try { HANDLE inPipeRead = nullptr, inPipeWrite = nullptr; if (!CreatePipe(&inPipeRead, &inPipeWrite, nullptr, 0)) throwLastError(1); HANDLE outPipeRead = nullptr, outPipeWrite = nullptr; if (!CreatePipe(&outPipeRead, &outPipeWrite, nullptr, 0)) throwLastError(2); HPCON hPCon = nullptr; if (CreatePseudoConsole(COORD { 80, 25 }, inPipeRead, outPipeWrite, 0, &hPCon) != S_OK) throwLastError(3); CloseHandle(inPipeRead); CloseHandle(outPipeWrite); auto startupInfo = prepareStartupInformation(hPCon); auto processInfo = startProcess(startupInfo, L"powershell.exe"); std::cout << "PID: " << processInfo.dwProcessId << "\nTID: " << processInfo.dwThreadId << "\n"; char buffer[1024]; DWORD readBytes = 0; while (ReadFile( outPipeRead, buffer, sizeof buffer - 1, &readBytes, nullptr )) { buffer[readBytes] = 0; std::cout << "Read bytes: " << readBytes << "\n"; std::cout << buffer << "\n"; } } catch (const std::runtime_error &x) { std::cerr << x.what() << "\n"; } } ``` So, if I start this program from terminal or from an IDE, it works (and tells how much data it has read from the PTY pipe). But if I start it with output redirected (e.g. `myprogram.exe > C:\Temp\somefile.txt`, or even `myprogram.exe | out-host` in `pwsh`), then it stops working: PowerShell then somehow inherits the stdin and stdout, doesn't write anything to the PTY pipe and uses the stdout/stdin instead. For diagnostics, I've added output of `GetFileType(GetStdHandle(STD_OUTPUT_HANDLE))`, which helps to detect if the output has been redirected (since it's not always obvious in practice). For pristine stdout, it should write `2`, and other numbers for other output modes. How can I overcome this issue? Is it possible for my program to work even if its own stdout/stderr were redirected somewhere? I thought one of the points of PTY was to create a separate isolated environment, which won't interfere with the parent one.
claunia added the Needs-TriageNeeds-Tag-Fix labels 2026-01-31 04:33:21 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#15271