AllocConsole failing with ERROR_GEN_FAILURE as a service in rare cases #557

Open
opened 2026-01-30 21:55:13 +00:00 by claunia · 5 comments
Owner

Originally created by @tsmortenson on GitHub (Feb 13, 2019).

We have a software package running on thousands of Windows machines from XP onward. In 2 cases we have received a bug report that the users application fails to call AllocConsole with error code ERROR_GEN_FAILURE. This has been seen once on a Windows Server 2008 system and the second on a Windows Server 2012 system. In both cases the error was encountered while running as a Service.
The error message is "A device attached to the system is not functioning. (0x1f)"

Unfortunately we have not been able to reproduce this on any other systems. It happens 100% of the time of the failing system however.

I was able to find any documentation that shows this as a possible error.
Any thoughts on this would be appreciated.

Originally created by @tsmortenson on GitHub (Feb 13, 2019). We have a software package running on thousands of Windows machines from XP onward. In 2 cases we have received a bug report that the users application fails to call AllocConsole with error code ERROR_GEN_FAILURE. This has been seen once on a Windows Server 2008 system and the second on a Windows Server 2012 system. In both cases the error was encountered while running as a Service. The error message is "A device attached to the system is not functioning. (0x1f)" Unfortunately we have not been able to reproduce this on any other systems. It happens 100% of the time of the failing system however. I was able to find any documentation that shows this as a possible error. Any thoughts on this would be appreciated.
claunia added the Product-ConhostIssue-BugArea-Server labels 2026-01-30 21:55:13 +00:00
Author
Owner

@HBelusca commented on GitHub (Feb 14, 2019):

Does the problem also happen if the service is an "interactive service" (services.msc -> select your service in the list -> Properties -> "Log On" tab -> "Allow service to interact with desktop") ?

@HBelusca commented on GitHub (Feb 14, 2019): Does the problem also happen if the service is an "interactive service" (services.msc -> select your service in the list -> Properties -> "Log On" tab -> "Allow service to interact with desktop") ?
Author
Owner

@tsmortenson commented on GitHub (Feb 15, 2019):

Thank you for your reply. We will see if we can check that.
Unfortunately this is on a system that is a customer of a customer. So it is difficult to make a lot of requests.
I was hoping that someone familiar with the source might have an idea. My guess is that some other api call within the AllocConsole call is causing the error.

@tsmortenson commented on GitHub (Feb 15, 2019): Thank you for your reply. We will see if we can check that. Unfortunately this is on a system that is a customer of a customer. So it is difficult to make a lot of requests. I was hoping that someone familiar with the source might have an idea. My guess is that some other api call within the AllocConsole call is causing the error.
Author
Owner

@tsmortenson commented on GitHub (Feb 15, 2019):

I reread your question. As I said, I can't easily check exactly what you asked. But we do have the configuration of our program from that machine. And it was setup in a way that when the service was registered, it was to NOT be interactive. So quite sure that will be the case of looking on the properties page.
We have run this way on many other servers and they all work fine in this way.
We are allocating a console even though it is not visible to allow us to send a CTRL-BREAK signal to the the child process which shares the console. This is only possible if the console exists.

@tsmortenson commented on GitHub (Feb 15, 2019): I reread your question. As I said, I can't easily check exactly what you asked. But we do have the configuration of our program from that machine. And it was setup in a way that when the service was registered, it was to NOT be interactive. So quite sure that will be the case of looking on the properties page. We have run this way on many other servers and they all work fine in this way. We are allocating a console even though it is not visible to allow us to send a CTRL-BREAK signal to the the child process which shares the console. This is only possible if the console exists.
Author
Owner

@HBelusca commented on GitHub (Feb 15, 2019):

Note that my "suggestion" is in no way a fix. It's just something so that it may better indicate where the actual problem is.

@HBelusca commented on GitHub (Feb 15, 2019): Note that my "suggestion" is in no way a fix. It's just something so that it may better indicate where the actual problem is.
Author
Owner

@tsmortenson commented on GitHub (Mar 4, 2019):

We were able to run a test program on the customer machine to try a few things out.

One test was to have our service create a child cmd.exe process, then call AttachConsole to get that child's console. This works on all of our test machines, but on this machine the CreateProcess call with console on cmd.exe fails.

One idea is that something in the customer's Group Policy might be preventing a Service from creating a console? I was not able to locate any such thing with Google. But does something like that exist?

@tsmortenson commented on GitHub (Mar 4, 2019): We were able to run a test program on the customer machine to try a few things out. One test was to have our service create a child cmd.exe process, then call AttachConsole to get that child's console. This works on all of our test machines, but on this machine the CreateProcess call with console on cmd.exe fails. One idea is that something in the customer's Group Policy might be preventing a Service from creating a console? I was not able to locate any such thing with Google. But does something like that exist?
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#557