.NET Console.WriteLine throws 'The handle is invalid.' on large output when using ENABLE_VIRTUAL_TERMINAL_PROCESSING #5277

Open
opened 2026-01-31 00:09:28 +00:00 by claunia · 0 comments
Owner

Originally created by @gerardog on GitHub (Nov 28, 2019).

Environment

Windows build number: 10.0.18362.356
Windows Terminal version (if applicable): Using windows conHost or Terminal 0.7.3291.0
Visual Studio 2019 Professional 16.3.7
Net Framework 4.6, 4.6.1 or 4.7.2

Steps to reproduce

Create a simple C# Process that set console mode ENABLE_VIRTUAL_TERMINAL_PROCESSING, and then writes unlimited output to StdOut.
Eventually StdOut gets broken and throws System.IO.IOException: The handle is invalid.

C# Repro code:

using Microsoft.Win32.SafeHandles;
using System;
using System.Runtime.InteropServices;

class Program
{
    static void Main(string[] args)
    {
        if (SetConsoleModeVT())
        {
            for (int i = 0; ; i++)
            {
                try
                {
                    Console.WriteLine(i);
                }
                catch (Exception ex)
                {
                    Console.Error.WriteLine(ex.ToString());
                    Console.ReadLine();
                }
            }
        }
    }

    private static bool SetConsoleModeVT()
    {
        var hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
        if (!GetConsoleMode(hStdOut, out uint outConsoleMode))
        {
            return false;
        }

        outConsoleMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
        if (!SetConsoleMode(hStdOut, outConsoleMode))
        {
            return false;
        }

        return true;
    }

    internal const int STD_OUTPUT_HANDLE = -11;
    internal const uint ENABLE_VIRTUAL_TERMINAL_PROCESSING = 0x0004;

    [DllImport("kernel32.dll", SetLastError = true)]
    internal static extern SafeFileHandle GetStdHandle(int nStdHandle);

    [DllImport("kernel32.dll", SetLastError = true)]
    internal static extern bool SetConsoleMode(SafeFileHandle hConsoleHandle, uint mode);

    [DllImport("kernel32.dll", SetLastError = true)]
    internal static extern bool GetConsoleMode(SafeFileHandle handle, out uint mode);
}

Expected behavior

The app should not throw IOException.

Actual behavior

This is the output:

1
2
(...)
163543
163544
163545
System.IO.IOException: The handle is invalid.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.__ConsoleStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.IO.StreamWriter.Write(String value)
   at System.IO.TextWriter.Write(UInt64 value)
   at System.IO.TextWriter.WriteLine(UInt64 value)
   at System.IO.TextWriter.SyncTextWriter.WriteLine(UInt64 value)
   at System.Console.WriteLine(UInt64 value)
   at ConsoleApp5.Program.Main(String[] args) in C:\Users\Me\source\repos\ConsoleApp5\Program.cs:line 17

The exception is only thrown with ENABLE_VIRTUAL_TERMINAL_PROCESSING console mode.
Either launching the app from a Windows CMD/ConHost, or the new Windows Terminal/PowerShell.

Thanks!

Originally created by @gerardog on GitHub (Nov 28, 2019). <!-- 🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨 I ACKNOWLEDGE THE FOLLOWING BEFORE PROCEEDING: 1. If I delete this entire template and go my own path, the core team may close my issue without further explanation or engagement. 2. If I list multiple bugs/concerns in this one issue, the core team may close my issue without further explanation or engagement. 3. If I write an issue that has many duplicates, the core team may close my issue without further explanation or engagement (and without necessarily spending time to find the exact duplicate ID number). 4. If I leave the title incomplete when filing the issue, the core team may close my issue without further explanation or engagement. 5. If I file something completely blank in the body, the core team may close my issue without further explanation or engagement. All good? Then proceed! --> <!-- This bug tracker is monitored by Windows Terminal development team and other technical folks. **Important: When reporting BSODs or security issues, DO NOT attach memory dumps, logs, or traces to Github issues**. Instead, send dumps/traces to secure@microsoft.com, referencing this GitHub issue. If this is an application crash, please also provide a Feedback Hub submission link so we can find your diagnostic data on the backend. Use the category "Apps > Windows Terminal (Preview)" and choose "Share My Feedback" after submission to get the link. Please use this form and describe your issue, concisely but precisely, with as much detail as possible. --> # Environment ```none Windows build number: 10.0.18362.356 Windows Terminal version (if applicable): Using windows conHost or Terminal 0.7.3291.0 Visual Studio 2019 Professional 16.3.7 Net Framework 4.6, 4.6.1 or 4.7.2 ``` # Steps to reproduce Create a simple C# Process that set console mode ENABLE_VIRTUAL_TERMINAL_PROCESSING, and then writes unlimited output to StdOut. Eventually StdOut gets broken and throws System.IO.IOException: The handle is invalid. C# Repro code: ``` csharp using Microsoft.Win32.SafeHandles; using System; using System.Runtime.InteropServices; class Program { static void Main(string[] args) { if (SetConsoleModeVT()) { for (int i = 0; ; i++) { try { Console.WriteLine(i); } catch (Exception ex) { Console.Error.WriteLine(ex.ToString()); Console.ReadLine(); } } } } private static bool SetConsoleModeVT() { var hStdOut = GetStdHandle(STD_OUTPUT_HANDLE); if (!GetConsoleMode(hStdOut, out uint outConsoleMode)) { return false; } outConsoleMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING; if (!SetConsoleMode(hStdOut, outConsoleMode)) { return false; } return true; } internal const int STD_OUTPUT_HANDLE = -11; internal const uint ENABLE_VIRTUAL_TERMINAL_PROCESSING = 0x0004; [DllImport("kernel32.dll", SetLastError = true)] internal static extern SafeFileHandle GetStdHandle(int nStdHandle); [DllImport("kernel32.dll", SetLastError = true)] internal static extern bool SetConsoleMode(SafeFileHandle hConsoleHandle, uint mode); [DllImport("kernel32.dll", SetLastError = true)] internal static extern bool GetConsoleMode(SafeFileHandle handle, out uint mode); } ``` # Expected behavior The app should not throw IOException. # Actual behavior This is the output: ``` 1 2 (...) 163543 163544 163545 System.IO.IOException: The handle is invalid. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.__ConsoleStream.Write(Byte[] buffer, Int32 offset, Int32 count) at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder) at System.IO.StreamWriter.Write(String value) at System.IO.TextWriter.Write(UInt64 value) at System.IO.TextWriter.WriteLine(UInt64 value) at System.IO.TextWriter.SyncTextWriter.WriteLine(UInt64 value) at System.Console.WriteLine(UInt64 value) at ConsoleApp5.Program.Main(String[] args) in C:\Users\Me\source\repos\ConsoleApp5\Program.cs:line 17 ``` The exception is only thrown with ENABLE_VIRTUAL_TERMINAL_PROCESSING console mode. Either launching the app from a Windows CMD/ConHost, or the new Windows Terminal/PowerShell. Thanks!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#5277