starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-06 05:34:59 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

Remove TranslateUnicodeToOem and all related code (#14745)

Browse Source 
The overarching intention of this PR is to improve our Unicode support.
Most
of our APIs still don't support anything beyond UCS-2 and DBCS
sequences.
This commit doesn't fix the UTF-16 support (by supporting surrogate
pairs),
but it does improve support for UTF-8 by allowing longer `char`
sequences.

It does so by removing `TranslateUnicodeToOem` which seems to have had
an almost
viral effect on code quality wherever it was used. It made the
assumption that
_all_ narrow glyphs encode to 1 `char` and most wide glyphs to 2
`char`s.
It also didn't bother to check whether `WideCharToMultiByte` failed or
returned
a different amount of `char`s. So up until now it was easily possible to
read
uninitialized stack memory from conhost. Any code that used this
function was
forced to do the same "measurement" of narrow/wide glyphs, because _of
course_
it didn't had any way to indicate to the caller how much memory it needs
to
store the result. Instead all callers were forced to sorta replicate how
it
worked to calculate the required storage ahead of time.
Unsurprisingly, none of the callers used the same algorithm...

Without it the code is much leaner and easier to understand now. The
best
example is `COOKED_READ_DATA::_handlePostCharInputLoop` which used to
contain 3
blocks of _almost_ identical code, but with ever so subtle differences.
After
reading the old code for hours I still don't know if they were relevant
or not.
It used to be 200 lines of code lacking any documentation and it's now
50 lines
with descriptive function names. I hope this doesn't break anything, but
to
be honest I can't imagine anyone having relied on this mess in the first
place.

I needed some helpers to handle byte slices (`std::span<char>`), which
is why
a new `til/bytes.h` header was added. Initially I wrote a `buf_writer`
class
but felt like such a wrapper around a slice/span was annoying to use.
As such I've opted for freestanding functions which take slices as
mutable
references and "advance" them (offset the start) whenever they're read
from
or written to. I'm not particularly happy with the design but they do
the job.

Related to #8000
Fixes #4551
Fixes #7589
Fixes #8663

## Validation Steps Performed
* Unit and feature tests ✅
* Far Manager ✅
* Fixes test cases in #4551, #7589 and #8663 ✅
This commit is contained in:
Leonard Hecker
2023-02-28 21:55:18 +01:00
committed by GitHub
parent cf87590b31
commit 599b550817
23 changed files with 608 additions and 1229 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
src/host/dbcs.cpp
View File

@@ -177,77 +177,3 @@ BOOL IsAvailableEastAsianCodePage(const UINT uiCodePage)
return false;
}
}
_Ret_range_(0, cbAnsi)
ULONG TranslateUnicodeToOem(_In_reads_(cchUnicode) PCWCHAR pwchUnicode,
const ULONG cchUnicode,
_Out_writes_bytes_(cbAnsi) PCHAR pchAnsi,
const ULONG cbAnsi,
_Out_ std::unique_ptr<IInputEvent>& partialEvent)
{
const auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
const auto TmpUni = new (std::nothrow) WCHAR[cchUnicode];
if (TmpUni == nullptr)
{
return 0;
}
memcpy(TmpUni, pwchUnicode, cchUnicode * sizeof(WCHAR));
CHAR AsciiDbcs[2];
AsciiDbcs[1] = 0;
ULONG i, j;
for (i = 0, j = 0; i < cchUnicode && j < cbAnsi; i++, j++)
{
if (IsGlyphFullWidth(TmpUni[i]))
{
const auto NumBytes = sizeof(AsciiDbcs);
ConvertToOem(gci.CP, &TmpUni[i], 1, (LPSTR)&AsciiDbcs[0], NumBytes);
if (IsDBCSLeadByteConsole(AsciiDbcs[0], &gci.CPInfo))
{
if (j < cbAnsi - 1)
{ // -1 is safe DBCS in buffer
pchAnsi[j] = AsciiDbcs[0];
j++;
pchAnsi[j] = AsciiDbcs[1];
AsciiDbcs[1] = 0;
}
else
{
pchAnsi[j] = AsciiDbcs[0];
break;
}
}
else
{
pchAnsi[j] = AsciiDbcs[0];
AsciiDbcs[1] = 0;
}
}
else
{
ConvertToOem(gci.CP, &TmpUni[i], 1, &pchAnsi[j], 1);
}
}
if (AsciiDbcs[1])
{
try
{
auto keyEvent = std::make_unique<KeyEvent>();
if (keyEvent.get())
{
keyEvent->SetCharData(AsciiDbcs[1]);
partialEvent.reset(static_cast<IInputEvent* const>(keyEvent.release()));
}
}
catch (...)
{
LOG_HR(wil::ResultFromCaughtException());
}
}
delete[] TmpUni;
return j;
}

7
src/host/dbcs.h
View File

@@ -29,10 +29,3 @@ bool IsDBCSLeadByteConsole(const CHAR ch, const CPINFO* const pCPInfo);
BYTE CodePageToCharSet(const UINT uiCodePage);
BOOL IsAvailableEastAsianCodePage(const UINT uiCodePage);
_Ret_range_(0, cbAnsi)
ULONG TranslateUnicodeToOem(_In_reads_(cchUnicode) PCWCHAR pwchUnicode,
const ULONG cchUnicode,
_Out_writes_bytes_(cbAnsi) PCHAR pchAnsi,
const ULONG cbAnsi,
_Out_ std::unique_ptr<IInputEvent>& partialEvent);

68
src/host/directio.cpp
View File

@@ -169,75 +169,21 @@ void EventsToUnicode(_Inout_ std::deque<std::unique_ptr<IInputEvent>>& inEvents,
LockConsole();
auto Unlock = wil::scope_exit([&] { UnlockConsole(); });
std::deque<std::unique_ptr<IInputEvent>> partialEvents;
if (!IsUnicode)
{
if (inputBuffer.IsReadPartialByteSequenceAvailable())
{
partialEvents.push_back(inputBuffer.FetchReadPartialByteSequence(IsPeek));
}
}
size_t amountToRead;
if (FAILED(SizeTSub(eventReadCount, partialEvents.size(), &amountToRead)))
{
return STATUS_INTEGER_OVERFLOW;
}
std::deque<std::unique_ptr<IInputEvent>> readEvents;
auto Status = inputBuffer.Read(readEvents,
amountToRead,
IsPeek,
true,
IsUnicode,
false);
const auto Status = inputBuffer.Read(outEvents,
eventReadCount,
IsPeek,
true,
IsUnicode,
false);
if (CONSOLE_STATUS_WAIT == Status)
{
FAIL_FAST_IF(!(readEvents.empty()));
// If we're told to wait until later, move all of our context
// to the read data object and send it back up to the server.
waiter = std::make_unique<DirectReadData>(&inputBuffer,
&readHandleState,
eventReadCount,
std::move(partialEvents));
}
else if (SUCCEEDED_NTSTATUS(Status))
{
// split key events to oem chars if necessary
if (!IsUnicode)
{
try
{
SplitToOem(readEvents);
}
CATCH_LOG();
}
// combine partial and readEvents
while (!partialEvents.empty())
{
readEvents.push_front(std::move(partialEvents.back()));
partialEvents.pop_back();
}
// move events over
for (size_t i = 0; i < eventReadCount; ++i)
{
if (readEvents.empty())
{
break;
}
outEvents.push_back(std::move(readEvents.front()));
readEvents.pop_front();
}
// store partial event if necessary
if (!readEvents.empty())
{
inputBuffer.StoreReadPartialByteSequence(std::move(readEvents.front()));
readEvents.pop_front();
FAIL_FAST_IF(!(readEvents.empty()));
}
std::move(outEvents));
}
return Status;
}

465
src/host/inputBuffer.cpp
View File

@@ -3,12 +3,13 @@
#include "precomp.h"
#include "inputBuffer.hpp"
#include "dbcs.h"
#include "stream.h"
#include "../types/inc/GlyphWidth.hpp"
#include <functional>
#include <til/bytes.h>
#include "misc.h"
#include "../interactivity/inc/ServiceLocator.hpp"
#define INPUT_BUFFER_DEFAULT_INPUT_MODE (ENABLE_LINE_INPUT | ENABLE_PROCESSED_INPUT | ENABLE_ECHO_INPUT | ENABLE_MOUSE_INPUT)
@@ -36,59 +37,199 @@ InputBuffer::InputBuffer() :
fInComposition = false;
}
// Routine Description:
// - This routine frees the resources associated with an input buffer.
// Arguments:
// - None
// Return Value:
InputBuffer::~InputBuffer() = default;
// Routine Description:
// - checks if any partial char data is available for reading operation
// Arguments:
// - None
// Return Value:
// - true if partial char data is available, false otherwise
bool InputBuffer::IsReadPartialByteSequenceAvailable()
// Transfer as many `wchar_t`s from source over to the `char`/`wchar_t` buffer `target`. After it returns,
// the start of the `source` and `target` slices will be offset by as many bytes as have been copied
// over, so that if you call this function again it'll continue copying from wherever it left off.
//
// It performs the necessary `WideCharToMultiByte` conversion if `isUnicode` is `false`.
// Since not all converted `char`s might fit into `target` it'll cache the remainder. The next
// time this function is called those cached `char`s will then be the first to be copied over.
void InputBuffer::Consume(bool isUnicode, std::wstring_view& source, std::span<char>& target)
{
return _readPartialByteSequence.get() != nullptr;
}
// `_cachedTextReaderA` might still contain target data from a previous invocation.
// `ConsumeCached` calls `_switchReadingMode` for us.
ConsumeCached(isUnicode, target);
// Routine Description:
// - reads any read partial char data available
// Arguments:
// - peek - if true, data will not be removed after being fetched
// Return Value:
// - the partial char data. may be nullptr if no data is available
std::unique_ptr<IInputEvent> InputBuffer::FetchReadPartialByteSequence(_In_ bool peek)
{
if (!IsReadPartialByteSequenceAvailable())
if (source.empty() || target.empty())
{
return std::unique_ptr<IInputEvent>();
return;
}
if (peek)
if (isUnicode)
{
return IInputEvent::Create(_readPartialByteSequence->ToInputRecord());
// The above block should either leave `target` or `_cachedTextReaderW` empty (or both).
// If we're here, `_cachedTextReaderW` should be empty.
assert(_cachedTextReaderW.empty());
til::bytes_transfer(target, source);
}
else
{
std::unique_ptr<IInputEvent> outEvent;
outEvent.swap(_readPartialByteSequence);
return outEvent;
// The above block should either leave `target` or `_cachedTextReaderA` empty (or both).
// If we're here, `_cachedTextReaderA` should be empty.
assert(_cachedTextReaderA.empty());
const auto cp = ServiceLocator::LocateGlobals().getConsoleInformation().CP;
// Fast path: Batch convert all data in case the user provided buffer is large enough.
{
const auto wideLength = gsl::narrow<ULONG>(source.size());
const auto narrowLength = gsl::narrow<ULONG>(target.size());
const auto length = WideCharToMultiByte(cp, 0, source.data(), wideLength, target.data(), narrowLength, nullptr, nullptr);
if (length > 0)
{
source = {};
til::bytes_advance(target, gsl::narrow_cast<size_t>(length));
return;
}
const auto error = GetLastError();
THROW_HR_IF(HRESULT_FROM_WIN32(error), error != ERROR_INSUFFICIENT_BUFFER);
}
// Slow path: Character-wise conversion otherwise. We do this in order to only
// consume as many characters from `source` as necessary to fill `target`.
{
size_t read = 0;
for (const auto& wch : source)
{
char buffer[8];
const auto length = WideCharToMultiByte(cp, 0, &wch, 1, &buffer[0], sizeof(buffer), nullptr, nullptr);
THROW_LAST_ERROR_IF(length <= 0);
std::string_view slice{ &buffer[0], gsl::narrow_cast<size_t>(length) };
til::bytes_transfer(target, slice);
++read;
if (!slice.empty())
{
_cachedTextA = slice;
_cachedTextReaderA = _cachedTextA;
break;
}
}
source = source.substr(read);
}
}
}
// Routine Description:
// - stores partial read char data for a later read. will overwrite
// any previously stored data.
// Arguments:
// - event - The event to store
// Return Value:
// - None
void InputBuffer::StoreReadPartialByteSequence(std::unique_ptr<IInputEvent> event)
// Same as `Consume`, but without any `source` characters.
void InputBuffer::ConsumeCached(bool isUnicode, std::span<char>& target)
{
_readPartialByteSequence.swap(event);
_switchReadingMode(isUnicode ? ReadingMode::StringW : ReadingMode::StringA);
if (isUnicode)
{
if (!_cachedTextReaderW.empty())
{
til::bytes_transfer(target, _cachedTextReaderW);
if (_cachedTextReaderW.empty())
{
// This is just so that we release memory eagerly.
_cachedTextW = std::wstring{};
}
}
}
else
{
if (!_cachedTextReaderA.empty())
{
til::bytes_transfer(target, _cachedTextReaderA);
if (_cachedTextReaderA.empty())
{
// This is just so that we release memory eagerly.
_cachedTextA = std::string{};
}
}
}
}
void InputBuffer::Cache(std::wstring_view source)
{
const auto off = _cachedTextW.empty() ? 0 : _cachedTextReaderW.data() - _cachedTextW.data();
_cachedTextW.append(source);
_cachedTextReaderW = std::wstring_view{ _cachedTextW }.substr(off);
}
// Moves up to `count`, previously cached events into `target`.
size_t InputBuffer::ConsumeCached(bool isUnicode, size_t count, InputEventQueue& target)
{
_switchReadingMode(isUnicode ? ReadingMode::InputEventsW : ReadingMode::InputEventsA);
size_t i = 0;
while (i < count && !_cachedInputEvents.empty())
{
target.push_back(std::move(_cachedInputEvents.front()));
_cachedInputEvents.pop_front();
i++;
}
return i;
}
// Copies up to `count`, previously cached events into `target`.
size_t InputBuffer::PeekCached(bool isUnicode, size_t count, InputEventQueue& target)
{
_switchReadingMode(isUnicode ? ReadingMode::InputEventsW : ReadingMode::InputEventsA);
size_t i = 0;
for (const auto& e : _cachedInputEvents)
{
if (i >= count)
{
break;
}
target.push_back(IInputEvent::Create(e->ToInputRecord()));
i++;
}
return i;
}
// Trims `source` to have a size below or equal to `expectedSourceSize` by
// storing any extra events in `_cachedInputEvents` for later retrieval.
void InputBuffer::Cache(bool isUnicode, InputEventQueue& source, size_t expectedSourceSize)
{
_switchReadingMode(isUnicode ? ReadingMode::InputEventsW : ReadingMode::InputEventsA);
if (source.size() > expectedSourceSize)
{
_cachedInputEvents.insert(
_cachedInputEvents.end(),
std::make_move_iterator(source.begin() + expectedSourceSize),
std::make_move_iterator(source.end()));
source.resize(expectedSourceSize);
}
}
void InputBuffer::_switchReadingMode(ReadingMode mode)
{
if (_readingMode != mode)
{
_switchReadingModeSlowPath(mode);
}
}
void InputBuffer::_switchReadingModeSlowPath(ReadingMode mode)
{
_cachedTextA = std::string{};
_cachedTextReaderA = {};
_cachedTextW = std::wstring{};
_cachedTextReaderW = {};
_cachedInputEvents = std::deque<std::unique_ptr<IInputEvent>>{};
_readingMode = mode;
}
// Routine Description:
@@ -261,47 +402,105 @@ void InputBuffer::PassThroughWin32MouseRequest(bool enable)
const bool WaitForData,
const bool Unicode,
const bool Stream)
try
{
try
assert(OutEvents.empty());
const auto cp = ServiceLocator::LocateGlobals().getConsoleInformation().CP;
if (Peek)
{
if (_storage.empty())
PeekCached(Unicode, AmountToRead, OutEvents);
}
else
{
ConsumeCached(Unicode, AmountToRead, OutEvents);
}
auto it = _storage.begin();
const auto end = _storage.end();
while (it != end && OutEvents.size() < AmountToRead)
{
auto event = IInputEvent::Create((*it)->ToInputRecord());
if (event->EventType() == InputEventType::KeyEvent)
{
if (!WaitForData)
const auto keyEvent = static_cast<KeyEvent*>(event.get());
WORD repeat = 1;
// for stream reads we need to split any key events that have been coalesced
if (Stream)
{
return STATUS_SUCCESS;
repeat = keyEvent->GetRepeatCount();
keyEvent->SetRepeatCount(1);
}
if (Unicode)
{
do
{
OutEvents.push_back(std::make_unique<KeyEvent>(*keyEvent));
repeat--;
} while (repeat > 0 && OutEvents.size() < AmountToRead);
}
else
{
const auto wch = keyEvent->GetCharData();
char buffer[8];
const auto length = WideCharToMultiByte(cp, 0, &wch, 1, &buffer[0], sizeof(buffer), nullptr, nullptr);
THROW_LAST_ERROR_IF(length <= 0);
const std::string_view str{ &buffer[0], gsl::narrow_cast<size_t>(length) };
do
{
for (const auto& ch : str)
{
auto tempEvent = std::make_unique<KeyEvent>(*keyEvent);
tempEvent->SetCharData(ch);
OutEvents.push_back(std::move(tempEvent));
}
repeat--;
} while (repeat > 0 && OutEvents.size() < AmountToRead);
}
if (repeat && !Peek)
{
const auto originalKeyEvent = static_cast<KeyEvent*>((*it).get());
originalKeyEvent->SetRepeatCount(repeat);
break;
}
return CONSOLE_STATUS_WAIT;
}
// read from buffer
std::deque<std::unique_ptr<IInputEvent>> events;
size_t eventsRead;
bool resetWaitEvent;
_ReadBuffer(events,
AmountToRead,
eventsRead,
Peek,
resetWaitEvent,
Unicode,
Stream);
// copy events to outEvents
while (!events.empty())
else
{
OutEvents.push_back(std::move(events.front()));
events.pop_front();
OutEvents.push_back(std::move(event));
}
if (resetWaitEvent)
{
ServiceLocator::LocateGlobals().hInputEvent.ResetEvent();
}
return STATUS_SUCCESS;
++it;
}
catch (...)
if (!Peek)
{
return NTSTATUS_FROM_HRESULT(wil::ResultFromCaughtException());
_storage.erase(_storage.begin(), it);
}
Cache(Unicode, OutEvents, AmountToRead);
if (OutEvents.empty())
{
return WaitForData ? CONSOLE_STATUS_WAIT : STATUS_SUCCESS;
}
if (_storage.empty())
{
ServiceLocator::LocateGlobals().hInputEvent.ResetEvent();
}
return STATUS_SUCCESS;
}
catch (...)
{
return NTSTATUS_FROM_HRESULT(wil::ResultFromCaughtException());
}
// Routine Description:
@@ -349,130 +548,6 @@ void InputBuffer::PassThroughWin32MouseRequest(bool enable)
return Status;
}
// Routine Description:
// - This routine reads from a buffer. It does the buffer manipulation.
// Arguments:
// - outEvents - where read events are placed
// - readCount - amount of events to read
// - eventsRead - where to store number of events read
// - peek - if true , don't remove data from buffer, just copy it.
// - resetWaitEvent - on exit, true if buffer became empty.
// - unicode - true if read should be done in unicode mode
// - streamRead - true if read should unpack KeyEvents that have a >1 repeat count. readCount must be 1 if streamRead is true.
// Return Value:
// - <none>
// Note:
// - The console lock must be held when calling this routine.
void InputBuffer::_ReadBuffer(_Out_ std::deque<std::unique_ptr<IInputEvent>>& outEvents,
const size_t readCount,
_Out_ size_t& eventsRead,
const bool peek,
_Out_ bool& resetWaitEvent,
const bool unicode,
const bool streamRead)
{
// when stream reading, the previous behavior was to only allow reading of a single
// event at a time.
FAIL_FAST_IF(streamRead && readCount != 1);
resetWaitEvent = false;
std::deque<std::unique_ptr<IInputEvent>> readEvents;
// we need another var to keep track of how many we've read
// because dbcs records count for two when we aren't doing a
// unicode read but the eventsRead count should return the number
// of events actually put into outRecords.
size_t virtualReadCount = 0;
while (!_storage.empty() && virtualReadCount < readCount)
{
auto performNormalRead = true;
// for stream reads we need to split any key events that have been coalesced
if (streamRead)
{
if (_storage.front()->EventType() == InputEventType::KeyEvent)
{
const auto pKeyEvent = static_cast<KeyEvent* const>(_storage.front().get());
if (pKeyEvent->GetRepeatCount() > 1)
{
// split the key event
auto streamKeyEvent = std::make_unique<KeyEvent>(*pKeyEvent);
streamKeyEvent->SetRepeatCount(1);
readEvents.push_back(std::move(streamKeyEvent));
pKeyEvent->SetRepeatCount(pKeyEvent->GetRepeatCount() - 1);
performNormalRead = false;
}
}
}
if (performNormalRead)
{
readEvents.push_back(std::move(_storage.front()));
_storage.pop_front();
}
++virtualReadCount;
if (!unicode)
{
if (readEvents.back()->EventType() == InputEventType::KeyEvent)
{
const auto pKeyEvent = static_cast<const KeyEvent* const>(readEvents.back().get());
if (IsGlyphFullWidth(pKeyEvent->GetCharData()))
{
++virtualReadCount;
}
}
}
}
// the amount of events that were actually read
eventsRead = readEvents.size();
// copy the events back if we were supposed to peek
if (peek)
{
if (streamRead)
{
// we need to check and see if the event was split from a coalesced key event
// or if it was unrelated to the current front event in storage
if (!readEvents.empty() &&
!_storage.empty() &&
readEvents.back()->EventType() == InputEventType::KeyEvent &&
_storage.front()->EventType() == InputEventType::KeyEvent &&
_CanCoalesce(static_cast<const KeyEvent&>(*readEvents.back()),
static_cast<const KeyEvent&>(*_storage.front())))
{
auto& keyEvent = static_cast<KeyEvent&>(*_storage.front());
keyEvent.SetRepeatCount(keyEvent.GetRepeatCount() + 1);
}
else
{
_storage.push_front(IInputEvent::Create(readEvents.back()->ToInputRecord()));
}
}
else
{
for (auto it = readEvents.crbegin(); it != readEvents.crend(); ++it)
{
_storage.push_front(IInputEvent::Create((*it)->ToInputRecord()));
}
}
}
// move events read to proper deque
while (!readEvents.empty())
{
outEvents.push_back(std::move(readEvents.front()));
readEvents.pop_front();
}
// signal if we emptied the buffer
if (_storage.empty())
{
resetWaitEvent = true;
}
}
// Routine Description:
// - Writes events to the beginning of the input buffer.
// Arguments:

38
src/host/inputBuffer.hpp
View File

@@ -41,12 +41,15 @@ public:
bool fInComposition; // specifies if there's an ongoing text composition
InputBuffer();
~InputBuffer();
// storage API for partial dbcs bytes being read from the buffer
bool IsReadPartialByteSequenceAvailable();
std::unique_ptr<IInputEvent> FetchReadPartialByteSequence(_In_ bool peek);
void StoreReadPartialByteSequence(std::unique_ptr<IInputEvent> event);
// String oriented APIs
void Consume(bool isUnicode, std::wstring_view& source, std::span<char>& target);
void ConsumeCached(bool isUnicode, std::span<char>& target);
void Cache(std::wstring_view source);
// INPUT_RECORD oriented APIs
size_t ConsumeCached(bool isUnicode, size_t count, InputEventQueue& target);
size_t PeekCached(bool isUnicode, size_t count, InputEventQueue& target);
void Cache(bool isUnicode, InputEventQueue& source, size_t expectedSourceSize);
// storage API for partial dbcs bytes being written to the buffer
bool IsWritePartialByteSequenceAvailable();
@@ -84,8 +87,22 @@ public:
void PassThroughWin32MouseRequest(bool enable);
private:
enum class ReadingMode : uint8_t
{
StringA,
StringW,
InputEventsA,
InputEventsW,
};
std::string _cachedTextA;
std::string_view _cachedTextReaderA;
std::wstring _cachedTextW;
std::wstring_view _cachedTextReaderW;
std::deque<std::unique_ptr<IInputEvent>> _cachedInputEvents;
ReadingMode _readingMode = ReadingMode::StringA;
std::deque<std::unique_ptr<IInputEvent>> _storage;
std::unique_ptr<IInputEvent> _readPartialByteSequence;
std::unique_ptr<IInputEvent> _writePartialByteSequence;
Microsoft::Console::VirtualTerminal::TerminalInput _termInput;
Microsoft::Console::Render::VtEngine* _pTtyConnection;
@@ -96,13 +113,8 @@ private:
// Otherwise, we should be calling them.
bool _vtInputShouldSuppress{ false };
void _ReadBuffer(_Out_ std::deque<std::unique_ptr<IInputEvent>>& outEvents,
const size_t readCount,
_Out_ size_t& eventsRead,
const bool peek,
_Out_ bool& resetWaitEvent,
const bool unicode,
const bool streamRead);
void _switchReadingMode(ReadingMode mode);
void _switchReadingModeSlowPath(ReadingMode mode);
void _WriteBuffer(_Inout_ std::deque<std::unique_ptr<IInputEvent>>& inRecords,
_Out_ size_t& eventsWritten,

48
src/host/misc.cpp
View File

@@ -2,13 +2,12 @@
// Licensed under the MIT license.
#include "precomp.h"
#include "misc.h"
#include <til/unicode.h>
#include "dbcs.h"
#include "../types/inc/convert.hpp"
#include "../types/inc/GlyphWidth.hpp"
#include "../interactivity/inc/ServiceLocator.hpp"
#pragma hdrstop
@@ -182,47 +181,6 @@ BOOL CheckBisectProcessW(const SCREEN_INFORMATION& ScreenInfo,
}
}
// Routine Description:
// - Converts all key events in the deque to the oem char data and adds
// them back to events.
// Arguments:
// - events - on input the IInputEvents to convert. on output, the
// converted input events
// Note: may throw on error
void SplitToOem(std::deque<std::unique_ptr<IInputEvent>>& events)
{
const auto cp = ServiceLocator::LocateGlobals().getConsoleInformation().CP;
std::deque<std::unique_ptr<IInputEvent>> convertedEvents;
for (auto& currentEvent : events)
{
if (currentEvent->EventType() == InputEventType::KeyEvent)
{
const auto pKeyEvent = static_cast<const KeyEvent* const>(currentEvent.get());
const auto wch = pKeyEvent->GetCharData();
char buffer[8];
const auto length = WideCharToMultiByte(cp, 0, &wch, 1, &buffer[0], sizeof(buffer), nullptr, nullptr);
THROW_LAST_ERROR_IF(length <= 0);
const std::string_view str{ &buffer[0], gsl::narrow_cast<size_t>(length) };
for (const auto& ch : str)
{
auto tempEvent = std::make_unique<KeyEvent>(*pKeyEvent);
tempEvent->SetCharData(ch);
convertedEvents.push_back(std::move(tempEvent));
}
}
else
{
convertedEvents.push_back(std::move(currentEvent));
}
}
events = std::move(convertedEvents);
}
// Routine Description:
// - Converts unicode characters to ANSI given a destination codepage
// Arguments:

2
src/host/misc.h
View File

@@ -44,8 +44,6 @@ int ConvertToOem(const UINT uiCodePage,
_Out_writes_(cchTarget) CHAR* const pchTarget,
const UINT cchTarget) noexcept;
void SplitToOem(std::deque<std::unique_ptr<IInputEvent>>& events);
int ConvertInputToUnicode(const UINT uiCodePage,
_In_reads_(cchSource) const CHAR* const pchSource,
const UINT cchSource,

206
src/host/readDataCooked.cpp
View File

@@ -47,7 +47,7 @@ COOKED_READ_DATA::COOKED_READ_DATA(_In_ InputBuffer* const pInputBuffer,
_In_ INPUT_READ_HANDLE_DATA* const pInputReadHandleData,
SCREEN_INFORMATION& screenInfo,
_In_ size_t UserBufferSize,
_In_ PWCHAR UserBuffer,
_In_ char* UserBuffer,
_In_ ULONG CtrlWakeupMask,
_In_ const std::wstring_view exeName,
_In_ const std::string_view initialData,
@@ -62,7 +62,7 @@ COOKED_READ_DATA::COOKED_READ_DATA(_In_ InputBuffer* const pInputBuffer,
_exeName{ exeName },
_pdwNumBytes{ nullptr },
_commandHistory{ CommandHistory::s_Find((HANDLE)pClientProcess) },
_commandHistory{ CommandHistory::s_Find(pClientProcess) },
_controlKeyState{ 0 },
_ctrlWakeupMask{ CtrlWakeupMask },
_visibleCharCount{ 0 },
@@ -881,10 +881,10 @@ size_t COOKED_READ_DATA::SavePromptToUserBuffer(const size_t cch)
try
{
const auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
const auto wstr = ConvertToW(gci.CP, { reinterpret_cast<char*>(_userBuffer), cch });
const auto copyAmount = std::min(wstr.size(), _userBufferSize / sizeof(wchar_t));
std::copy_n(wstr.begin(), copyAmount, _userBuffer);
return copyAmount * sizeof(wchar_t);
const auto wstr = ConvertToW(gci.CP, { _userBuffer, cch });
const auto copyAmount = std::min(wstr.size() * sizeof(wchar_t), _userBufferSize);
std::copy_n(reinterpret_cast<const char*>(wstr.data()), copyAmount, _userBuffer);
return copyAmount;
}
CATCH_LOG();
}
@@ -1003,211 +1003,55 @@ void COOKED_READ_DATA::SavePendingInput(const size_t index, const bool multiline
// - Status code that indicates success, out of memory, etc.
[[nodiscard]] NTSTATUS COOKED_READ_DATA::_handlePostCharInputLoop(const bool isUnicode, size_t& numBytes, ULONG& controlKeyState) noexcept
{
std::span writer{ _userBuffer, _userBufferSize };
std::wstring_view input{ _backupLimit, _bytesRead / sizeof(wchar_t) };
DWORD LineCount = 1;
if (_echoInput)
{
// Figure out where real string ends (at carriage return or end of buffer).
auto StringPtr = _backupLimit;
auto StringLength = _bytesRead / sizeof(WCHAR);
auto FoundCR = false;
for (size_t i = 0; i < StringLength; i++)
{
if (*StringPtr++ == UNICODE_CARRIAGERETURN)
{
StringLength = i;
FoundCR = true;
break;
}
}
if (FoundCR)
const auto idx = input.find(UNICODE_CARRIAGERETURN);
if (idx != decltype(input)::npos)
{
if (_commandHistory)
{
// add to command line recall list if we have a history list.
auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
LOG_IF_FAILED(_commandHistory->Add({ _backupLimit, StringLength },
WI_IsFlagSet(gci.Flags, CONSOLE_HISTORY_NODUP)));
LOG_IF_FAILED(_commandHistory->Add({ _backupLimit, idx }, WI_IsFlagSet(gci.Flags, CONSOLE_HISTORY_NODUP)));
}
Tracing::s_TraceCookedRead(_clientProcess,
_backupLimit,
base::saturated_cast<ULONG>(StringLength));
// check for alias
Tracing::s_TraceCookedRead(_clientProcess, _backupLimit, base::saturated_cast<ULONG>(idx));
ProcessAliases(LineCount);
if (LineCount > 1)
{
input = input.substr(0, idx + 1);
}
}
}
auto fAddDbcsLead = false;
size_t NumBytes = 0;
// at this point, a->NumBytes contains the number of bytes in
// the UNICODE string read. UserBufferSize contains the converted
// size of the app's buffer.
if (_bytesRead > _userBufferSize || LineCount > 1)
GetInputBuffer()->Consume(isUnicode, input, writer);
if (!input.empty())
{
if (LineCount > 1)
{
PWSTR Tmp;
if (!isUnicode)
{
if (_pInputBuffer->IsReadPartialByteSequenceAvailable())
{
fAddDbcsLead = true;
auto event = GetInputBuffer()->FetchReadPartialByteSequence(false);
const auto pKeyEvent = static_cast<const KeyEvent* const>(event.get());
*_userBuffer = static_cast<char>(pKeyEvent->GetCharData());
_userBuffer++;
_userBufferSize -= sizeof(wchar_t);
}
NumBytes = 0;
for (Tmp = _backupLimit;
*Tmp != UNICODE_LINEFEED && _userBufferSize / sizeof(WCHAR) > NumBytes;
Tmp++)
{
NumBytes += IsGlyphFullWidth(*Tmp) ? 2 : 1;
}
}
// clang-format off
#pragma prefast(suppress: __WARNING_BUFFER_OVERFLOW, "LineCount > 1 means there's a UNICODE_LINEFEED")
// clang-format on
for (Tmp = _backupLimit; *Tmp != UNICODE_LINEFEED; Tmp++)
{
FAIL_FAST_IF(!(Tmp < (_backupLimit + _bytesRead)));
}
numBytes = (ULONG)(Tmp - _backupLimit + 1) * sizeof(*Tmp);
GetInputReadHandleData()->SaveMultilinePendingInput(input);
}
else
{
if (!isUnicode)
{
PWSTR Tmp;
if (_pInputBuffer->IsReadPartialByteSequenceAvailable())
{
fAddDbcsLead = true;
auto event = GetInputBuffer()->FetchReadPartialByteSequence(false);
const auto pKeyEvent = static_cast<const KeyEvent* const>(event.get());
*_userBuffer = static_cast<char>(pKeyEvent->GetCharData());
_userBuffer++;
_userBufferSize -= sizeof(wchar_t);
}
NumBytes = 0;
auto NumToWrite = _bytesRead;
for (Tmp = _backupLimit;
NumToWrite && _userBufferSize / sizeof(WCHAR) > NumBytes;
Tmp++, NumToWrite -= sizeof(WCHAR))
{
NumBytes += IsGlyphFullWidth(*Tmp) ? 2 : 1;
}
}
numBytes = _userBufferSize;
}
__analysis_assume(numBytes <= _userBufferSize);
memmove(_userBuffer, _backupLimit, numBytes);
const auto pInputReadHandleData = GetInputReadHandleData();
const std::wstring_view pending{ _backupLimit + (numBytes / sizeof(wchar_t)), (_bytesRead - numBytes) / sizeof(wchar_t) };
if (LineCount > 1)
{
pInputReadHandleData->SaveMultilinePendingInput(pending);
}
else
{
pInputReadHandleData->SavePendingInput(pending);
GetInputReadHandleData()->SavePendingInput(input);
}
}
else
{
if (!isUnicode)
{
PWSTR Tmp;
if (_pInputBuffer->IsReadPartialByteSequenceAvailable())
{
fAddDbcsLead = true;
auto event = GetInputBuffer()->FetchReadPartialByteSequence(false);
const auto pKeyEvent = static_cast<const KeyEvent* const>(event.get());
*_userBuffer = static_cast<char>(pKeyEvent->GetCharData());
_userBuffer++;
_userBufferSize -= sizeof(wchar_t);
if (_userBufferSize == 0)
{
numBytes = 1;
return STATUS_SUCCESS;
}
}
NumBytes = 0;
auto NumToWrite = _bytesRead;
for (Tmp = _backupLimit;
NumToWrite && _userBufferSize / sizeof(WCHAR) > NumBytes;
Tmp++, NumToWrite -= sizeof(WCHAR))
{
NumBytes += IsGlyphFullWidth(*Tmp) ? 2 : 1;
}
}
numBytes = _bytesRead;
if (numBytes > _userBufferSize)
{
return STATUS_BUFFER_OVERFLOW;
}
memmove(_userBuffer, _backupLimit, numBytes);
}
numBytes = _userBufferSize - writer.size();
controlKeyState = _controlKeyState;
if (!isUnicode)
{
// if ansi, translate string.
std::unique_ptr<char[]> tempBuffer;
try
{
tempBuffer = std::make_unique<char[]>(NumBytes);
}
catch (...)
{
return STATUS_NO_MEMORY;
}
std::unique_ptr<IInputEvent> partialEvent;
numBytes = TranslateUnicodeToOem(_userBuffer,
gsl::narrow<ULONG>(numBytes / sizeof(wchar_t)),
tempBuffer.get(),
gsl::narrow<ULONG>(NumBytes),
partialEvent);
if (partialEvent.get())
{
GetInputBuffer()->StoreReadPartialByteSequence(std::move(partialEvent));
}
if (numBytes > _userBufferSize)
{
return STATUS_BUFFER_OVERFLOW;
}
memmove(_userBuffer, tempBuffer.get(), numBytes);
if (fAddDbcsLead)
{
numBytes++;
}
}
return STATUS_SUCCESS;
}
void COOKED_READ_DATA::MigrateUserBuffersOnTransitionToBackgroundWait(const void* oldBuffer, void* newBuffer)
{
// See the comment in WaitBlock.cpp for more information.
if (_userBuffer == reinterpret_cast<const wchar_t*>(oldBuffer))
if (_userBuffer == oldBuffer)
{
_userBuffer = reinterpret_cast<wchar_t*>(newBuffer);
_userBuffer = static_cast<char*>(newBuffer);
}
}

4
src/host/readDataCooked.hpp
View File

@@ -37,7 +37,7 @@ public:
_In_ INPUT_READ_HANDLE_DATA* const pInputReadHandleData,
SCREEN_INFORMATION& screenInfo,
_In_ size_t UserBufferSize,
_In_ PWCHAR UserBuffer,
_In_ char* UserBuffer,
_In_ ULONG CtrlWakeupMask,
_In_ const std::wstring_view exeName,
_In_ const std::string_view initialData,
@@ -129,7 +129,7 @@ private:
wchar_t* _backupLimit;
size_t _userBufferSize; // doubled size in ansi case
wchar_t* _userBuffer;
char* _userBuffer;
size_t* _pdwNumBytes;

135
src/host/readDataDirect.cpp
View File

@@ -10,9 +10,7 @@
// Routine Description:
// - Constructs direct read data class to hold context across sessions
// generally when there's not enough data to return. Also used a bit
// internally to just pass some information along the stack
// (regardless of wait necessity).
// generally when there's not enough data to return.
// Arguments:
// - pInputBuffer - Buffer that data will be read from.
// - pInputReadHandleData - Context stored across calls from the same
@@ -27,17 +25,10 @@ DirectReadData::DirectReadData(_In_ InputBuffer* const pInputBuffer,
const size_t eventReadCount,
_In_ std::deque<std::unique_ptr<IInputEvent>> partialEvents) :
ReadData(pInputBuffer, pInputReadHandleData),
_eventReadCount{ eventReadCount },
_partialEvents{ std::move(partialEvents) },
_outEvents{}
_eventReadCount{ eventReadCount }
{
}
// Routine Description:
// - Destructs a read data class.
// - Decrements count of readers waiting on the given handle.
DirectReadData::~DirectReadData() = default;
// Routine Description:
// - This routine is called to complete a direct read that blocked in
// ReadInputBuffer. The context of the read was saved in the DirectReadData
@@ -68,18 +59,17 @@ bool DirectReadData::Notify(const WaitTerminationReason TerminationReason,
_Out_ size_t* const pNumBytes,
_Out_ DWORD* const pControlKeyState,
_Out_ void* const pOutputData)
try
{
FAIL_FAST_IF_NULL(pOutputData);
FAIL_FAST_IF(_pInputReadHandleData->GetReadCount() == 0);
const auto& gci = Microsoft::Console::Interactivity::ServiceLocator::LocateGlobals().getConsoleInformation();
FAIL_FAST_IF(!gci.IsConsoleLocked());
assert(Microsoft::Console::Interactivity::ServiceLocator::LocateGlobals().getConsoleInformation().IsConsoleLocked());
*pReplyStatus = STATUS_SUCCESS;
*pControlKeyState = 0;
*pNumBytes = 0;
auto retVal = true;
std::deque<std::unique_ptr<IInputEvent>> readEvents;
// If ctrl-c or ctrl-break was seen, ignore it.
@@ -88,105 +78,58 @@ bool DirectReadData::Notify(const WaitTerminationReason TerminationReason,
return false;
}
// check if a partial byte is already stored that we should read
if (!fIsUnicode &&
_pInputBuffer->IsReadPartialByteSequenceAvailable() &&
_eventReadCount == 1)
{
_partialEvents.push_back(_pInputBuffer->FetchReadPartialByteSequence(false));
}
// See if called by CsrDestroyProcess or CsrDestroyThread
// via ConsoleNotifyWaitBlock. If so, just decrement the ReadCount and return.
if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::ThreadDying))
{
*pReplyStatus = STATUS_THREAD_IS_TERMINATING;
return true;
}
// We must see if we were woken up because the handle is being
// closed. If so, we decrement the read count. If it goes to
// zero, we wake up the close thread. Otherwise, we wake up any
// other thread waiting for data.
else if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::HandleClosing))
if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::HandleClosing))
{
*pReplyStatus = STATUS_ALERTED;
return true;
}
else
// if we get to here, this routine was called either by the input
// thread or a write routine. both of these callers grab the
// current console lock.
size_t amountToRead;
if (FAILED(SizeTSub(_eventReadCount, _outEvents.size(), &amountToRead)))
{
// if we get to here, this routine was called either by the input
// thread or a write routine. both of these callers grab the
// current console lock.
// calculate how many events we need to read
size_t amountAlreadyRead;
if (FAILED(SizeTAdd(_partialEvents.size(), _outEvents.size(), &amountAlreadyRead)))
{
*pReplyStatus = STATUS_INTEGER_OVERFLOW;
return retVal;
}
size_t amountToRead;
if (FAILED(SizeTSub(_eventReadCount, amountAlreadyRead, &amountToRead)))
{
*pReplyStatus = STATUS_INTEGER_OVERFLOW;
return retVal;
}
*pReplyStatus = _pInputBuffer->Read(readEvents,
amountToRead,
false,
false,
fIsUnicode,
false);
if (*pReplyStatus == CONSOLE_STATUS_WAIT)
{
retVal = false;
}
*pReplyStatus = STATUS_INTEGER_OVERFLOW;
return true;
}
if (*pReplyStatus != CONSOLE_STATUS_WAIT)
*pReplyStatus = _pInputBuffer->Read(_outEvents,
amountToRead,
false,
false,
fIsUnicode,
false);
if (*pReplyStatus == CONSOLE_STATUS_WAIT)
{
// split key events to oem chars if necessary
if (*pReplyStatus == STATUS_SUCCESS && !fIsUnicode)
{
try
{
SplitToOem(readEvents);
}
CATCH_LOG();
}
// combine partial and whole events
while (!_partialEvents.empty())
{
readEvents.push_front(std::move(_partialEvents.back()));
_partialEvents.pop_back();
}
// move read events to out storage
for (size_t i = 0; i < _eventReadCount; ++i)
{
if (readEvents.empty())
{
break;
}
_outEvents.push_back(std::move(readEvents.front()));
readEvents.pop_front();
}
// store partial event if necessary
if (!readEvents.empty())
{
_pInputBuffer->StoreReadPartialByteSequence(std::move(readEvents.front()));
readEvents.pop_front();
FAIL_FAST_IF(!(readEvents.empty()));
}
// move events to pOutputData
const auto pOutputDeque = reinterpret_cast<std::deque<std::unique_ptr<IInputEvent>>* const>(pOutputData);
*pNumBytes = _outEvents.size() * sizeof(INPUT_RECORD);
pOutputDeque->swap(_outEvents);
return false;
}
return retVal;
// move events to pOutputData
const auto pOutputDeque = static_cast<std::deque<std::unique_ptr<IInputEvent>>* const>(pOutputData);
*pNumBytes = _outEvents.size() * sizeof(INPUT_RECORD);
*pOutputDeque = std::move(_outEvents);
return true;
}
catch (...)
{
*pReplyStatus = wil::StatusFromCaughtException();
return true;
}
void DirectReadData::MigrateUserBuffersOnTransitionToBackgroundWait(const void* /*oldBuffer*/, void* /*newBuffer*/)

3
src/host/readDataDirect.hpp
View File

@@ -38,8 +38,6 @@ public:
DirectReadData(DirectReadData&&) = default;
~DirectReadData() override;
void MigrateUserBuffersOnTransitionToBackgroundWait(const void* oldBuffer, void* newBuffer) override;
bool Notify(const WaitTerminationReason TerminationReason,
const bool fIsUnicode,
@@ -50,6 +48,5 @@ public:
private:
const size_t _eventReadCount;
std::deque<std::unique_ptr<IInputEvent>> _partialEvents;
std::deque<std::unique_ptr<IInputEvent>> _outEvents;
};

131
src/host/readDataRaw.cpp
View File

@@ -31,7 +31,6 @@ RAW_READ_DATA::RAW_READ_DATA(_In_ InputBuffer* const pInputBuffer,
_BufferSize{ BufferSize },
_BufPtr{ THROW_HR_IF_NULL(E_INVALIDARG, BufPtr) }
{
THROW_HR_IF(E_INVALIDARG, _BufferSize % sizeof(wchar_t) != 0);
THROW_HR_IF(E_INVALIDARG, _BufferSize == 0);
}
@@ -82,146 +81,50 @@ bool RAW_READ_DATA::Notify(const WaitTerminationReason TerminationReason,
*pControlKeyState = 0;
*pNumBytes = 0;
size_t NumBytes = 0;
PWCHAR lpBuffer;
auto RetVal = true;
auto fAddDbcsLead = false;
auto fSkipFinally = false;
// If a ctrl-c is seen, don't terminate read. If ctrl-break is seen, terminate read.
if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::CtrlC))
{
return false;
}
else if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::CtrlBreak))
if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::CtrlBreak))
{
*pReplyStatus = STATUS_ALERTED;
return true;
}
// See if we were called because the thread that owns this wait block is exiting.
else if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::ThreadDying))
if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::ThreadDying))
{
*pReplyStatus = STATUS_THREAD_IS_TERMINATING;
return true;
}
// We must see if we were woken up because the handle is being
// closed. If so, we decrement the read count. If it goes to zero,
// we wake up the close thread. Otherwise, we wake up any other
// thread waiting for data.
else if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::HandleClosing))
if (WI_IsFlagSet(TerminationReason, WaitTerminationReason::HandleClosing))
{
*pReplyStatus = STATUS_ALERTED;
}
else
{
// If we get to here, this routine was called either by the input
// thread or a write routine. Both of these callers grab the current
// console lock.
lpBuffer = _BufPtr;
if (!fIsUnicode && _pInputBuffer->IsReadPartialByteSequenceAvailable())
{
auto event = _pInputBuffer->FetchReadPartialByteSequence(false);
const auto pKeyEvent = static_cast<const KeyEvent* const>(event.get());
*lpBuffer = static_cast<char>(pKeyEvent->GetCharData());
_BufferSize -= sizeof(wchar_t);
*pReplyStatus = STATUS_SUCCESS;
fAddDbcsLead = true;
if (_BufferSize == 0)
{
*pNumBytes = 1;
RetVal = false;
fSkipFinally = true;
}
}
else
{
// This call to GetChar may block.
*pReplyStatus = GetChar(_pInputBuffer,
lpBuffer,
true,
nullptr,
nullptr,
nullptr);
}
if (FAILED_NTSTATUS(*pReplyStatus) || fSkipFinally)
{
if (*pReplyStatus == CONSOLE_STATUS_WAIT)
{
RetVal = false;
}
}
else
{
NumBytes += IsGlyphFullWidth(*lpBuffer) ? 2 : 1;
lpBuffer++;
*pNumBytes += sizeof(WCHAR);
while (*pNumBytes < _BufferSize)
{
// This call to GetChar won't block.
*pReplyStatus = GetChar(_pInputBuffer,
lpBuffer,
false,
nullptr,
nullptr,
nullptr);
if (FAILED_NTSTATUS(*pReplyStatus))
{
*pReplyStatus = STATUS_SUCCESS;
break;
}
NumBytes += IsGlyphFullWidth(*lpBuffer) ? 2 : 1;
lpBuffer++;
*pNumBytes += sizeof(WCHAR);
}
}
return true;
}
// If the read was completed (status != wait), free the raw read data.
if (*pReplyStatus != CONSOLE_STATUS_WAIT &&
!fSkipFinally &&
!fIsUnicode)
{
// It's ansi, so translate the string.
std::unique_ptr<char[]> tempBuffer;
try
{
tempBuffer = std::make_unique<char[]>(NumBytes);
}
catch (...)
{
return true;
}
// If we get to here, this routine was called either by the input
// thread or a write routine. Both of these callers grab the current
// console lock.
lpBuffer = _BufPtr;
std::unique_ptr<IInputEvent> partialEvent;
*pNumBytes = TranslateUnicodeToOem(lpBuffer,
gsl::narrow<ULONG>(*pNumBytes / sizeof(wchar_t)),
tempBuffer.get(),
gsl::narrow<ULONG>(NumBytes),
partialEvent);
if (partialEvent.get())
{
_pInputBuffer->StoreReadPartialByteSequence(std::move(partialEvent));
}
memmove(lpBuffer, tempBuffer.get(), *pNumBytes);
if (fAddDbcsLead)
{
(*pNumBytes)++;
}
}
return RetVal;
std::span buffer{ reinterpret_cast<char*>(_BufPtr), _BufferSize };
*pReplyStatus = ReadCharacterInput(*_pInputBuffer, buffer, *pNumBytes, *_pInputReadHandleData, fIsUnicode);
return *pReplyStatus != CONSOLE_STATUS_WAIT;
}
void RAW_READ_DATA::MigrateUserBuffersOnTransitionToBackgroundWait(const void* oldBuffer, void* newBuffer)
{
// See the comment in WaitBlock.cpp for more information.
if (_BufPtr == reinterpret_cast<const wchar_t*>(oldBuffer))
if (_BufPtr == static_cast<const wchar_t*>(oldBuffer))
{
_BufPtr = reinterpret_cast<wchar_t*>(newBuffer);
_BufPtr = static_cast<wchar_t*>(newBuffer);
}
}

309
src/host/stream.cpp
View File

@@ -6,7 +6,6 @@
#include "_stream.h"
#include "stream.h"
#include "dbcs.h"
#include "handle.h"
#include "misc.h"
#include "readDataRaw.hpp"
@@ -285,151 +284,38 @@ til::CoordType RetrieveNumberOfSpaces(_In_ til::CoordType sOriginalCursorPositio
size_t& bytesRead,
INPUT_READ_HANDLE_DATA& readHandleState,
const bool unicode)
try
{
// TODO: MSFT: 18047766 - Correct this method to not play byte counting games.
auto fAddDbcsLead = FALSE;
size_t NumToWrite = 0;
size_t NumToBytes = 0;
auto pBuffer = reinterpret_cast<wchar_t*>(buffer.data());
auto bufferRemaining = buffer.size_bytes();
bytesRead = 0;
if (buffer.size_bytes() < sizeof(wchar_t))
{
return STATUS_BUFFER_TOO_SMALL;
}
const auto pending = readHandleState.GetPendingInput();
auto pendingBytes = pending.size() * sizeof(wchar_t);
auto Tmp = pending.cbegin();
auto pending = readHandleState.GetPendingInput();
if (readHandleState.IsMultilineInput())
{
if (!unicode)
const auto idx = pending.find(UNICODE_LINEFEED);
if (idx != decltype(pending)::npos)
{
if (inputBuffer.IsReadPartialByteSequenceAvailable())
{
auto event = inputBuffer.FetchReadPartialByteSequence(false);
const auto pKeyEvent = static_cast<const KeyEvent* const>(event.get());
*pBuffer = static_cast<char>(pKeyEvent->GetCharData());
++pBuffer;
bufferRemaining -= sizeof(wchar_t);
pendingBytes -= sizeof(wchar_t);
fAddDbcsLead = TRUE;
}
if (pendingBytes == 0 || bufferRemaining == 0)
{
readHandleState.CompletePending();
bytesRead = 1;
return STATUS_SUCCESS;
}
else
{
for (NumToWrite = 0, Tmp = pending.cbegin(), NumToBytes = 0;
NumToBytes < pendingBytes &&
NumToBytes < bufferRemaining / sizeof(wchar_t) &&
*Tmp != UNICODE_LINEFEED;
Tmp++, NumToWrite += sizeof(wchar_t))
{
NumToBytes += IsGlyphFullWidth(*Tmp) ? 2 : 1;
}
}
}
NumToWrite = 0;
Tmp = pending.cbegin();
while (NumToWrite < pendingBytes &&
*Tmp != UNICODE_LINEFEED)
{
++Tmp;
NumToWrite += sizeof(wchar_t);
}
NumToWrite += sizeof(wchar_t);
if (NumToWrite > bufferRemaining)
{
NumToWrite = bufferRemaining;
// +1 to include the newline.
pending = pending.substr(0, idx + 1);
}
}
else
{
if (!unicode)
{
if (inputBuffer.IsReadPartialByteSequenceAvailable())
{
auto event = inputBuffer.FetchReadPartialByteSequence(false);
const auto pKeyEvent = static_cast<const KeyEvent* const>(event.get());
*pBuffer = static_cast<char>(pKeyEvent->GetCharData());
++pBuffer;
bufferRemaining -= sizeof(wchar_t);
pendingBytes -= sizeof(wchar_t);
fAddDbcsLead = TRUE;
}
if (pendingBytes == 0)
{
readHandleState.CompletePending();
bytesRead = 1;
return STATUS_SUCCESS;
}
else
{
for (NumToWrite = 0, Tmp = pending.cbegin(), NumToBytes = 0;
NumToBytes < pendingBytes && NumToBytes < bufferRemaining / sizeof(wchar_t);
Tmp++, NumToWrite += sizeof(wchar_t))
{
NumToBytes += IsGlyphFullWidth(*Tmp) ? 2 : 1;
}
}
}
std::span writer{ buffer };
inputBuffer.Consume(unicode, pending, writer);
NumToWrite = (bufferRemaining < pendingBytes) ? bufferRemaining : pendingBytes;
}
memmove(pBuffer, pending.data(), NumToWrite);
pendingBytes -= NumToWrite;
if (pendingBytes != 0)
{
std::wstring_view remainingPending{ pending.data() + (NumToWrite / sizeof(wchar_t)), pendingBytes / sizeof(wchar_t) };
readHandleState.UpdatePending(remainingPending);
}
else
if (pending.empty())
{
readHandleState.CompletePending();
}
if (!unicode)
else
{
// if ansi, translate string. we allocated the capture buffer
// large enough to handle the translated string.
auto tempBuffer = std::make_unique<char[]>(NumToBytes);
std::unique_ptr<IInputEvent> partialEvent;
NumToWrite = TranslateUnicodeToOem(pBuffer,
gsl::narrow<ULONG>(NumToWrite / sizeof(wchar_t)),
tempBuffer.get(),
gsl::narrow<ULONG>(NumToBytes),
partialEvent);
if (partialEvent.get())
{
inputBuffer.StoreReadPartialByteSequence(std::move(partialEvent));
}
// clang-format off
#pragma prefast(suppress: __WARNING_POTENTIAL_BUFFER_OVERFLOW_HIGH_PRIORITY, "This access is fine but prefast can't follow it, evidently")
// clang-format on
memmove(pBuffer, tempBuffer.get(), NumToWrite);
if (fAddDbcsLead)
{
NumToWrite++;
}
readHandleState.UpdatePending(pending);
}
bytesRead = NumToWrite;
bytesRead = buffer.size() - writer.size();
return STATUS_SUCCESS;
}
NT_CATCH_RETURN()
// Routine Description:
// - read in characters until the buffer is full or return is read.
@@ -478,7 +364,7 @@ til::CoordType RetrieveNumberOfSpaces(_In_ til::CoordType sOriginalCursorPositio
&readHandleState, // pInputReadHandleData
screenInfo, // pScreenInfo
buffer.size_bytes(), // UserBufferSize
reinterpret_cast<wchar_t*>(buffer.data()), // UserBuffer
buffer.data(), // UserBuffer
ctrlWakeupMask, // CtrlWakeupMask
exeName, // exe name
initialData,
@@ -521,140 +407,51 @@ til::CoordType RetrieveNumberOfSpaces(_In_ til::CoordType sOriginalCursorPositio
// populated.
// - STATUS_SUCCESS on success
// - Other NTSTATUS codes as necessary
[[nodiscard]] static NTSTATUS _ReadCharacterInput(InputBuffer& inputBuffer,
std::span<char> buffer,
size_t& bytesRead,
INPUT_READ_HANDLE_DATA& readHandleState,
const bool unicode,
std::unique_ptr<IWaitRoutine>& waiter)
[[nodiscard]] NTSTATUS ReadCharacterInput(InputBuffer& inputBuffer,
std::span<char> buffer,
size_t& bytesRead,
INPUT_READ_HANDLE_DATA& readHandleState,
const bool unicode)
try
{
size_t NumToWrite = 0;
auto addDbcsLead = false;
auto Status = STATUS_SUCCESS;
auto pBuffer = reinterpret_cast<wchar_t*>(buffer.data());
auto bufferRemaining = buffer.size_bytes();
UNREFERENCED_PARAMETER(readHandleState);
bytesRead = 0;
if (buffer.size() < 1)
const auto charSize = unicode ? sizeof(wchar_t) : sizeof(char);
std::span writer{ buffer };
if (writer.size() < charSize)
{
return STATUS_BUFFER_TOO_SMALL;
}
if (bytesRead < bufferRemaining)
inputBuffer.ConsumeCached(unicode, writer);
// We don't need to wait for input if `ConsumeCached` read something already, which is
// indicated by the writer having been advanced (= it's shorter than the original buffer).
auto wait = writer.size() == buffer.size();
auto status = STATUS_SUCCESS;
while (writer.size() >= charSize)
{
auto pwchBufferTmp = pBuffer;
NumToWrite = 0;
if (!unicode && inputBuffer.IsReadPartialByteSequenceAvailable())
wchar_t wch;
status = GetChar(&inputBuffer, &wch, wait, nullptr, nullptr, nullptr);
if (!NT_SUCCESS(status))
{
auto event = inputBuffer.FetchReadPartialByteSequence(false);
const auto pKeyEvent = static_cast<const KeyEvent* const>(event.get());
*pBuffer = static_cast<char>(pKeyEvent->GetCharData());
++pBuffer;
bufferRemaining -= sizeof(wchar_t);
addDbcsLead = true;
if (bufferRemaining == 0)
{
bytesRead = 1;
return STATUS_SUCCESS;
}
}
else
{
Status = GetChar(&inputBuffer,
pBuffer,
true,
nullptr,
nullptr,
nullptr);
break;
}
if (Status == CONSOLE_STATUS_WAIT)
{
waiter = std::make_unique<RAW_READ_DATA>(&inputBuffer,
&readHandleState,
gsl::narrow<ULONG>(buffer.size_bytes()),
reinterpret_cast<wchar_t*>(buffer.data()));
}
std::wstring_view wchView{ &wch, 1 };
inputBuffer.Consume(unicode, wchView, writer);
if (FAILED_NTSTATUS(Status))
{
bytesRead = 0;
return Status;
}
if (!addDbcsLead)
{
bytesRead += IsGlyphFullWidth(*pBuffer) ? 2 : 1;
NumToWrite += sizeof(wchar_t);
pBuffer++;
}
while (NumToWrite < static_cast<ULONG>(bufferRemaining))
{
Status = GetChar(&inputBuffer,
pBuffer,
false,
nullptr,
nullptr,
nullptr);
if (FAILED_NTSTATUS(Status))
{
break;
}
bytesRead += IsGlyphFullWidth(*pBuffer) ? 2 : 1;
NumToWrite += sizeof(wchar_t);
pBuffer++;
}
// if ansi, translate string. we allocated the capture buffer large enough to handle the translated string.
if (!unicode)
{
std::unique_ptr<char[]> tempBuffer;
try
{
tempBuffer = std::make_unique<char[]>(bytesRead);
}
catch (...)
{
return STATUS_NO_MEMORY;
}
pBuffer = pwchBufferTmp;
std::unique_ptr<IInputEvent> partialEvent;
bytesRead = TranslateUnicodeToOem(pBuffer,
gsl::narrow<ULONG>(NumToWrite / sizeof(wchar_t)),
tempBuffer.get(),
gsl::narrow<ULONG>(bytesRead),
partialEvent);
if (partialEvent.get())
{
inputBuffer.StoreReadPartialByteSequence(std::move(partialEvent));
}
#pragma prefast(suppress : 26053 26015, "PREfast claims read overflow. *pReadByteCount is the exact size of tempBuffer as allocated above.")
memmove(pBuffer, tempBuffer.get(), bytesRead);
if (addDbcsLead)
{
++bytesRead;
}
}
else
{
// We always return the byte count for A & W modes, so in
// the Unicode case where we didn't translate back, set
// the return to the byte count that we assembled while
// pulling characters from the internal buffers.
bytesRead = NumToWrite;
}
wait = false;
}
return STATUS_SUCCESS;
bytesRead = buffer.size() - writer.size();
return status;
}
NT_CATCH_RETURN()
// Routine Description:
// - This routine reads in characters for stream input and does the
@@ -730,12 +527,16 @@ til::CoordType RetrieveNumberOfSpaces(_In_ til::CoordType sOriginalCursorPositio
}
else
{
return _ReadCharacterInput(inputBuffer,
buffer,
bytesRead,
readHandleState,
unicode,
waiter);
const auto status = ReadCharacterInput(inputBuffer,
buffer,
bytesRead,
readHandleState,
unicode);
if (status == CONSOLE_STATUS_WAIT)
{
waiter = std::make_unique<RAW_READ_DATA>(&inputBuffer, &readHandleState, gsl::narrow<ULONG>(buffer.size()), reinterpret_cast<wchar_t*>(buffer.data()));
}
return status;
}
}
CATCH_RETURN();

6
src/host/stream.h
View File

@@ -29,6 +29,12 @@ Revision History:
_Out_opt_ bool* const pPopupKeys,
_Out_opt_ DWORD* const pdwKeyState) noexcept;
[[nodiscard]] NTSTATUS ReadCharacterInput(InputBuffer& inputBuffer,
std::span<char> buffer,
size_t& bytesRead,
INPUT_READ_HANDLE_DATA& readHandleState,
const bool unicode);
// Routine Description:
// - This routine returns the total number of screen spaces the characters up to the specified character take up.
til::CoordType RetrieveTotalNumberOfSpaces(const til::CoordType sOriginalCursorPositionX,

2
src/host/ut_host/CommandLineTests.cpp
View File

@@ -77,7 +77,7 @@ class CommandLineTests
const size_t cchBuffer)
{
cookedReadData._commandHistory = pHistory;
cookedReadData._userBuffer = pBuffer;
cookedReadData._userBuffer = reinterpret_cast<char*>(pBuffer);
cookedReadData._userBufferSize = cchBuffer * sizeof(wchar_t);
cookedReadData._bufferSize = cchBuffer * sizeof(wchar_t);
cookedReadData._backupLimit = pBuffer;

1
src/host/ut_host/Host.UnitTests.vcxproj
View File

@@ -35,7 +35,6 @@
<ClCompile Include="UtilsTests.cpp" />
<ClCompile Include="Utf8ToWideCharParserTests.cpp" />
<ClCompile Include="InputBufferTests.cpp" />
<ClCompile Include="ReadWaitTests.cpp" />
<ClCompile Include="ViewportTests.cpp" />
<ClCompile Include="VtIoTests.cpp" />
<ClCompile Include="VtRendererTests.cpp" />

3
src/host/ut_host/Host.UnitTests.vcxproj.filters
View File

@@ -54,9 +54,6 @@
<ClCompile Include="InputBufferTests.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="ReadWaitTests.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="ConsoleArgumentsTests.cpp">
<Filter>Source Files</Filter>
</ClCompile>

92
src/host/ut_host/InputBufferTests.cpp
View File

@@ -367,7 +367,7 @@ class InputBufferTests
TEST_METHOD(EmptyingBufferDuringReadSetsResetWaitEvent)
{
Log::Comment(L"ResetWaitEvent should be true if a read to the buffer completely empties it");
Log::Comment(L"hInputEvent should be reset if a read to the buffer completely empties it");
InputBuffer inputBuffer;
@@ -381,48 +381,62 @@ class InputBufferTests
}
VERIFY_IS_GREATER_THAN(inputBuffer.Write(inEvents), 0u);
// read one record, make sure ResetWaitEvent isn't set
std::deque<std::unique_ptr<IInputEvent>> outEvents;
size_t eventsRead = 0;
auto resetWaitEvent = false;
inputBuffer._ReadBuffer(outEvents,
1,
eventsRead,
false,
resetWaitEvent,
true,
false);
VERIFY_ARE_EQUAL(eventsRead, 1u);
VERIFY_IS_FALSE(!!resetWaitEvent);
auto& waitEvent = ServiceLocator::LocateGlobals().hInputEvent;
waitEvent.SetEvent();
// read the rest, resetWaitEvent should be set to true
// read one record, hInputEvent should still be signaled
std::deque<std::unique_ptr<IInputEvent>> outEvents;
VERIFY_NT_SUCCESS(inputBuffer.Read(outEvents,
1,
false,
false,
true,
false));
VERIFY_ARE_EQUAL(outEvents.size(), 1u);
VERIFY_IS_TRUE(waitEvent.is_signaled());
// read the rest, hInputEvent should be reset
waitEvent.SetEvent();
outEvents.clear();
inputBuffer._ReadBuffer(outEvents,
RECORD_INSERT_COUNT - 1,
eventsRead,
false,
resetWaitEvent,
true,
false);
VERIFY_ARE_EQUAL(eventsRead, RECORD_INSERT_COUNT - 1);
VERIFY_IS_TRUE(!!resetWaitEvent);
VERIFY_NT_SUCCESS(inputBuffer.Read(outEvents,
RECORD_INSERT_COUNT - 1,
false,
false,
true,
false));
VERIFY_ARE_EQUAL(outEvents.size(), RECORD_INSERT_COUNT - 1);
VERIFY_IS_FALSE(waitEvent.is_signaled());
}
TEST_METHOD(ReadingDbcsCharsPadsOutputArray)
{
Log::Comment(L"During a non-unicode read, the input buffer should count twice for each dbcs key event");
auto& codepage = ServiceLocator::LocateGlobals().getConsoleInformation().CP;
const auto restoreCP = wil::scope_exit([&, previous = codepage]() {
codepage = previous;
});
codepage = CP_JAPANESE;
// write a mouse event, key event, dbcs key event, mouse event
InputBuffer inputBuffer;
const unsigned int recordInsertCount = 4;
INPUT_RECORD inRecords[recordInsertCount];
std::array<INPUT_RECORD, 4> inRecords{};
inRecords[0].EventType = MOUSE_EVENT;
inRecords[1] = MakeKeyEvent(TRUE, 1, L'A', 0, L'A', 0);
inRecords[2] = MakeKeyEvent(TRUE, 1, 0x3042, 0, 0x3042, 0); // U+3042 hiragana A
inRecords[3].EventType = MOUSE_EVENT;
std::array<INPUT_RECORD, 5> outRecordsExpected{};
outRecordsExpected[0].EventType = MOUSE_EVENT;
outRecordsExpected[1] = MakeKeyEvent(TRUE, 1, L'A', 0, L'A', 0);
outRecordsExpected[2] = MakeKeyEvent(TRUE, 1, 0x3042, 0, 0x82, 0);
outRecordsExpected[3] = MakeKeyEvent(TRUE, 1, 0x3042, 0, 0xa0, 0);
outRecordsExpected[4].EventType = MOUSE_EVENT;
std::deque<std::unique_ptr<IInputEvent>> inEvents;
for (size_t i = 0; i < recordInsertCount; ++i)
for (size_t i = 0; i < inRecords.size(); ++i)
{
inEvents.push_back(IInputEvent::Create(inRecords[i]));
}
@@ -432,22 +446,16 @@ class InputBufferTests
// read them out non-unicode style and compare
std::deque<std::unique_ptr<IInputEvent>> outEvents;
size_t eventsRead = 0;
auto resetWaitEvent = false;
inputBuffer._ReadBuffer(outEvents,
recordInsertCount,
eventsRead,
false,
resetWaitEvent,
false,
false);
// the dbcs record should have counted for two elements in
// the array, making it so that we get less events read
VERIFY_ARE_EQUAL(eventsRead, recordInsertCount - 1);
VERIFY_ARE_EQUAL(eventsRead, outEvents.size());
for (size_t i = 0; i < eventsRead; ++i)
VERIFY_NT_SUCCESS(inputBuffer.Read(outEvents,
outRecordsExpected.size(),
false,
false,
false,
false));
VERIFY_ARE_EQUAL(outEvents.size(), outRecordsExpected.size());
for (size_t i = 0; i < outEvents.size(); ++i)
{
VERIFY_ARE_EQUAL(outEvents[i]->ToInputRecord(), inRecords[i]);
VERIFY_ARE_EQUAL(outEvents[i]->ToInputRecord(), outRecordsExpected[i]);
}
}

129
src/host/ut_host/ReadWaitTests.cpp
View File

@@ -1,129 +0,0 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "precomp.h"
#include "WexTestClass.h"
#include "../../inc/consoletaeftemplates.hpp"
#include "misc.h"
#include "dbcs.h"
#include "../../types/inc/IInputEvent.hpp"
#include "../interactivity/inc/ServiceLocator.hpp"
#include <deque>
#include <memory>
using namespace WEX::Logging;
using Microsoft::Console::Interactivity::ServiceLocator;
class InputRecordConversionTests
{
TEST_CLASS(InputRecordConversionTests);
static const size_t INPUT_RECORD_COUNT = 10;
UINT savedCodepage = 0;
TEST_CLASS_SETUP(ClassSetup)
{
auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
savedCodepage = gci.CP;
gci.CP = CP_JAPANESE;
VERIFY_IS_TRUE(!!GetCPInfo(gci.CP, &gci.CPInfo));
return true;
}
TEST_CLASS_CLEANUP(ClassCleanup)
{
auto& gci = ServiceLocator::LocateGlobals().getConsoleInformation();
gci.CP = savedCodepage;
VERIFY_IS_TRUE(!!GetCPInfo(gci.CP, &gci.CPInfo));
return true;
}
TEST_METHOD(SplitToOemLeavesNonKeyEventsAlone)
{
Log::Comment(L"nothing should happen to input events that aren't key events");
std::deque<std::unique_ptr<IInputEvent>> inEvents;
INPUT_RECORD inRecords[INPUT_RECORD_COUNT] = { 0 };
for (size_t i = 0; i < INPUT_RECORD_COUNT; ++i)
{
inRecords[i].EventType = MOUSE_EVENT;
inRecords[i].Event.MouseEvent.dwMousePosition.X = static_cast<SHORT>(i);
inRecords[i].Event.MouseEvent.dwMousePosition.Y = static_cast<SHORT>(i * 2);
inEvents.push_back(IInputEvent::Create(inRecords[i]));
}
SplitToOem(inEvents);
VERIFY_ARE_EQUAL(INPUT_RECORD_COUNT, inEvents.size());
for (size_t i = 0; i < INPUT_RECORD_COUNT; ++i)
{
VERIFY_ARE_EQUAL(inRecords[i], inEvents[i]->ToInputRecord());
}
}
TEST_METHOD(SplitToOemLeavesNonDbcsCharsAlone)
{
Log::Comment(L"non-dbcs chars shouldn't be split");
std::deque<std::unique_ptr<IInputEvent>> inEvents;
INPUT_RECORD inRecords[INPUT_RECORD_COUNT] = { 0 };
for (size_t i = 0; i < INPUT_RECORD_COUNT; ++i)
{
inRecords[i].EventType = KEY_EVENT;
inRecords[i].Event.KeyEvent.uChar.UnicodeChar = static_cast<wchar_t>(L'a' + i);
inEvents.push_back(IInputEvent::Create(inRecords[i]));
}
SplitToOem(inEvents);
VERIFY_ARE_EQUAL(INPUT_RECORD_COUNT, inEvents.size());
for (size_t i = 0; i < INPUT_RECORD_COUNT; ++i)
{
VERIFY_ARE_EQUAL(inRecords[i], inEvents[i]->ToInputRecord());
}
}
TEST_METHOD(SplitToOemSplitsDbcsChars)
{
Log::Comment(L"dbcs chars should be split");
const auto codepage = ServiceLocator::LocateGlobals().getConsoleInformation().CP;
INPUT_RECORD inRecords[INPUT_RECORD_COUNT * 2] = { 0 };
std::deque<std::unique_ptr<IInputEvent>> inEvents;
// U+3042 hiragana letter A
wchar_t hiraganaA = 0x3042;
wchar_t inChars[INPUT_RECORD_COUNT];
for (size_t i = 0; i < INPUT_RECORD_COUNT; ++i)
{
auto currentChar = static_cast<wchar_t>(hiraganaA + (i * 2));
inRecords[i].EventType = KEY_EVENT;
inRecords[i].Event.KeyEvent.uChar.UnicodeChar = currentChar;
inChars[i] = currentChar;
inEvents.push_back(IInputEvent::Create(inRecords[i]));
}
SplitToOem(inEvents);
VERIFY_ARE_EQUAL(INPUT_RECORD_COUNT * 2, inEvents.size());
// create the data to compare the output to
char dbcsChars[INPUT_RECORD_COUNT * 2] = { 0 };
auto writtenBytes = WideCharToMultiByte(codepage,
0,
inChars,
INPUT_RECORD_COUNT,
dbcsChars,
INPUT_RECORD_COUNT * 2,
nullptr,
FALSE);
VERIFY_ARE_EQUAL(writtenBytes, static_cast<int>(INPUT_RECORD_COUNT * 2));
for (size_t i = 0; i < INPUT_RECORD_COUNT * 2; ++i)
{
const auto pKeyEvent = static_cast<const KeyEvent* const>(inEvents[i].get());
VERIFY_ARE_EQUAL(static_cast<char>(pKeyEvent->GetCharData()), dbcsChars[i]);
}
}
};

57
src/inc/til/bytes.h Normal file
View File

@@ -0,0 +1,57 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#pragma once
#include "type_traits.h"
namespace til
{
template<ContiguousBytes Target>
constexpr void bytes_advance(Target& target, size_t count)
{
if (count > target.size())
{
throw std::length_error{ "insufficient space left" };
}
target = { target.data() + count, target.size() - count };
}
template<TriviallyCopyable T, ContiguousBytes Target>
constexpr bool bytes_can_put(const Target& target)
{
return target.size() >= sizeof(T);
}
template<TriviallyCopyable T, ContiguousBytes Target>
constexpr void bytes_put(Target& target, const T& value)
{
using TargetType = typename Target::value_type;
constexpr auto size = sizeof(value);
if (size > target.size())
{
throw std::length_error{ "insufficient space left" };
}
std::copy_n(reinterpret_cast<const TargetType*>(&value), size, target.data());
target = { target.data() + size, target.size() - size };
}
template<ContiguousBytes Target, ContiguousView Source>
requires TriviallyCopyable<typename Source::value_type>
constexpr void bytes_transfer(Target& target, Source& source)
{
using TargetType = typename Target::value_type;
constexpr auto elementSize = sizeof(typename Source::value_type);
const auto sourceCount = std::min(source.size(), target.size() / elementSize);
const auto targetCount = sourceCount * elementSize;
std::copy_n(reinterpret_cast<const TargetType*>(source.data()), targetCount, target.data());
target = { target.data() + targetCount, target.size() - targetCount };
source = { source.data() + sourceCount, source.size() - sourceCount };
}
}

48
src/inc/til/type_traits.h Normal file
View File

@@ -0,0 +1,48 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#pragma once
namespace til
{
namespace details
{
template<typename T>
struct is_contiguous_view : std::false_type
{
};
template<typename U, std::size_t E>
struct is_contiguous_view<std::span<U, E>> : std::true_type
{
};
template<typename U, typename V>
struct is_contiguous_view<std::basic_string_view<U, V>> : std::true_type
{
};
template<typename T>
struct is_byte : std::false_type
{
};
template<>
struct is_byte<char> : std::true_type
{
};
template<>
struct is_byte<std::byte> : std::true_type
{
};
}
template<typename T>
concept Byte = details::is_byte<std::remove_cv_t<T>>::value;
template<typename T>
concept ContiguousView = details::is_contiguous_view<std::remove_cv_t<T>>::value;
template<typename T>
concept ContiguousBytes = ContiguousView<T> && Byte<typename T::value_type>;
template<typename T>
concept TriviallyCopyable = std::is_trivially_copyable_v<T>;
}

7
src/server/ApiDispatchers.cpp
View File

@@ -270,13 +270,8 @@
// Get output parameter buffer.
PVOID pvBuffer;
ULONG cbBufferSize;
// TODO: This is dumb. We should find out how much we need, not guess.
// If the request is not in Unicode mode, we must allocate an output buffer that is twice as big as the actual caller buffer.
RETURN_IF_FAILED(m->GetAugmentedOutputBuffer((a->Unicode != FALSE) ? 1 : 2,
&pvBuffer,
&cbBufferSize));
RETURN_IF_FAILED(m->GetOutputBuffer(&pvBuffer, &cbBufferSize));
// TODO: This is also rather strange and will also probably make more sense if we stop guessing that we need 2x buffer to convert.
// This might need to go on the other side of the fence (inside host) because the server doesn't know what we're going to do with initial num bytes.
// (This restriction exists because it's going to copy initial into the final buffer, but we don't know that.)
RETURN_HR_IF(E_INVALIDARG, a->InitialNumBytes > cbBufferSize);

2
src/types/inc/IInputEvent.hpp
View File

@@ -64,6 +64,8 @@ public:
inline IInputEvent::~IInputEvent() = default;
using InputEventQueue = std::deque<std::unique_ptr<IInputEvent>>;
#ifdef UNIT_TESTING
std::wostream& operator<<(std::wostream& stream, const IInputEvent* pEvent);
#endif