This is a refresh of spell-check-this, more or less as of
e089393b4e.
## References and Relevant Issues
A number of changes take advantage of features from
http://github.com/check-spelling/check-spelling/releases/v0.0.26
1.
[`load-config-from`](https://docs.check-spelling.dev/Configuration#load-config-from)
will allow future PRs to switch cleanly w/o the mess that this PR has --
once this merges, the `config.json` file will be used for the three
dictionary configuration elements instead of the ones in the workflow.
2. `contents: read` is no longer needed by the comment jobs as the data
is provided by the main job
Contains fixes for the following specific issues:
`without`, `with`, `with the`, `with the window`, `will be`,
`whether or not`, `where the`, `using`, `uppercase or lowercase`,
`unit testing`, `to`, `to which...`, `to which`,
`to which the view refers`, `to which the pane was moved`,
`to run a command/switch to a tab/...`,
`to retrieve the user selected command`, `time,`, `the...that the`,
`the session's initial directory`, `that`, `that will ask`, `that the`,
`that opened the first flyout`, `same as terminal,`, `results,`,
`queue,`, `please`, `pane,`, `out-of-date`, `our`, `one`, `on-screen`,
`often`, `off-screen`, `of a`, `little-endian`, `left over`,
`includes, at a minimum,`, `know of`, `its`,
`if, after the calculation,`, `if`, `if we have an`, `if dragging`,
`if commands`, `guard,`, `given process information in a list`,
`from which`, `from creating`, `for which...`, `for the axis`,
`for initializing the buffer`, `containing the cursor`, `console-wait`,
`change`, `bytes`, `be`, `baseline,`, `aumid`, `at`, `ask me *again*`,
`an`, `also need to`, `again`, `add`, `add event`, `about spelunking`,
(rewrite `Appearances::_UpdateWithNewViewModel` comment), `'a'`, and ` (`
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
This commit fixes a regression in v1.24, where composing Korean text
in-between existing characters causes text to the right to be hidden.
This is problematic for the Korean IME, because it commonly inserts
the composed syllable between existing characters.
The fix compresses (removes/absorbs) available whitespace to the right
of the composition to make space for any existing text. This means
that a composition now virtually acts in insert mode.
Closes#20040
Refs #19738
Refs #20039
## Validation Steps Performed
1. Korean IME: compose `ㄷ` between `가` and `나` → display shows `가ㄷ나`,
`나` visible.
2. TUI box (vim prompt with padding): compose inside padded text box →
border preserved in place.
3. Composition at end of line (no chars to right): unaffected.
4. English and other IME input: not affected (no active composition
row).
Co-authored-by: jason <drvoss@users.noreply.github.com>
Co-authored-by: Leonard Hecker <lhecker@microsoft.com>
We have been looking at a set of conhost crashes due to low-memory
conditions. I've observed that they all happen during the first pass of
rendering, before there's even an engine set up, when we try to estimate
whether there are any blink attributes. I also found that we'll commit
the buffer to check the cursor's double-width status and whether it's on
a double-width-rendition line.
Uncommitted rows _never_ contain a blinker, are never double-width and
never contain DBCS.
We can't necessarily avoid committing an empty buffer _forever,_ but
this at least moves the first commit until after the renderer truly
starts.
This prevents us from committing an empty buffer for headless console
apps (!)
This required the following changes:
- Delayloading ICU in all of our tests, and skipping tests where it
could not be loaded
- Adding a new test group which skips the PolicyTests, which cannot be
run on some editions which have no app platform.
In addition, instead of onboarding new failing test passes, this pull
request finally repairs the broken ConPTY tests (which diverged from the
OSS implementation and had to be re-en-verged)
Closes MSFT-61409507
Reflected from OS PR !15022958
## Summary of the Pull Request
Simplifies the word expansion functions in TextBuffer by removing the
following functions:
- `GetWordStart2()`
- `GetWordEnd2()`
- `MoveToNextWord()`
- `MoveToPreviousWord()`
- `_GetWordStartForAccessibility()`
- `_GetWordStartForSelection()`
- `_GetWordEndForAccessibility()`
- `_GetWordEndForSelection()`
In favor of a simple:
- `GetWordStart()`
- `GetWordEnd()`
- _GetDelimiterClassRunStart()`
- `_GetDelimiterClassRunEnd()`
Tests were used to help ensure a regression doesn't occur. That said,
there were a few modifications there too:
- Removed `MoveByWord` test
- It directly called `MoveToNextWord()` and `MoveToPreviousWord()`,
which no longer exist. These were helper functions for
`UiaTextRangeBase`, and any special logic has been moved into
`_moveEndpointByUnitWord` using the unified
`GetWordStart()`/`GetWordEnd()` APIs.
- The tested behavior is still covered by `MoveToPreviousWord`,
`MovementAtExclusiveEnd`, and the generated word movement tests.
- updated `GetWordBoundaries` tests
- Inclusive --> exclusive end positions for `GetWordEnd()`: The old
`_GetWordEndForSelection()` returned inclusive end positions, whereas
the new `GetWordEnd()` returns exclusive end positions. This is what
`GetWordEnd2()` already used, so every old expected value shifted +1 in
the x direction (or to {RightExclusive, y} at row boundaries) to account
for that.
- `ControlChar` wrap-crossing behavior: The old
`_GetWordStartForSelection()` had a special check at the left boundary
that prevented whitespace runs from crossing wrapped row boundaries. The
new `_GetDelimiterClassRunStart()` doesn't have this special case (it
treats ControlChar the same as other delimiter classes when the row was
wrap-forced). This changed one test case: `GetWordStart({1, 4})` in
selection mode went from `{0, 4}` to `{6, 3}` (the whitespace run now
crosses the wrap boundary to row 3). This matches the behavior
TerminalSelection was already getting from `GetWordStart2()`.
## Validation Steps Performed
Tests passed:
✅ Conhost.Interactivity.Win32.Unit.Tests.dll
✅ UnitTests_TerminalCore\Terminal.Core.Unit.Tests.dll
Word navigation feels good for...
✅ Narrator
✅ NVDA
✅ Mouse selection
✅ Mark mode
Closes#4423
## Summary of the Pull Request
Fixes a bug where a partially visible URL would not be detected. This is
fixed by expanding the search space by 1 viewport height in both
directions.
The `_patternIntervalTree` now operates in the absolute-buffer space as
opposed to the viewport-relative space. It's a bit of an annoying
change, but the alternative would be to keep track of the offset used by
the method above, which I find more annoying, personally. As a part of
this change, I made it a bit more clear when something is
viewport-relative vs buffer-absolute.
Regarding mark mode hyperlink navigation, now that everything is in the
absolute-buffer space, I'm able to fix some of the issues in #13854. I
removed `_selectionIsTargetingUrl` and fixed/validated navigating to
hyperlinks that are partially visible.
## Validation Steps Performed
Detects URL that is...
✅ fully visible
✅ partially cropped off the top
✅ partially cropped off the bottom
✅ Above scenarios work with mark mode hyperlink navigation
✅Tests added
Closes#18177Closes#13854
This essentially rewrites `TerminalInput` from scratch.
There's significant overlap between what kind of information
the Kitty protocol needs from the OS and our existing code.
The rewrite allows us to share large parts of the implementation.
Closes#11509
## Validation Steps Performed
* `kitten show-key -m kitty` ✅
* US Intern. " + ' produces `\'` ✅
* Hebrew base keys produce Unicode ✅
* Hebrew AltGr combinations produce Unicode ✅
* French AltGr+Space produces U+00A0 ✅
* German AltGr+Decimals produce []{}... ✅
Previously, launching an unelevated session after an elevated one would
delete the latter's persisted buffers, and vice versa of course. Also,
elevated buffers didn't have an ACL forbidding access to unelevated
users. That's also fixed now.
Closes#19526
## Validation Steps Performed
* Unelevated/elevated WT doesn't erase each other's buffers ✅
* Old buffers named `buffer_` are renamed to `elevated_` if needed ✅
This PR moves the cursor blinker and VT blink rendition timer into
`Renderer`. To do so, this PR introduces a generic timer system with
which you can schedule arbitrary timer jobs. Thanks to this, this PR
removes a crapton of code, particularly throughout conhost.
## Validation Steps Performed
* Focus/unfocus starts/stops blinking ✅
* OS-wide blink settings apply on focus ✅
Goal: Remove `CursorBlinker`.
Problem: Spooky action at a distance via `Cursor::HasMoved`.
Solution: Moved all the a11y event raising into `_stream.cpp` and pray
for the best.
Goal: Prevent node.js from tanking conhost performance via MSAA (WHY).
Problem: `ServiceLocator`.
Solution: Unserviced the locator. Debounced event raising. Performance
increased by >10x.
Problem 2: Lots of files changed.
This PR is a prerequisite for #19330
## Validation Steps Performed
Ran NVDA with and without UIA enabled and with different delays. ✅
- Various spelling fixes
- Refresh metadata (including dictionaries)
- Upgrade to v0.0.25
## Validation Steps Performed
- check-spelling has been automatically testing this repository for a
while now on a daily basis to ensure that it works fairly reliably:
https://github.com/check-spelling-sandbox/autotest-check-spelling/actions/workflows/microsoft-terminal-spelling2.yml
Specific in-code fixes:
- winget
- whereas
- tl;dr
- set up
- otherwise,
- more,
- macbook
- its
- invalid
- in order to
- if
- if the
- for this tab,...
- fall back
- course,
- cch
- aspect
- archaeologists
- an
- all at once
- a
- `...`
- ; otherwise,
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
## Summary of the Pull Request
Updates the New Tab Menu's Match Profiles entry to support regex instead
of doing a direct match. Also adds validation to ensure the regex is
valid. Updated the UI to help make it more clear that this supports
regexes and even added a link to some helpful docs.
## Validation Steps Performed
✅ Invalid regex displays a warning
✅ Valid regex works nicely
✅ profile matcher with source=`Windows.Terminal.VisualStudio` still
works as expected
## PR Checklist
Closes#18553
The `SGR 1` VT attribute can either be interpreted as a brighter color,
or as a bolder font, depending on the _Intense text style_ setting.
However, the concept of brightness only applies to the eight standard
ANSI colors, so when `SGR 1` is configured as _bright_, it has no effect
on the ITU T.416 colors (RGB and the 256 index colors).
To address that, we now interpret `SGR 1` as a bolder font when applied
to ITU colors, regardless of whether the _Intense text style_ option is
set to bold or not.
Note that this only applies to the Atlas render engine, since the GDI
engine doesn't support bold fonts.
## Validation Steps Performed
I've manually tested `SGR 1` applied to different color formats with the
_Intense text style_ option set to _None_, and confirmed that the text
is now rendered with a bold font for ITU colors, but not for ANSI/AIX
colors.
Closes#18284
This PR fixes two cases where image content wasn't correctly erased when
overwritten.
1. When legacy console APIs fill an area of the buffer using a starting
coordinate and a length, the affected area could potentially wrap over
multiple rows, but we were only erasing the overwritten image content on
the first affected row.
2. When copying an area of the buffer with text content over another
area that contained image content, the image in the target area would
sometimes not be erased, because we ignored the `_eraseCells` return
value which indicated that the image slice needed to be removed.
## References and Relevant Issues
The original code was from the Sixel implementation in PR #17421.
## Validation Steps Performed
I've manually verified that these two cases are now working as expected.
## PR Checklist
- [x] Closes#18568
After taking in 1.22, our CodeQL process caught a few locations where we
weren't following the right guidance:
- Performing integer comparisons of different sizes which could lead to
an infinite loop if the larger integer goes out of range of the smaller
integer
- Not checking HResult of a called method
Co-authored-by: aphistra <102989060+aphistra@users.noreply.github.com>
Selection is generally stored as an inclusive start and end. This PR
makes the end exclusive which now allows degenerate selections, namely
in mark mode. This also modifies mouse selection to round to the nearest
cell boundary (see #5099) and improves word boundaries to be a bit more
modern and make sense for degenerate selections (similar to #15787).
Closes#5099Closes#13447Closes#17892
## Detailed Description of the Pull Request / Additional comments
- Buffer, Viewport, and Point
- Introduced a few new functions here to find word boundaries, delimiter
class runs, and glyph boundaries.
- 📝These new functions should be able to replace a few other functions
(i.e. `GetWordStart` --> `GetWordStart2`). That migration is going to be
a part of #4423 to reduce the risk of breaking UIA.
- Viewport: added a few functions to handle navigating the _exclusive_
bounds (namely allowing RightExclusive as a position for buffer
coordinates). This is important for selection to be able to highlight
the entire line.
- 📝`BottomInclusiveRightExclusive()` will replace `EndExclusive` in the
UIA code
- Point: `iterate_rows_exclusive` is similar to `iterate_rows`, except
it has handling for RightExclusive
- Renderer
- Use `iterate_rows_exclusive` for proper handling (this actually fixed
a lot of our issues)
- Remove some workarounds in `_drawHighlighted` (this is a boundary
where we got inclusive coords and made them exclusive, but now we don't
need that!)
- Terminal
- fix selection marker rendering
- `_ConvertToBufferCell()`: add a param to allow for RightExclusive or
clamp it to RightInclusive (original behavior). Both are useful!
- Use new `GetWordStart2` and `GetWordEnd2` to improve word boundaries
and make them feel right now that the selection an exclusive range.
- Convert a few `IsInBounds` --> `IsInExclusiveBounds` for safety and
correctness
- Add `TriggerSelection` to `SelectNewRegion`
- 📝 We normally called `TriggerSelection` in a different layer, but it
turns out, UIA's `Select` function wouldn't actually update the
renderer. Whoops! This fixes that.
- TermControl
- `_getTerminalPosition` now has a new param to round to the nearest
cell (see #5099)
- UIA
- `TermControlUIAProvider::GetSelectionRange` no need to convert from
inclusive range to exclusive range anymore!
- `TextBuffer::GetPlainText` now works on an exclusive range, so no need
to convert the range anymore!
## Validation Steps Performed
This fundamental change impacts a lot of scenarios:
- ✅Rendering selections
- ✅Selection markers
- ✅Copy text
- ✅Session restore
- ✅Mark mode navigation (i.e. character, word, line, buffer)
- ✅Mouse selection (i.e. click+drag, shift+click, multi-click,
alt+click)
- ✅Hyperlinks (interaction and rendering)
- ✅Accessibility (i.e. get selection, movement, text extraction,
selecting text)
- [ ] Prev/Next Command/Output (untested)
- ✅Unit tests
## Follow-ups
- Refs #4423
- Now that selection and UIA are both exclusive ranges, it should be a
lot easier to deduplicate code between selection and UIA. We should be
able to remove `EndExclusive` as well when we do that. This'll also be
an opportunity to modernize that code and use more `til` classes.
## Summary of the Pull Request
This extends the copy command to be able to include control sequences,
for use in tools that subsequently know how to parse and display that.
## References and Relevant Issues
https://github.com/microsoft/terminal/issues/15703
## Detailed Description of the Pull Request / Additional comments
At a high level, this:
- Expands the `CopyTextArgs` to have a `withControlSequences` bool.
- Plumbs that bool down through many layers to where we actuall get
data out of the text buffer.
- Modifies the existing `TextBuffer::Serialize` to be more generic
and renames it to `TextBuffer::ChunkedSerialize`.
- Uses the new `ChunkedSerialize` to generate the data for the copy
request.
## Validation Steps Performed
To test this I've manually:
- Generated some styled terminal contents, copied it with the control
sequences, pasted it into a file, `cat`ed the file and seen that it
looks the same.
- Set `"firstWindowPreference": "persistedWindowLayout"` and
validated that the contents of windows are saved and
restored with styling intact.
I also checked that `Invoke-OpenConsoleTests` passed.
## PR Checklist
- [x] Closes#15703
- [ ] Tests added/passed
- [x] Documentation updated
- If checked, please file a pull request on [our docs
repo](https://github.com/MicrosoftDocs/terminal) and link it here:
https://github.com/MicrosoftDocs/terminal/pull/756
- [x] Schema updated (if necessary)
* Don't reset the position entirely when changing the needle
* Don't change the scroll position when output arrives
* Don't interfere with the search when output arrives constantly
Closes#17301
## Validation Steps Performed
* In pwsh, run `10000..20000 | % { sleep 0.25; $_ }`
* You can search for e.g. `1004` and it'll find 10 results. ✅
* You can scroll up and down past it and it won't snap back
when new output arrives. ✅
* `while ($true) { Write-Host -NoNewline "`e[Ha"; sleep 0.0001; }`
* You can cycle between the hits effortlessly. ✅ (This tests that
the constantly reset `OutputIdle` event won't interfere.)
* On input change, the focused result is near the previous one. ✅
Because `_layoutLine` would never return `column == columnLimit` for
control character visualizers, we'd get a deadlock in `_redisplay`,
as it tries to fill the line until it's full, but never achieve it.
Closes#17893
## Validation Steps Performed
* Press Ctrl-A to insert "^A"
* Press Home to get to the start of the prompt
* Press and hold "A" until the line wraps
* The line wraps and there's no deadlock ✅
This is particularly relevant to pwsh with the "ghost text" enabled. In
that scenario, pwsh writes out the predicted command to the right of the
cursor. With `showSuggestions(useCommandline=true)`, we'd auto-include
that text in the filter, and that was effectively useless.
This instead defaults us to not use anything to the right of the cursor
(inclusive) for what we consider "the current commandline"
closes#17772
This pull request adds support for setting and querying the selection
color with `OSC 17`.
To make this possible, I had to move selection color down into the color
table where it always belonged. This lets us get rid of the special
`SetSelectionColor` method from the surface of AtlasEngine, and reunites
selection colors with the rest of the special colors.
When we have a series of image slices of differing widths, which also
don't align with the cell boundaries, we can get rounding errors in the
scaling which makes the different slices appear misaligned.
This PR fixes the issue by removing the 4 pixel width alignment that was
enforced in the `ImageSlice` class, since that's not actually necessary
when the pixels themselves are already 4 bytes in size. And without
that, the widths should be correctly aligned with the cell boundaries.
## References and Relevant Issues
The initial Sixel implementation was added in PR #17421.
## Validation Steps Performed
I've confirmed that this fixes the rendering glitches reported in
#17711, and all my existing Sixel tests still work as expected.
Closes#17711
`RealUnicodeToFalseUnicode` was described as:
> This routine converts a unicode string into the correct characters
> for an OEM (cp 437) font. This code is needed because the gdi glyph
> mapper converts unicode to ansi using codepage 1252 to index font.
> This is how the data is stored internally.
In other words, it takes a UCS2 string, translates it to the current
codepage and translates it back to UCS2 in the US version of Windows.
In the "eastern" DBCS version it "reinterprets" the DBCS string as
`CP_USA` (a particularly weird quirk).
The original implementation used to do this translation at every
opportunity where text went into or out of conhost.
The translation was weird, but it was consistent.
In Windows 10 RS1 conhost got a new UCS2-aware text buffer and
this translation was removed from most places, as the text buffer
was converted to store proper UCS2. This broke the entire concept
of the translation though. Whatever data you previously wrote with
something like `WriteConsoleOutputCharacter` now came back with
something entirely else via `ReadConsoleOutput`.
In other words, I believe past RS1 there was technically never any
point in "munging" `CHAR_INFO`s, as this only covered 2 API functions.
Still, this does mean that this PR represents an API breaking change.
It's a minor one though, because it only affects 2 API functions.
And more importantly, it's a necessary breaking change as we move
further and further away from correlating codepoint and column counts.
## Validation Steps Performed
* Remaining tests pass ✅
Roughly 4 years ago we gave Windows Terminal the ability to
differentiate between black/white and the default colors.
One of the victims was PowerShell and most importantly PSReadLine,
which emit SRG 37 & 40 when what they really want is 38 & 48.
We fixed this on our side by adding a shim.
Since the addition of VT passthrough in #17510 we now intentionally
lost the ability to translate VT sequences from one thing to another.
This meant we also lost the ability to do this shim and as such
this PR removes it. Luckily Windows 11 now ships PSReadLine 2.0.0,
which contains a proper fix for this.
Unfortunately, this is not the case for Windows 10, which ships
PSReadLine 2.0.0-beta2. Users affected by this will have to install
a newer version of PSReadLine or use the default black/white theme.
See 1bf4c082b4Closes#13037
The idea is that we can translate Console API calls directly to VT at
least as well as the current VtEngine setup can. For instance, a call
to `SetConsoleCursorPosition` clearly translates directly to a `CUP`
escape sequence. Effectively, instead of translating output
asynchronously in the renderer thread, we'll do it synchronously
right during the Console API call.
Most importantly, the this means that any VT output that an
application generates will now be given to the terminal unmodified.
Aside from reducing our project's complexity quite a bit and opening
the path towards various interesting work like sixels, Device Control
Strings, buffer snapshotting, synchronized updates, and more, it also
improves performance for mixed text output like enwik8.txt in conhost
to 1.3-2x and in Windows Terminal via ConPTY to roughly 20x.
This adds support for overlapped IO, because now that output cannot
be "skipped" anymore (VtEngine worked like a renderer after all)
it's become crucial to block conhost as little as possible.
⚠️ Intentionally unresolved changes/quirks:
* To force a delayed EOL wrap to wrap, `WriteCharsLegacy` emits a
`\r\n` if necessary. This breaks text reflow on window resize.
We cannot emit ` \r` the way readline does it, because this would
overwrite the first column in the next row with a whitespace.
The alternative is to read back the affected cell from the buffer
and emit that character and its attributes followed by a `\r`.
I chose to not do that, because buffer read-back is lossy (= UCS2).
Unless the window is resized, the difference is unnoticeable
and historically, conhost had no support for buffer reflow anyway.
* If `ENABLE_VIRTUAL_TERMINAL_PROCESSING` is set while
`DISABLE_NEWLINE_AUTO_RETURN` is reset, we'll blindly replace all
LF with CRLF. This may hypothetically break DCS sequences, but it's
the only way to do this without parsing the given VT string and
thus the only way we can achieve passthrough mode in the future.
* `ENABLE_WRAP_AT_EOL_OUTPUT` is translated to `DECAWM`.
Between Windows XP and Windows 11 21H2, `ENABLE_WRAP_AT_EOL_OUTPUT`
being reset would cause the cursor position to reset to wherever
a write started, _if_ the write, including expanded control chars,
was less than 100 characters long. If it was longer than that,
the cursor position would end up in an effectively random position.
After lengthy research I believe that this is a bug introduced in
Windows XP and that the original intention was for this mode to be
equivalent to `DECAWM`. This is compounded by MSDN's description
(emphasis mine):
> If this mode is disabled, the **last character** in the row is
> overwritten with any subsequent characters.
⚠️ Unresolved issues/quirks:
* Focus/Unfocus events are injected into the output stream without
checking whether the VT output is currently in a ground state.
This may break whatever VT sequence is currently ongoing.
This is an existing issue.
* `VtIo::Writer::WriteInfos` should properly verify the width of
each individual character.
* Using `SetConsoleActiveScreenBuffer` destroys surrogate pairs
and extended (VT) attributes. It could be translated to VT pages
in the long term.
* Similarly, `ScrollConsoleScreenBuffer` results in the same and
could be translated to `DECCRA` and `DECFRA` in the near term.
This is important because otherwise `vim` output may loose
its extended attributes during scrolling.
* Reflowing a long line until it wraps results in the cooked read
prompt to be misaligned vertically.
* `SCREEN_INFORMATION::s_RemoveScreenBuffer` should trigger a
buffer switch similar to `SetConsoleActiveScreenBuffer`.
* Translation of `COMMON_LVB_GRID_HORIZONTAL` to `SGR 53` was dropped
and may be reintroduced alongside `UNDERSCORE` = `SGR 4`.
* Move the `OSC 0 ; P t BEL` sequence to `WriteWindowTitle`
and swap the `BEL` with the `ST` (`ESC \`).
* PowerShell on Windows 10 ships with PSReadLine 2.0.0-beta2
which emits SGR 37/40 instead of 39/49. This results in black
spaces when typing and there's no good way to fix that.
* A test is missing that ensures that `FillConsoleOutputCharacterW`
results in a `CSI n J` during the PowerShell shim.
* A test is missing that ensures that `PtySignal::ClearBuffer`
does not result in any VT being generated.
Closes#262Closes#1173Closes#3016Closes#4129Closes#5228Closes#8698Closes#12336Closes#15014Closes#15888Closes#16461Closes#16911Closes#17151Closes#17313
The "copy the remaining attributes" loop assumes that it has full
ownership over the rows that it copies. For that to be true,
we have to of course make sure that the current write-cursor
is at a fresh, new row in the first place.
## Validation Steps Performed
* In a new pwsh tab with 120 colums:
``Write-Host -NoNewline "`e[36m$('a'*120)`e[m"; sleep 10``
* Resize the window wider
* Color doesn't get lost
* Add a revision to `ImageSlice` so that the renderers
can use it to cache them as bitmaps across frames.
* Hooked up the revision tracking to AtlasEngine to cache the
slices into `Buffer`s so we can own them into the `Present`.
* Hooked up those snapshots to BackendD3D with a straightforward
hashmap -> atlas-rect logic. Just like rendering text.
* Hooked up BackendD2D with a bad, but simple & direct drawing logic.
* Bonus: Modify `ImageSlice` to be returned as a raw pointers
as this helps performance slightly. (Trivial type == good.)
* Bonus: Fixed the `_debugShowDirty` code (disabled by default).
## Validation Steps Performed
* `mpv --really-quiet --vo=sixel foo.mp4` looks good ✅
* Scroll up down & observe dirty rects ✅
This abstraction will help #17510 inject its ConPTY-specific behavior
into all 6 relevant console API functions simultaneously. This avoids
having to repeat the same prologue and epilogue 4 times.
Ideally, we'd use composition here, but I found it to be a bad fit.
* Wide glyphs that don't fit into the last column got treated
as narrow glyphs which broke line layout.
* Wide glyphs that don't fit into the last column got manually
padded with whitespace which broke Ctrl+A + Ctrl+C in conhost.
* Sudden increases/decreases in the pager height would leave
parts of the viewport with leftover text and not clear it away.
* Deleting an entire word at the start of a line would only delete
its first two characters.
Closes#17554
STL iterators have a significant overhead. This improves performance
of `GetLastNonSpaceColumn` by >100x (it's too large to measure),
and reflow by ~15x in debug builds. This makes text reflow in debug
builds today ~10x faster than it used to be in release builds before
the large rewrites in #15701 and #13626.
By rewriting `COOKED_READ_DATA` to use VT for its output we make it
possible to pass this VT output 1:1 straight to the hosting terminal
if we're running under ConPTY. This is also possible with the current
console APIs it uses, but it's somewhat janky. In particular the
usage of `ReadConsoleOutput` to backup/restore the popup contents
could be considered bad faith "rules for thee, not for me",
given that we're telling people to move away from those APIs.
The new implementation contains a bare bones "pager" to fit even
very long prompt contents into the VT viewport.
I fully expect this initial PR to not be entirely bug free, because
writing a proper pager with line wrapping is a little bit complex.
This PR takes some significant shortcuts by leveraging the fact
that the prompt line is always left-to-right and always a series
of fully filled lines followed by one potentially semi-full line.
This allows us to skip using a front/back-buffer for diffing the
contents between two redisplay calls.
Part of #14000
## Validation Steps Performed
* ASCII input
* Chinese input (中文維基百科) ✅
* Surrogate pair input (🙂) ✅
* In cmd.exe
* Create 2 files: "a😊b.txt" and "a😟b.txt"
* Press tab: Autocomplete to "a😊b.txt" ✅
* Navigate the cursor right past the "a"
* Press tab twice: Autocomplete to "a😟b.txt" ✅
* Execute `printf(" "); gets(buffer);` in C (or equivalent)
* Press Tab, A, Ctrl+V, Tab, A ✅
* The prompt is " A^V A" ✅
* Cursor navigation works ✅
* Backspacing/Deleting random parts of it works ✅
* It never deletes the initial 4 spaces ✅
* Backspace deletes preceding glyphs ✅
* Ctrl+Backspace deletes preceding words ✅
* Escape clears input ✅
* Home navigates to start ✅
* Ctrl+Home deletes text between cursor and start ✅
* End navigates to end ✅
* Ctrl+End deletes text between cursor and end ✅
* Left navigates over previous code points ✅
* Ctrl+Left navigates to previous word-starts ✅
* Right and F1 navigate over next code points ✅
* Pressing right at the end of input copies characters
from the previous command ✅
* Ctrl+Right navigates to next word-ends ✅
* Insert toggles overwrite mode ✅
* Delete deletes next code point ✅
* Up and F5 cycle through history ✅
* Doesn't crash with no history ✅
* Stops at first entry ✅
* Down cycles through history ✅
* Doesn't crash with no history ✅
* Stops at last entry ✅
* PageUp retrieves the oldest command ✅
* PageDown retrieves the newest command ✅
* F2 starts "copy to char" prompt ✅
* Escape dismisses prompt ✅
* Typing a character copies text from the previous command up
until that character into the current buffer (acts identical
to F3, but with automatic character search) ✅
* F3 copies the previous command into the current buffer,
starting at the current cursor position,
for as many characters as possible ✅
* Doesn't erase trailing text if the current buffer
is longer than the previous command ✅
* Puts the cursor at the end of the copied text ✅
* F4 starts "copy from char" prompt ✅
* Escape dismisses prompt ✅
* Erases text between the current cursor position and the
first instance of a given char (but not including it) ✅
* F6 inserts Ctrl+Z ✅
* F7 without modifiers starts "command list" prompt ✅
* Escape dismisses prompt ✅
* Entries wider than the window width are truncated ✅
* Height expands up to 20 rows with longer histories ✅
* F9 starts "command number" prompt ✅
* Left/Right replace the buffer with the given command ✅
* And put cursor at the end of the buffer ✅
* Up/Down navigate selection through history ✅
* Stops at start/end with <10 entries ✅
* Stops at start/end with >20 entries ✅
* Scrolls through the entries if there are too many ✅
* Shift+Up/Down moves history items around ✅
* Home navigates to first entry ✅
* End navigates to last entry ✅
* PageUp navigates by $height items at a time or to first ✅
* PageDown navigates by $height items at a time or to last ✅
* Alt+F7 clears command history ✅
* F8 cycles through commands that start with the same text as
the current buffer up until the current cursor position ✅
* Doesn't crash with no history ✅
* F9 starts "command number" prompt ✅
* Escape dismisses prompt ✅
* Ignores non-ASCII-decimal characters ✅
* Allows entering between 1 and 5 digits ✅
* Pressing Enter fetches the given command from the history ✅
* Alt+F10 clears doskey aliases ✅
* In cmd.exe, with an empty prompt in an empty directory:
Pressing tab produces an audible bing and prints no text ✅
* When Narrator is enabled, in cmd.exe:
* Typing individual characters announces only
exactly each character that is being typed ✅
* Backspacing at the end of a prompt announces
only exactly each deleted character ✅
This removes all of the 2D iteration machinery. Imagine the text buffer
as a `Cell[w][h]` grid. Clearly, this is identical to a `Cell[w*h]`
array, which shows that copying between overlapping ranges only needs
either forward or backward copying, and not left/right/top/down.
With `WalkDir` removed, `WalkInBounds` can be rewritten with basic
arithmetic which allows `pos` to be an exclusive end coordinate.
## Summary of the Pull Request
This PR introduces basic support for the Sixel graphics protocol in
conhost, limited to the GDI renderer.
## References and Relevant Issues
This is a first step towards supporting Sixel graphics in Windows
Terminal (#448), but that will first require us to have some form of
ConPTY passthrough (#1173).
## Detailed Description of the Pull Request / Additional comments
There are three main parts to the architecture:
* The `SixelParser` class takes care of parsing the incoming Sixel `DCS`
sequence.
* The resulting image content is stored in the text buffer in a series
of `ImageSlice` objects, which represent per-row image content.
* The renderer then takes care of painting those image slices for each
affected row.
The parser is designed to support multiple conformance levels so we can
one day provide strict compatibility with the original DEC hardware. But
for now the default behavior is intended to work with more modern Sixel
applications. This is essentially the equivalent of a VT340 with 256
colors, so it should still work reasonably well as a VT340 emulator too.
## Validation Steps Performed
Thanks to the work of @hackerb9, who has done extensive testing on a
real VT340, we now have a fairly good understanding of how the original
Sixel hardware terminals worked, and I've tried to make sure that our
implementation matches that behavior as closely as possible.
I've also done some testing with modern Sixel libraries like notcurses
and jexer, but those typically rely on the terminal implementing certain
proprietary Xterm query sequences which I haven't included in this PR.
---------
Co-authored-by: Dustin L. Howett <dustin@howett.net>
First, this adds `GraphemeTableGen` which
* parses `ucd.nounihan.grouped.xml`
* computes the cluster break property for each codepoint
* computes the East Asian Width property for each codepoint
* compresses everything into a 4-stage trie
* computes a LUT of cluster break rules between 2 codepoints
* and serializes everything to C++ tables and helper functions
Next, this adds `GraphemeTestTableGen` which
* parses `GraphemeBreakTest.txt`
* splits each test into graphemes and break opportunities
* and serializes everything to a C++ table for use as unit tests
`CodepointWidthDetector.cpp` was rewritten from scratch to
* use an iterator struct (`GraphemeState`) to maintain state
* accumulate codepoints until a break opportunity arises
* accumulate the total width of a grapheme
* support 3 different measurement modes: Grapheme clusters,
`wcswidth`-style, and a mode identical to the old conhost
With this in place the following changes were made:
* `ROW::WriteHelper::_replaceTextUnicode` now uses the new
grapheme cluster text iterators
* The same function was modified to join new text with existing
contents of the current cell if they join to form a cluster
* Otherwise, a ton of places were modified to funnel the selection
of the measurement mode over from WT's settings to ConPTY
This is part of #1472
## Validation Steps Performed
* So many tests ✅
* https://github.com/apparebit/demicode works fantastic ✅
* UTF8-torture-test.txt works fantastic ✅
If `VtEngine` gets removed from conhost, we need to be able to run
without any renderer present whatsoever. To make this possible,
I've turned all `Renderer&` into `Renderer*`.
Part of #14000
I think I forgot to complete that section of the code...
The parentheses were missing and `beg` was repeated twice. The last
line in the comment above this explains what I intended it to be.
Closes#17365
## Validation Steps Performed
* In a new PowerShell tab
* Run ``"`e[999C`e[2D`e[42mfoo`e[m"``
* Newline until it scrolls
* Run it again
* Close and reopen
* The green "foo" is still green ✅
`utextAccess` apparently doesn't actually need to clamp the
`chunkOffset` to be in range of the current chunk. Also, I missed to
implement the part of the spec that says to leave the iterator on the
first/last chunk of the `UText` in case of an out-of-bounds index.
This PR fixes the issue by simply not returning early, doing a more
liberal clamp of the offset, and then checking whether it was in range.
As an aside, this also fixes a one-off bug when hovering URLs that
end on the very last cell of the viewport (or are cut off).
Closes#17343
## Validation Steps Performed
* Write an URL that wraps across the last 2 lines in the buffer
* Scroll 1 line up
* No assert ✅
* Hovering the URL shows the full, still visible parts of the URL ✅
It may be more accurate to say: "Fix _known_ remaining buffer
serialization bugs", but I'll try to be positive about my code.
Initially, the buffer is initialized with the default attributes,
but once it begins to scroll, newly scrolled in rows are initialized
with the current attributes. This means we need to set the current
attributes to those of the upcoming row before the row comes up.
This is related to #17074.
## Validation Steps Performed
* Persist and restore a buffer 10 times
* All previous "Restore" status messages look correct ✅
* The escape sequences in the buffer file look correct ✅
This addresses a review comment left by tusharsnx in #17092 which I
forgot to fix before merging the PR. The fix itself is somewhat simple:
`Terminal::SetSearchHighlightFocused` triggers a scroll if the target
is outside of the current (scrolled) viewport and avoiding the call
unless necessary fixes it. To do it properly though, I've split up
`Search::ResetIfStale` into `IsStale` and `Reset`. Now we can properly
detect staleness in advance and branch out the search reset cleanly.
Additionally, I've taken the liberty to replace the `IVector` in
`SearchResultRows` with a direct `const std::vector&` into `Searcher`.
This removes a bunch of code and makes it faster to boot.
## Validation Steps Performed
* Print lots of text
* Search a common letter
* Scroll up
* Doesn't scroll back down ✅
* Hold enter to search more occurrences scrolls up as needed ✅
* `showMarksOnScrollbar` still works ✅
This PR achieves two things:
* When encountering rows with newlines (`WasForceWrapped` = `false`)
we'll now copy the contents out of the row and append a `\n`.
To make `utext_clone` cheap, it adds a reference counted buffer.
* Text extraction in `Terminal::GetHyperlinkAtBufferPosition`
was fixed by using a higher level `TextBuffer::GetPlainText`
instead of iterating through each cell.
Closes#16676Closes#17065
## Validation Steps Performed
* In pwsh execute the following:
``"`e[999C`e[22Dhttps://example.com/foo`nbar"``
* Hovering over the URL only underlines `.../foo` and not `bar` ✅
* The tooltip ends in `.../foo` and not `.../fo` ✅
This PR extends `til::throttled_func` to also support debouncing:
* throttling: "At most 1 call every N seconds"
* debouncing: "Exactly 1 call after N seconds of inactivity"
Based on the latter the following series of changes were made:
* An `OutputIdle` event was added to `ControlCore` which is
raised once there hasn't been any incoming data in 100ms.
This also triggers an update of our regex patterns (URL detection).
* The event is then caught by `TermControl` which calls `Search()`.
* `Search()` in turn was modified to return its results by-value
as a struct, which avoids the need for a search-update event
and simplifies how we update the UI.
This architectural change, most importantly the removal of the
`TextLayoutUpdated` event, fixes a DoS bug in Windows Terminal:
As the event leads to UI thread activity, printing lots of text
continuously results in the UI thread becoming unresponsive.
On top of these, a number of improvements were made:
* `IRenderEngine::InvalidateHighlight` was changed to take the
`TextBuffer` by-reference which avoids the need to accumulate the
line renditions in a `std::vector` first. This improves Debug build
performance during reflow by what I guess must be roughly
a magnitude faster. This difference is very noticeable.
* When closing the search box, `ClearSearch()` is called to remove
the highlights. The search text is restored when it's reopened,
however the current search position isn't.
Closes#17073Closes#17089
## Validation Steps Performed
* UIA announcements:
* Pressing Ctrl+Shift+F the first time does not lead to one ✅
* Typing the first letter does ✅
* Closing doesn't ✅
* Reopening does (as it restores the letter) ✅
* Closing the search box dismisses the highlights ✅
* Resizing the window recalculates the highlights ✅
* Changing the terminal output while the box is open
recalculates the highlights ✅
