Files
terminal/src/cascadia/TerminalControl/TSFInputControl.cpp

383 lines
17 KiB
C++
Raw Normal View History

Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
// Copyright (c) Microsoft Corporation.
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Licensed under the MIT license.
#include "pch.h"
#include "TSFInputControl.h"
#include "TSFInputControl.g.cpp"
#include <Utils.h>
using namespace winrt::Windows::Foundation;
using namespace winrt::Windows::Graphics::Display;
using namespace winrt::Windows::UI::Core;
using namespace winrt::Windows::UI::Text;
using namespace winrt::Windows::UI::Text::Core;
using namespace winrt::Windows::UI::Xaml;
namespace winrt::Microsoft::Terminal::TerminalControl::implementation
{
TSFInputControl::TSFInputControl() :
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
_editContext{ nullptr },
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
_inComposition{ false },
_activeTextStart{ 0 }
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
{
InitializeComponent();
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Create a CoreTextEditingContext for since we are acting like a custom edit control
auto manager = Core::CoreTextServicesManager::GetForCurrentView();
_editContext = manager.CreateEditContext();
// InputPane is manually shown inside of TermControl.
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_editContext.InputPaneDisplayPolicy(Core::CoreTextInputPaneDisplayPolicy::Manual);
// set the input scope to Text because this control is for any text.
_editContext.InputScope(Core::CoreTextInputScope::Text);
_textRequestedRevoker = _editContext.TextRequested(winrt::auto_revoke, { this, &TSFInputControl::_textRequestedHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_selectionRequestedRevoker = _editContext.SelectionRequested(winrt::auto_revoke, { this, &TSFInputControl::_selectionRequestedHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_focusRemovedRevoker = _editContext.FocusRemoved(winrt::auto_revoke, { this, &TSFInputControl::_focusRemovedHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_textUpdatingRevoker = _editContext.TextUpdating(winrt::auto_revoke, { this, &TSFInputControl::_textUpdatingHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_selectionUpdatingRevoker = _editContext.SelectionUpdating(winrt::auto_revoke, { this, &TSFInputControl::_selectionUpdatingHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_formatUpdatingRevoker = _editContext.FormatUpdating(winrt::auto_revoke, { this, &TSFInputControl::_formatUpdatingHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_layoutRequestedRevoker = _editContext.LayoutRequested(winrt::auto_revoke, { this, &TSFInputControl::_layoutRequestedHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_compositionStartedRevoker = _editContext.CompositionStarted(winrt::auto_revoke, { this, &TSFInputControl::_compositionStartedHandler });
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_compositionCompletedRevoker = _editContext.CompositionCompleted(winrt::auto_revoke, { this, &TSFInputControl::_compositionCompletedHandler });
}
// Method Description:
// - Prepares this TSFInputControl to be removed from the UI hierarchy.
void TSFInputControl::Close()
{
// Explicitly disconnect the LayoutRequested handler -- it can cause problems during application teardown.
// See GH#4159 for more info.
_layoutRequestedRevoker.revoke();
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
}
// Method Description:
// - NotifyFocusEnter handler for notifying CoreEditTextContext of focus enter
// when TerminalControl receives focus.
// Arguments:
// - <none>
// Return Value:
// - <none>
void TSFInputControl::NotifyFocusEnter()
{
if (_editContext != nullptr)
{
_editContext.NotifyFocusEnter();
}
}
// Method Description:
// - NotifyFocusEnter handler for notifying CoreEditTextContext of focus leaving.
// when TerminalControl no longer has focus.
// Arguments:
// - <none>
// Return Value:
// - <none>
void TSFInputControl::NotifyFocusLeave()
{
if (_editContext != nullptr)
{
_editContext.NotifyFocusLeave();
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
}
}
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
// Method Description:
// - Clears the input buffer and tells the text server to clear their buffer as well.
// Also clears the TextBlock and sets the active text starting point to 0.
// Arguments:
// - <none>
// Return Value:
// - <none>
void TSFInputControl::ClearBuffer()
{
if (!_inputBuffer.empty())
{
TextBlock().Text(L"");
const auto bufLen = ::base::ClampedNumeric<int32_t>(_inputBuffer.length());
_inputBuffer.clear();
_editContext.NotifyFocusLeave();
_editContext.NotifyTextChanged({ 0, bufLen }, 0, { 0, 0 });
_editContext.NotifyFocusEnter();
_activeTextStart = 0;
_inComposition = false;
}
}
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Method Description:
// - Handler for LayoutRequested event by CoreEditContext responsible
// for returning the current position the IME should be placed
// in screen coordinates on the screen. TSFInputControls internal
// XAML controls (TextBlock/Canvas) are also positioned and updated.
// NOTE: documentation says application should handle this event
// Arguments:
// - sender: CoreTextEditContext sending the request.
// - args: CoreTextLayoutRequestedEventArgs to be updated with position information.
// Return Value:
// - <none>
void TSFInputControl::_layoutRequestedHandler(CoreTextEditContext sender, CoreTextLayoutRequestedEventArgs const& args)
{
auto request = args.Request();
// Get window in screen coordinates, this is the entire window including tabs
const auto windowBounds = CoreWindow::GetForCurrentThread().Bounds();
// Get the cursor position in text buffer position
auto cursorArgs = winrt::make_self<CursorPositionEventArgs>();
_CurrentCursorPositionHandlers(*this, *cursorArgs);
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
const COORD cursorPos = { ::base::ClampedNumeric<short>(cursorArgs->CurrentPosition().X), ::base::ClampedNumeric<short>(cursorArgs->CurrentPosition().Y) };
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Get Font Info as we use this is the pixel size for characters in the display
auto fontArgs = winrt::make_self<FontInfoEventArgs>();
_CurrentFontInfoHandlers(*this, *fontArgs);
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
const auto fontWidth = fontArgs->FontSize().Width;
const auto fontHeight = fontArgs->FontSize().Height;
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Convert text buffer cursor position to client coordinate position within the window
COORD clientCursorPos;
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
clientCursorPos.X = ::base::ClampMul(cursorPos.X, ::base::ClampedNumeric<short>(fontWidth));
clientCursorPos.Y = ::base::ClampMul(cursorPos.Y, ::base::ClampedNumeric<short>(fontHeight));
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Convert from client coordinate to screen coordinate by adding window position
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
COORD screenCursorPos;
screenCursorPos.X = ::base::ClampAdd(clientCursorPos.X, ::base::ClampedNumeric<short>(windowBounds.X));
screenCursorPos.Y = ::base::ClampAdd(clientCursorPos.Y, ::base::ClampedNumeric<short>(windowBounds.Y));
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// get any offset (margin + tabs, etc..) of the control within the window
const auto offsetPoint = this->TransformToVisual(nullptr).TransformPoint(winrt::Windows::Foundation::Point(0, 0));
// add the margin offsets if any
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
screenCursorPos.X = ::base::ClampAdd(screenCursorPos.X, ::base::ClampedNumeric<short>(offsetPoint.X));
screenCursorPos.Y = ::base::ClampAdd(screenCursorPos.Y, ::base::ClampedNumeric<short>(offsetPoint.Y));
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Get scale factor for view
const double scaleFactor = DisplayInformation::GetForCurrentView().RawPixelsPerViewPixel();
// position textblock to cursor position
Canvas().SetLeft(TextBlock(), clientCursorPos.X);
Canvas().SetTop(TextBlock(), ::base::ClampedNumeric<double>(clientCursorPos.Y));
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// calculate FontSize in pixels from DIPs
const double fontSizePx = (fontHeight * 72) / USER_DEFAULT_SCREEN_DPI;
TextBlock().FontSize(fontSizePx);
TextBlock().FontFamily(Media::FontFamily(fontArgs->FontFace()));
Process actions sync. on startup; don't dupe nonexistent profile (#5090) This PR has evolved to encapsulate two related fixes that I can't really untie anymore. #2455 - Duplicating a tab that doesn't exist anymore This was the bug I was originally fixing in #4429. When the user tries to `duplicateTab` with a profile that doesn't exist anymore (like might happen after a settings reload), don't crash. As I was going about adding tests for this, got blocked by the fact that the Terminal couldn't open _any_ panes while the `TerminalPage` was size 0x0. This had two theoretical solutions: * Fake the `TerminalPage` into thinking it had a real size in the test - probably possible, though I'm unsure how it would work in practice. * Change `Pane`s to not require an `ActualWidth`, `ActualHeight` on initialization. Fortuately, the second option was something else that was already on my backlog of bugs. #4618 - `wt` command-line can't consistently parse more than one arg Presently, the Terminal just arbitrarily dispatches a bunch of handlers to try and handle all the commands provided on the commandline. That's lead to a bunch of reports that not all the commands will always get executed, nor will they all get executed in the same order. This PR also changes the `TerminalPage` to be able to dispatch all the commands sequentially, all at once in the startup. No longer will there be a hot second where the commands seem to execute themselves in from of the user - they'll all happen behind the scenes on startup. This involved a couple other changes areound the `TerminalPage` * I had to make sure that panes could be opened at a 0x0 size. Now they use a star sizing based off the percentage of the parent they're supposed to consume, so that when the parent _does_ get laid out, they'll take the appropriate size of that parent. * I had to do some math ahead of time to try and calculate what a `SplitState::Automatic` would be evaluated as, despite the fact that we don't actually know how big the pane will be. * I had to ensure that `focus-tab` commands appropriately mark a single tab as focused while we're in startup, without roundtripping to the Dispatcher thread and back ## References #4429 - the original PR for #2455 #5047 - a follow-up task from discussion in #4429 #4953 - a PR for making panes use star sizing, which was immensly helpful for this PR. ## Detailed Description of the Pull Request / Additional comments `CascadiaSettings::BuildSettings` can throw if the GUID doesn't exist. This wraps those calls up with a try/catch. It also adds a couple tests - a few `SettingsTests` for try/catching this state. It also adds a XAML-y test in `TabTests` that creates a `TerminalPage` and then performs som UI-like actions on it. This test required a minor change to how we generate the new tab dropdown - in the tests, `Application::Current()` is _not_ a `TerminalApp::App`, so it doesn't have a `Logic()` to query. So wrap that in a try/catch as well. While working on these tests, I found that we'd crash pretty agressively for mysterious reasons if the TestHostApp became focused while the test was running. This was due to a call in `TSFInputControl::NotifyFocusEnter` that would callback to `TSFInputControl::_layoutRequested`, which would crash on setting the `MaxSize` of the canvas to a negative value. This PR includes a hotfix for that bug as well. ## Validation Steps Performed * Manual testing with a _lot_ of commands in a commandline * run the tests * Team tested in selfhost Closes #2455 Closes #4618
2020-03-25 19:03:32 -05:00
const auto canvasActualWidth = Canvas().ActualWidth();
const auto widthToTerminalEnd = canvasActualWidth - ::base::ClampedNumeric<double>(clientCursorPos.X);
// Make sure that we're setting the MaxWidth to a positive number - a
// negative number here will crash us in mysterious ways with a useless
// stack trace
const auto newMaxWidth = std::max<double>(0.0, widthToTerminalEnd);
TextBlock().MaxWidth(newMaxWidth);
// Set the text block bounds
const auto yOffset = ::base::ClampedNumeric<float>(TextBlock().ActualHeight()) - fontHeight;
const auto textBottom = ::base::ClampedNumeric<float>(screenCursorPos.Y) + yOffset;
Rect selectionRect = Rect(screenCursorPos.X, textBottom, 0, fontHeight);
request.LayoutBounds().TextBounds(ScaleRect(selectionRect, scaleFactor));
// Set the control bounds of the whole control
Rect controlRect = Rect(screenCursorPos.X, screenCursorPos.Y, 0, fontHeight);
request.LayoutBounds().ControlBounds(ScaleRect(controlRect, scaleFactor));
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
}
// Method Description:
// - Handler for CompositionStarted event by CoreEditContext responsible
// for making internal TSFInputControl controls visible.
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - args: CoreTextCompositionStartedEventArgs. Not used in method.
// Return Value:
// - <none>
void TSFInputControl::_compositionStartedHandler(CoreTextEditContext sender, CoreTextCompositionStartedEventArgs const& /*args*/)
{
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
_inComposition = true;
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
}
// Method Description:
// - Handler for CompositionCompleted event by CoreEditContext responsible
// for making internal TSFInputControl controls visible.
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - args: CoreTextCompositionCompletedEventArgs. Not used in method.
// Return Value:
// - <none>
void TSFInputControl::_compositionCompletedHandler(CoreTextEditContext sender, CoreTextCompositionCompletedEventArgs const& /*args*/)
{
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
_inComposition = false;
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// only need to do work if the current buffer has text
if (!_inputBuffer.empty())
{
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
_SendAndClearText();
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
}
}
// Method Description:
// - Handler for FocusRemoved event by CoreEditContext responsible
// for removing focus for the TSFInputControl control accordingly
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
// when focus was forcibly removed from text input control.
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// NOTE: Documentation says application should handle this event
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - object: CoreTextCompositionStartedEventArgs. Not used in method.
// Return Value:
// - <none>
void TSFInputControl::_focusRemovedHandler(CoreTextEditContext sender, winrt::Windows::Foundation::IInspectable const& /*object*/)
{
}
// Method Description:
// - Handler for TextRequested event by CoreEditContext responsible
// for returning the range of text requested.
// NOTE: Documentation says application should handle this event
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - args: CoreTextTextRequestedEventArgs to be updated with requested range text.
// Return Value:
// - <none>
void TSFInputControl::_textRequestedHandler(CoreTextEditContext sender, CoreTextTextRequestedEventArgs const& args)
{
// the range the TSF wants to know about
const auto range = args.Request().Range();
try
{
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
const auto textEnd = ::base::ClampMin<size_t>(range.EndCaretPosition, _inputBuffer.length());
const auto length = ::base::ClampSub<size_t>(textEnd, range.StartCaretPosition);
const auto textRequested = _inputBuffer.substr(range.StartCaretPosition, length);
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
args.Request().Text(textRequested);
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
}
CATCH_LOG();
}
// Method Description:
// - Handler for SelectionRequested event by CoreEditContext responsible
// for returning the currently selected text.
// TSFInputControl currently doesn't allow selection, so nothing happens.
// NOTE: Documentation says application should handle this event
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - args: CoreTextSelectionRequestedEventArgs for providing data for the SelectionRequested event. Not used in method.
// Return Value:
// - <none>
void TSFInputControl::_selectionRequestedHandler(CoreTextEditContext sender, CoreTextSelectionRequestedEventArgs const& /*args*/)
{
}
// Method Description:
// - Handler for SelectionUpdating event by CoreEditContext responsible
// for handling modifications to the range of text currently selected.
// TSFInputControl doesn't currently allow selection, so nothing happens.
// NOTE: Documentation says application should set its selection range accordingly
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - args: CoreTextSelectionUpdatingEventArgs for providing data for the SelectionUpdating event. Not used in method.
// Return Value:
// - <none>
void TSFInputControl::_selectionUpdatingHandler(CoreTextEditContext sender, CoreTextSelectionUpdatingEventArgs const& /*args*/)
{
}
// Method Description:
// - Handler for TextUpdating event by CoreEditContext responsible
// for handling text updates.
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - args: CoreTextTextUpdatingEventArgs contains new text to update buffer with.
// Return Value:
// - <none>
void TSFInputControl::_textUpdatingHandler(CoreTextEditContext sender, CoreTextTextUpdatingEventArgs const& args)
{
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
const auto incomingText = args.Text();
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
const auto range = args.Range();
try
{
2020-03-24 17:00:18 -07:00
// When a user deletes the last character in their current composition, some machines
// will fire a CompositionCompleted before firing a TextUpdating event that deletes the last character.
// The TextUpdating will have a lower StartCaretPosition, so in this scenario, _activeTextStart
// needs to update to be the StartCaretPosition.
// A known issue related to this behavior is that the last character that's deleted from a composition
// will get sent to the terminal before we receive the TextUpdate to delete the character.
// See GH #5054.
_activeTextStart = ::base::ClampMin(_activeTextStart, ::base::ClampedNumeric<size_t>(range.StartCaretPosition));
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
_inputBuffer = _inputBuffer.replace(
range.StartCaretPosition,
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
::base::ClampSub<size_t>(range.EndCaretPosition, range.StartCaretPosition),
incomingText);
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Emojis/Kaomojis/Symbols chosen through the IME without starting composition
// will be sent straight through to the terminal.
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
if (!_inComposition)
{
_SendAndClearText();
}
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
else
{
Canvas().Visibility(Visibility::Visible);
const auto text = _inputBuffer.substr(_activeTextStart);
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
TextBlock().Text(text);
}
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Notify the TSF that the update succeeded
args.Result(CoreTextTextUpdatingResult::Succeeded);
}
catch (...)
{
LOG_CAUGHT_EXCEPTION();
// indicate updating failed.
args.Result(CoreTextTextUpdatingResult::Failed);
}
}
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
// Method Description:
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
// - Send the portion of the textBuffer starting at _activeTextStart to the end of the buffer.
// Then clear the TextBlock and hide it until the next time text is received.
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
// Arguments:
// - <none>
// Return Value:
// - <none>
void TSFInputControl::_SendAndClearText()
{
const auto text = _inputBuffer.substr(_activeTextStart);
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
_compositionCompletedHandlers(text);
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
Make Korean IME input more consistent (#4796) ## Summary of the Pull Request Korean IME was not working correctly due to way we were clearing the input buffer inside of `TSFInputControl`. We wanted to clear our input buffer and tell TSF to clear its input buffer as well when we receive a `CompositionCompleted` event. This works fine in some IME languages such as Chinese and Japanese. However, Korean IME composes characters differently in such a way where we can't tell TSF to clear their buffer during a `CompositionCompleted` event because it would clear the character that triggered the `CompositionCompleted` event in the first place. The solution in this PR is to keep our `_inputBuffer` intact until the user presses <kbd>Enter</kbd> or <kbd>Esc</kbd>, in which case we clear our buffer and the TSF buffer. I've chosen these two keys because it seems to make sense to clear the buffer after text is sent to the terminal with <kbd>Enter</kbd>, and <kbd>Esc</kbd> usually means to cancel a current composition anyway. This means we need to keep track of our last known "Composition Start Point", which is represented by `_activeTextStart`. Whenever we complete a composition, we'll send the portion of the input buffer between `_activeTextStart` and the end of the input buffer to the terminal. Then, we'll update `_activeTextStart` to be the end of the input buffer so that the next time we send text to the terminal, we'll only send the portion of our buffer that's "active". ## PR Checklist * [x] Closes #4226 * [x] CLA signed * [x] Tests added/passed ## Validation Steps Performed Manual testing with Chinese, Japanese, and Korean IME.
2020-03-04 20:01:01 +00:00
_activeTextStart = _inputBuffer.length();
TextBlock().Text(L"");
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
// hide the controls until text input starts again
Canvas().Visibility(Visibility::Collapsed);
Display Emojis, Kaomojis, and symbols while in IME mode (#4688) ## Summary of the Pull Request Currently, while in IME mode, selections with the Emoji/Kaomoji/Symbol Picker (which is brought up with <kbd>win+.</kbd>) are not displayed until the user starts a new composition. This is due to the fact that we hide the TextBlock when we receive a CompositionCompleted event, and we only show the TextBlock when we receive a CompositionStarted event. Input from the picker does not count as a composition, so we were never showing the text box, even if the symbols were thrown into the inputBuffer. In addition, we weren't receiving CompositionStarted events when we expected to. We should be showing the TextBlock when we receive _any_ text, so we should make the TextBlock visible inside of `TextUpdatingHandler`. Furthermore, some really helpful discussion in #3745 around wrapping the `NotifyTextChanged` call with a `NotifyFocusLeave` and a `NotifyFocusEnter` allowed the control to much more consistently determine when a CompositionStarted and a CompositionEnded. I've also went around and replaced casts with saturating casts, and have removed the line that sets the `textBlock.Width()` so that it would automatically set its width. This resolves the issue where while composing a sentence, the textBlock would be too small to contain all the text, so it would be cut off, but the composition is still valid and still able to continue. ## PR Checklist * [x] Closes #4148 * [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [x] Tests added/passed ## Validation Steps Performed Tested picking emojis, kaomojis, and symbols with numerous different languages.
2020-02-26 10:36:02 -08:00
}
Add a Windows.UI.Text.Core IME overlay to TerminalControl (#1919) TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs (Windows.UI.Text.Core) namespace to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a CoreTextEditContext object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample[1]. TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo) handled IME. # Details TSFInputControl is a Windows.UI.Xaml.Controls.UserControl TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within its containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here[2]. ## Text Updates Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). ## Completed Text Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. ## Positioning Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. ### Calculating Screen Coordinates 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which lets the IME know where to position itself on the Screen. ## Font Information/Cursor/Writing to Terminal 3 events were added to the TSFInputControl to create a loosely-coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. ## Known Issues - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console (#3640) - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better (#3657) ## Future Considerations Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. ## Validation General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear [1]: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl [2]: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Closes #459 Closes #2213 Closes #3641
2019-11-21 16:25:50 -08:00
// Method Description:
// - Handler for FormatUpdating event by CoreEditContext responsible
// for handling different format updates for a particular range of text.
// TSFInputControl doesn't do anything with this event.
// Arguments:
// - sender: CoreTextEditContext sending the request. Not used in method.
// - args: CoreTextFormatUpdatingEventArgs Provides data for the FormatUpdating event. Not used in method.
// Return Value:
// - <none>
void TSFInputControl::_formatUpdatingHandler(CoreTextEditContext sender, CoreTextFormatUpdatingEventArgs const& /*args*/)
{
}
DEFINE_EVENT(TSFInputControl, CompositionCompleted, _compositionCompletedHandlers, TerminalControl::CompositionCompletedEventArgs);
}