starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-03 21:25:34 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

[PR #1919] [MERGED] #459 IME UI does not follow the cursor in Windows Terminal #24708

New Issue
Open
opened 2026-01-31 09:04:53 +00:00 by claunia · 0 comments
claunia commented 2026-01-31 09:04:53 +00:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/microsoft/terminal/pull/1919
Author: @philnach
Created: 7/11/2019
Status: ✅ Merged
Merged: 11/22/2019
Merged by: @DHowett-MSFT

Base: master ← Head: dev/philnach/459-TSF30-IME_2

📝 Commits (10+)

  • b57310d Add TSF3.0 IME UserControl
  • 17b624d Trying to figure out why Surface Pro 6 isn't working right
  • 1a69d5c Add documentation, getting ready for review
  • 77c6197 Minor code formatting updates
  • 466fd96 More changes to get ready for draft PR
  • 078c69f Add Exception handling around std::wstring operations
  • 3a097ca Add TSF3.0 IME UserControl
  • 578578c Trying to figure out why Surface Pro 6 isn't working right
  • 882e97f Add documentation, getting ready for review
  • 28e99d0 Minor code formatting updates

📊 Changes

9 files changed (+583 additions, -16 deletions)

View changed files

➕ src/cascadia/TerminalControl/TSFInputControl.cpp (+349 -0)
➕ src/cascadia/TerminalControl/TSFInputControl.h (+73 -0)
➕ src/cascadia/TerminalControl/TSFInputControl.idl (+31 -0)
📝 src/cascadia/TerminalControl/TermControl.cpp (+69 -1)
📝 src/cascadia/TerminalControl/TermControl.h (+8 -1)
📝 src/cascadia/TerminalControl/TerminalControl.vcxproj (+8 -1)
📝 src/cascadia/TerminalControl/pch.h (+7 -3)
📝 src/cascadia/WinRTUtils/WinRTUtils.vcxproj (+2 -10)
➕ src/cascadia/WinRTUtils/inc/Utils.h (+36 -0)

📄 Description

Summary of the Pull Request

TerminalControl doesn't use any of the built in text input and edit controls provided by XAML
for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF)
in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get
to take advantage of newer APIs (Windows.UI.Text.Core namespace) to provide support vs. the old TSF 1.0.

Windows.UI.Text.Core handles communication between a text edit control and the text services primarily
through a CoretextEditContext object.

This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the CustomEditControl sample

TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo).

References

PR Checklist

  • Closes #459, Closes #3641, Closes #2213.
  • CLA signed. If not, go over here and sign the CLA
  • Tests added/passed
  • Requires documentation to be updated
  • I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Detailed Description of the Pull Request / Additional comments

TSFInputControl is a Windows.UI.Xaml.Controls.UserControl

TSFInputControl contains a Canvas control for absolution positioning a TextBlock control within it's containing control (TerminalControl).

The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text.

TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME.

A good write-up on how to interact with CoreTextEditContext can be found here: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input

Text Updates:
Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer).

Completed Text:
Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again.

Positioning:
Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder.

How screen coordinate is calculated is by:

  1. Obtaining the Window position in Screen coordinates.
  2. Determining the Client coordinate of the cursor.
  3. Converting the Client coordinate of the cursor to Screen coordinates.
  4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..).
  5. Applying any scale factor of the display.

Once we have the right position in screen coordinates, this is supplied in the LayoutBounds of the CoreTextLayoutRequestedEventArgs which let's the IME know where to position itself on the Screen.

Font Information/Cursor/Writing to Terminal:
3 events were added to the TSFInputControl to create a loosely coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer.

Known issues with this change:

  • Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console.
  • Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text)
  • Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better.

Future considerations:
Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types.

Validation Steps Performed

General steps to try functionality

  • Open Console
  • Switch to Simplified Chinese (Shortcut: Windows+Spacebar)
  • Switch to Chinese mode on language bar

Scenarios validated:

  • As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters.
  • User can dismiss IME and text doesn't appear on command line
  • Switch back to English mode, functions like normal
  • New tab has proper behavior
  • Switching between tabs has proper behavior
  • Switching away from Terminal Window with IME present causes IME to disappear

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/microsoft/terminal/pull/1919 **Author:** [@philnach](https://github.com/philnach) **Created:** 7/11/2019 **Status:** ✅ Merged **Merged:** 11/22/2019 **Merged by:** [@DHowett-MSFT](https://github.com/DHowett-MSFT) **Base:** `master` ← **Head:** `dev/philnach/459-TSF30-IME_2` --- ### 📝 Commits (10+) - [`b57310d`](https://github.com/microsoft/terminal/commit/b57310de654cbc2ea21ce20425fd1211801c71be) Add TSF3.0 IME UserControl - [`17b624d`](https://github.com/microsoft/terminal/commit/17b624d575cbe62b7b22798a3824ee4fc936e7b8) Trying to figure out why Surface Pro 6 isn't working right - [`1a69d5c`](https://github.com/microsoft/terminal/commit/1a69d5ca176c32cf2881dcd94ba3d17d4af2c86b) Add documentation, getting ready for review - [`77c6197`](https://github.com/microsoft/terminal/commit/77c619752633ee917578245ecec557d495552297) Minor code formatting updates - [`466fd96`](https://github.com/microsoft/terminal/commit/466fd967b79296b614381c1778ee4becf392ca52) More changes to get ready for draft PR - [`078c69f`](https://github.com/microsoft/terminal/commit/078c69f16fbcc875120e1fc49a1ffc7d2d4da537) Add Exception handling around std::wstring operations - [`3a097ca`](https://github.com/microsoft/terminal/commit/3a097ca8ed113aea3632d2b92e53c5467f1a4943) Add TSF3.0 IME UserControl - [`578578c`](https://github.com/microsoft/terminal/commit/578578cb697799a7a91e8e74beb0e8137e7a9ba8) Trying to figure out why Surface Pro 6 isn't working right - [`882e97f`](https://github.com/microsoft/terminal/commit/882e97f886b78abe9c5f9f22d5f673cfc338887b) Add documentation, getting ready for review - [`28e99d0`](https://github.com/microsoft/terminal/commit/28e99d0b57c7be04185f05d457f188116561f9ad) Minor code formatting updates ### 📊 Changes **9 files changed** (+583 additions, -16 deletions) <details> <summary>View changed files</summary> ➕ `src/cascadia/TerminalControl/TSFInputControl.cpp` (+349 -0) ➕ `src/cascadia/TerminalControl/TSFInputControl.h` (+73 -0) ➕ `src/cascadia/TerminalControl/TSFInputControl.idl` (+31 -0) 📝 `src/cascadia/TerminalControl/TermControl.cpp` (+69 -1) 📝 `src/cascadia/TerminalControl/TermControl.h` (+8 -1) 📝 `src/cascadia/TerminalControl/TerminalControl.vcxproj` (+8 -1) 📝 `src/cascadia/TerminalControl/pch.h` (+7 -3) 📝 `src/cascadia/WinRTUtils/WinRTUtils.vcxproj` (+2 -10) ➕ `src/cascadia/WinRTUtils/inc/Utils.h` (+36 -0) </details> ### 📄 Description <!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? --> ## Summary of the Pull Request TerminalControl doesn't use any of the built in text input and edit controls provided by XAML for text input, which means TermianlControl needs to communicate with the Text Services Framework (TSF) in order to provide Input Method Editor (IME) support. Just like the rest of Terminal we get to take advantage of newer APIs ([Windows.UI.Text.Core namespace](https://docs.microsoft.com/en-us/uwp/api/windows.ui.text.core)) to provide support vs. the old TSF 1.0. Windows.UI.Text.Core handles communication between a text edit control and the text services primarily through a [CoretextEditContext](https://docs.microsoft.com/en-us/uwp/api/Windows.UI.Text.Core.CoreTextEditContext) object. This change introduces a new UserControl TSFInputControl which is a custom EditControl similar to the [CustomEditControl sample](https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CustomEditControl) TSFInputControl is similar (overlay with IME text) to how old console (conimeinfo). <!-- Other than the issue solved, is this relevant to any other issues/existing PRs? --> ## References <!-- Please review the items on the PR checklist before submitting--> ## PR Checklist * [X] Closes #459, Closes #3641, Closes #2213. * [X] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA * [ ] Tests added/passed * [ ] Requires documentation to be updated * [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx <!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here --> ## Detailed Description of the Pull Request / Additional comments TSFInputControl is a [Windows.UI.Xaml.Controls.UserControl](https://docs.microsoft.com/en-us/uwp/api/Windows.UI.Xaml.Controls.UserControl) TSFInputControl contains a [Canvas](https://docs.microsoft.com/en-us/uwp/api/Windows.UI.Xaml.Controls.Canvas) control for absolution positioning a [TextBlock](https://docs.microsoft.com/en-us/uwp/api/windows.ui.xaml.controls.textblock) control within it's containing control (TerminalControl). The TextBlock control is used for displaying candidate text from the IME. When the user makes a choice in the IME the TextBlock is cleared and the text is written to the Terminal buffer like normal text. TSFInputControl creates an instance of the CoreTextEditContext and attaches appropriate event handlers to CoreTextEditContext in order to interact with the IME. A good write-up on how to interact with CoreTextEditContext can be found here: https://docs.microsoft.com/en-us/windows/uwp/design/input/custom-text-input Text Updates: Text updates from the IME come in on the TextUpdating event handler, text updates are stored in an internal buffer (_inputBuffer). Completed Text: Once a user selects a text in the IME, the CompositionCompleted handler is invoked. The input buffer (_inputBuffer) is written to the Terminal buffer, _inputBuffer is cleared and Canvas and TextBlock controls are hidden until the user starts a composition session again. Positioning: Telling the IME where to properly position itself was the hardest part of this change. The IME expects to know it's location in screen coordinates as supposed to client coordinates. This is pretty easy if you are a pure UWP, but since we are hosted inside a XAMLIsland the client to screen coordinate translation is a little harder. How screen coordinate is calculated is by: 1. Obtaining the Window position in Screen coordinates. 2. Determining the Client coordinate of the cursor. 3. Converting the Client coordinate of the cursor to Screen coordinates. 4. Offsetting the X and Y coordinate of the cursor by the position of the TerminalControl within the window (tabs if present, margins, etc..). 5. Applying any scale factor of the display. Once we have the right position in screen coordinates, this is supplied in the [LayoutBounds](https://docs.microsoft.com/en-us/uwp/api/windows.ui.text.core.coretextlayoutbounds) of the [CoreTextLayoutRequestedEventArgs](https://docs.microsoft.com/en-us/uwp/api/windows.ui.text.core.coretextlayoutrequestedeventargs) which let's the IME know where to position itself on the Screen. Font Information/Cursor/Writing to Terminal: 3 events were added to the TSFInputControl to create a loosely coupled implementation between the TerminalControl and the TSFInputControl. These events are used for obtaining Font information from the TerminalControl, getting the Cursor position and writing to the terminal buffer. Known issues with this change: - Width of TextBlock is hardcoded to 200 pixels and most likely should adjust to the available width of the current input line on the console. - Entering text in the middle of an existing set of text has TextBlock render under existing text. Current Console behavior here isn't good experience either (writes over text) - Text input at edges of window is clipped versus wrapping around to next line. This isn't any worse than the original command line, but Terminal should be better. Future considerations: Ideally, we'd be able to interact with the console buffer directly and replace characters as the user types. <!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well --> ## Validation Steps Performed General steps to try functionality - Open Console - Switch to Simplified Chinese (Shortcut: Windows+Spacebar) - Switch to Chinese mode on language bar Scenarios validated: - As user types unformatted candidates appear on command line and IME renders in correct position under unformatted characters. - User can dismiss IME and text doesn't appear on command line - Switch back to English mode, functions like normal - New tab has proper behavior - Switching between tabs has proper behavior - Switching away from Terminal Window with IME present causes IME to disappear --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-31 09:04:53 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#24708
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?