Korean IME does not work as expected #5920

@rkttu commented on GitHub (Jan 15, 2020):

@zadjii-msft Currently it only happens in WT, but legacy console's behavior also weird. It does not display character compositions. For example, to make a character '한', user can typing the key 'ㅎ', 'ㅏ', and 'ㄴ'. Usually, this sequence displayed like 'ㅎ' -> '하' -> '한', but currently it displays just a complete character directly (ex. '' -> '한').

FYI, I'm using Dubeolsik (두벌식) Korean IME.

@rkttu commented on GitHub (Jan 15, 2020): @zadjii-msft Currently it only happens in WT, but legacy console's behavior also weird. It does not display character compositions. For example, to make a character '한', user can typing the key 'ㅎ', 'ㅏ', and 'ㄴ'. Usually, this sequence displayed like 'ㅎ' -> '하' -> '한', but currently it displays just a complete character directly (ex. '' -> '한'). FYI, I'm using Dubeolsik (두벌식) Korean IME.

claunia commented

@cupsos commented on GitHub (Jan 16, 2020):

@zadjii-msft Same issue here. Terminal with WSL(Debian), pwsh, powershell, cmd not work expectly. //'한그ㄱ글'

Legacy WSL(Debian), pwsh, powershell, cmd can type properly, //'한글'
but not showing character compositions

@cupsos commented on GitHub (Jan 16, 2020): @zadjii-msft Same issue here. Terminal with WSL(Debian), pwsh, powershell, cmd not work expectly. //'한그ㄱ글' Legacy WSL(Debian), pwsh, powershell, cmd can type properly, //'한글' but not showing character compositions

claunia commented

@zadjii-msft commented on GitHub (Jan 21, 2020):

This is another great repro gif from @juhokang in #4311

Windows build number: Microsoft Windows [Version 10.0.18362.592]
Windows Terminal version (if applicable): Windows Terminal (Preview) Version: 0.8.10091.0
ex) Type 안녕하세요 with the keyboard

Expected behavior

show 안녕하세요 on the terminal

Actual behavior

shows 안녀ㄴ녕하ㅎ하셍세세요요

@zadjii-msft commented on GitHub (Jan 21, 2020): This is another great repro gif from @juhokang in #4311 > ``` > Windows build number: Microsoft Windows [Version 10.0.18362.592] > Windows Terminal version (if applicable): Windows Terminal (Preview) Version: 0.8.10091.0 > ``` > > ex) Type 안녕하세요 with the keyboard > > ![koreannotnormal](https://user-images.githubusercontent.com/4745294/72769037-a7414b80-3c3c-11ea-9fdb-a5738c724175.gif) > # Expected behavior > > show 안녕하세요 on the terminal > # Actual behavior > > shows 안녀ㄴ녕하ㅎ하셍세세요요

claunia commented

@ioklo commented on GitHub (Jan 29, 2020):

FYI, expected behavior during composition.

@ioklo commented on GitHub (Jan 29, 2020): FYI, expected behavior during composition. ![ime](https://user-images.githubusercontent.com/3759882/73317904-1a6d4200-427b-11ea-924c-9e6580e08559.gif)

claunia commented

@hatsunearu commented on GitHub (Feb 2, 2020):

https://github.com/microsoft/vscode/issues/89853 Possibly similar issue happening in VS Code

@hatsunearu commented on GitHub (Feb 2, 2020): https://github.com/microsoft/vscode/issues/89853 Possibly similar issue happening in VS Code

claunia commented

@rkttu commented on GitHub (Feb 2, 2020):

@zadjii-msft Is there any progress on this issue? It seems like quite a lot of people encountered this problem.

@rkttu commented on GitHub (Feb 2, 2020): @zadjii-msft Is there any progress on this issue? It seems like quite a lot of people encountered this problem.

claunia commented

@zadjii-msft commented on GitHub (Feb 3, 2020):

@rkttu Nope, when there is progress, someone will make sure to chime in on this thread. It's been triaged as a P1 bug for 1.0, so we won't be shipping 1.0 without a fix for this, so stay tuned. If anyone is particularly passionate about this bug, we'd be happy to review a PR. Until someone's been assigned to this bug, you can be sure you won't be stepping on our toes ☺️

@zadjii-msft commented on GitHub (Feb 3, 2020): @rkttu Nope, when there _is_ progress, someone will make sure to chime in on this thread. It's been triaged as a P1 bug for 1.0, so we won't be shipping 1.0 without a fix for this, so stay tuned. If anyone is particularly passionate about this bug, we'd be happy to review a PR. Until someone's been _assigned_ to this bug, you can be sure you won't be stepping on our toes ☺️

claunia commented

https://github.com/microsoft/vscode/issues/89853#issuecomment-581495374

@hatsunearu commented on GitHub (Feb 3, 2020):

Possible workaround (not tested): try removing Droid Sans Fallback from the fonts list of the app if it's there.

@hatsunearu commented on GitHub (Feb 3, 2020): Possible workaround (not tested): try removing Droid Sans Fallback from the fonts list of the app if it's there. https://github.com/microsoft/vscode/issues/89853#issuecomment-581495374

claunia commented

@simnalamburt commented on GitHub (Feb 19, 2020):

Note that this problem did not occur in version 0.7.3451.0.

@simnalamburt commented on GitHub (Feb 19, 2020): Note that this problem did not occur in version 0.7.3451.0.

claunia commented

@simnalamburt commented on GitHub (Feb 26, 2020):

Possible workaround (not tested): try removing Droid Sans Fallback from the fonts list of the app if it's there.

microsoft/vscode#89853 (comment)

@hatsunearu Unfortunately https://github.com/microsoft/vscode/issues/89853 is caused by wrong glyph rendering and that workaround dosn't work here.

@simnalamburt commented on GitHub (Feb 26, 2020): > > > Possible workaround (not tested): try removing Droid Sans Fallback from the fonts list of the app if it's there. > > [microsoft/vscode#89853 (comment)](https://github.com/microsoft/vscode/issues/89853#issuecomment-581495374) @hatsunearu Unfortunately https://github.com/microsoft/vscode/issues/89853 is caused by wrong glyph rendering and that workaround dosn't work here.

claunia commented

@simnalamburt commented on GitHub (Feb 26, 2020):

@guswns0528 discovered that this bug is first caused by https://github.com/microsoft/terminal/commit/dfa7b4a1. https://github.com/microsoft/terminal/commit/dfa7b4a1 itself is right commit so we can't simply revert it.

Looks like this issue is caused by strange behavior of Core text APIs. Originally, the Composition complete event should only be fired when the letter composition is completed. Like this:

If you are typing ㅎ ㅏ ㄴ ㄱ ㅡ ㄹ, composition event should be fired twice.
한, and 글

But in here it's fired before that. Like this:

If you type ㅎ ㅏ ㄴ ㄱ ㅡ ㄹ in here, composition event is fired three times.
한, 그, and 글

It might be a bug of Core text APIs but core text API is just a wrapper of Text Services Framework, which is super stable framework inheritted from Windows XP era. So I need to investigate further.

I'm currently debugging this issue and I'll comment on here if I find something. Please feel free to share any information if you find something. Thanks

@simnalamburt commented on GitHub (Feb 26, 2020): @guswns0528 discovered that this bug is first caused by https://github.com/microsoft/terminal/commit/dfa7b4a1. https://github.com/microsoft/terminal/commit/dfa7b4a1 itself is *right* commit so we can't simply revert it. Looks like this issue is caused by strange behavior of Core text APIs. Originally, the Composition complete event should only be fired when the letter composition is completed. Like this: - If you are typing <kbd>ㅎ</kbd> <kbd>ㅏ</kbd> <kbd>ㄴ</kbd> <kbd>ㄱ</kbd> <kbd>ㅡ</kbd> <kbd>ㄹ</kbd>, composition event should be fired twice.\ `한`, and `글` But in here it's fired before that. Like this: - If you type <kbd>ㅎ</kbd> <kbd>ㅏ</kbd> <kbd>ㄴ</kbd> <kbd>ㄱ</kbd> <kbd>ㅡ</kbd> <kbd>ㄹ</kbd> in here, composition event is fired three times.\ `한`, `그`, and `글` It might be a bug of Core text APIs but core text API is just a wrapper of Text Services Framework, which is super stable framework inheritted from Windows XP era. So I need to investigate further. I'm currently debugging this issue and I'll comment on here if I find something. Please feel free to share any information if you find something. Thanks

claunia commented

@DHowett-MSFT commented on GitHub (Feb 26, 2020):

@simnalamburt thanks! Note that @leonMSFT is currently working in this area and has a couple pull requests out for this; perhaps you two can coordinate?

@DHowett-MSFT commented on GitHub (Feb 26, 2020): @simnalamburt thanks! Note that @leonMSFT is currently working in this area and has a couple pull requests out for this; perhaps you two can coordinate?

claunia commented

@simnalamburt commented on GitHub (Feb 26, 2020):

Cool! Currently I am suspecting wrong parameter of NotifyTextChanged function as the cause of the bug. But in fact, since I first saw the Core text API today, I still don't know what's wrong. Any help or information is always welcomed!

@simnalamburt commented on GitHub (Feb 26, 2020): Cool! Currently I am suspecting wrong parameter of NotifyTextChanged function as the cause of the bug. But in fact, since I first saw the Core text API today, I still don't know what's wrong. Any help or information is always welcomed!

claunia commented

@leonMSFT commented on GitHub (Feb 26, 2020):

@simnalamburt thanks a lot for the investigation! I tried to see if I can repro the two vs three composition completed events issue, but I only see two composition completed events. Maybe you could provide a screenshot of the debugging logs you used to find this? Update: I just found this out, but you're likely seeing multiple composition completed events after the first character is finished because when we call NotifyTextChanged to reset the text server's buffer, it'll fire another composition completed event. However, since we have the if-statement to check if there's anything in _inputBuffer, we don't do anything on our side.

~~I was also taking a look at this bug earlier, and perhaps my findings could explain why you're seeing the bug you're seeing.~~ Here's the behavior I'm observing and why I think we're messing up Korean IME input:

So, going through your example keysequence, pressing ㅎㅏㄴ, would result in three TextUpdated events being received, with the _inputBuffer and the _textBlock having the character 한.

Now the user presses ㄱ, and what will happen is the following:

TextUpdated is received with the character 한.
CompositionCompleted is received, signaling that 한 is the finished composition.
As part of our CompositionCompletedHandler, we send the contents of _inputBuffer (which is 한) to the terminal and reset the _inputBuffer and _textBlock. Then we also notify the text server that they should also make their "buffer" empty as well. (This is the NotifyTextChanged call that you mentioned).

What should happen now is that we should receive a TextUpdated event with the text as 한ㄱ.
However, we're actually not getting any TextUpdated events after our CompositionCompletedHandler is finished because we're telling the text server to reset their text buffer. Since their buffer is empty, it won't tell us that we should update our text to be anything.

This is why you'll run into the weird issue where you'll be pressing ㅎㅏㄴ, which works fine, and you'll see 한 on the screen, but once you make another input, like ㄱ, nothing happens. The ㄱ keypress triggers the CompositionCompleted event for the previous character 한, which tells the text server to clear their buffer. So, you will need to press ㄱ again to make ㄱ show up on the screen.

So, the core of the problem is that we need to send the IME input to the terminal when we believe composition is finished, and we naturally also need to clear our buffer whenever we send some input to the terminal. We also need to keep the text server's buffer and our _inputBuffer in sync, so whenever we clear our buffer, we tell the text server to clear theirs as well.

As a small test, I've tried commenting out the code where we're telling the text server to reset their text buffer, and lo and behold, text comes out as you would expect, without having to double-press any characters. The only problem here is that if we don't reset our text buffer, every CompositionCompleted event will cause us to send the whole _inputBuffer (which included literally everything you've ever typed while in IME mode) to the terminal, resulting in lots of duplicate input.

I'm currently trying to think of a way around this, but I'm giving you a summary of my findings so maybe you can also repro and investigate further to see if I've missed something! 😄

@leonMSFT commented on GitHub (Feb 26, 2020): @simnalamburt thanks a lot for the investigation! ~~I tried to see if I can repro the two vs three composition completed events issue, but I only see two composition completed events. Maybe you could provide a screenshot of the debugging logs you used to find this?~~ Update: I just found this out, but you're likely seeing multiple composition completed events after the first character is finished because when we call `NotifyTextChanged` to reset the text server's buffer, it'll fire _another_ composition completed event. However, since we have the if-statement to check if there's anything in `_inputBuffer`, we don't do anything on our side. ~~I was also taking a look at this bug earlier, and perhaps my findings could explain why you're seeing the bug you're seeing.~~ Here's the behavior I'm observing and why I _think_ we're messing up Korean IME input: So, going through your example keysequence, pressing <kbd>ㅎ</kbd><kbd>ㅏ</kbd><kbd>ㄴ</kbd>, would result in three `TextUpdated` events being received, with the `_inputBuffer` and the `_textBlock` having the character `한`. Now the user presses <kbd>ㄱ</kbd>, and what will happen is the following: 1. TextUpdated is received with the character `한`. 2. CompositionCompleted is received, signaling that `한` is the finished composition. 3. As part of our CompositionCompletedHandler, we send the contents of `_inputBuffer` (which is 한) to the terminal and reset the `_inputBuffer` and `_textBlock`. Then we also notify the text server that they should also make their "buffer" empty as well. (This is the `NotifyTextChanged` call that you mentioned). What should happen now is that we should receive a `TextUpdated` event with the text as `한ㄱ`. However, we're actually not getting any `TextUpdated` events after our CompositionCompletedHandler is finished _because_ we're telling the text server to reset their text buffer. Since their buffer is empty, it won't tell us that we should update our text to be anything. This is why you'll run into the weird issue where you'll be pressing <kbd>ㅎ</kbd><kbd>ㅏ</kbd><kbd>ㄴ</kbd>, which works fine, and you'll see `한` on the screen, but once you make another input, like <kbd>ㄱ</kbd>, nothing happens. The <kbd>ㄱ</kbd> keypress triggers the CompositionCompleted event for the previous character 한, which tells the text server to clear their buffer. So, you will need to press <kbd>ㄱ</kbd> again to make ㄱ show up on the screen. So, the core of the problem is that we need to send the IME input to the terminal when we believe composition is finished, and we naturally also need to clear our buffer whenever we send some input to the terminal. We also need to keep the text server's buffer and our `_inputBuffer` in sync, so whenever we clear our buffer, we tell the text server to clear theirs as well. As a small test, I've tried commenting out the code where we're telling the text server to reset their text buffer, and lo and behold, text comes out as you would expect, without having to double-press any characters. The only problem here is that if we don't reset our text buffer, every CompositionCompleted event will cause us to send the whole `_inputBuffer` (which included literally everything you've ever typed while in IME mode) to the terminal, resulting in lots of duplicate input. I'm currently trying to think of a way around this, but I'm giving you a summary of my findings so maybe you can also repro and investigate further to see if I've missed something! 😄

claunia commented

@simnalamburt commented on GitHub (Feb 28, 2020):

@leonMSFT Wow thanks for your detailed explanation! Now I understand what was going on in my development environment.

Currently I’m trying to leave some unfinished characters in text buffer instead of totally clearing it in CompositionCompletedHandler.

Please share any information or updates and let me help anything I can! Actually there are lots of people waiting for this issue to be resolved since there are not much options that Korean developer can choose in Windows. Any share will be helpful and the whole Korean developer community will be grateful to you! 😄

@simnalamburt commented on GitHub (Feb 28, 2020): @leonMSFT Wow thanks for your detailed explanation! Now I understand what was going on in my development environment. Currently I’m trying to leave some unfinished characters in text buffer instead of totally clearing it in CompositionCompletedHandler. Please share any information or updates and let me help anything I can! Actually there are lots of people waiting for this issue to be resolved since there are not much options that Korean developer can choose in Windows. Any share will be helpful and the whole Korean developer community will be grateful to you! 😄

claunia commented

@leonMSFT commented on GitHub (Feb 29, 2020):

@simnalamburt Yup, not clearing the whole text buffer, but leaving unfinished characters in is key! Luckily, I think I'm close to getting the fix for this out! 🎉I'm specifically testing out trying to type out this sequence: 안녕하세요, which was provided earlier in this thread.

It seems to work as expected, but before I have a PR out for this fix, I'll need to make sure I haven't messed up any other IME input modes, so hang tight! 😃

One thing I would like your help on is letting me know of other sample character sequences that might possibly break the way I'm handling Korean IME! I don't know Korean at all, so having the sequence laid out in english characters like it was above with "dkssudgktpdy" (which comes out as 안녕하세요) really helped!

@leonMSFT commented on GitHub (Feb 29, 2020): @simnalamburt Yup, not clearing the whole text buffer, but leaving unfinished characters in is key! Luckily, I think I'm close to getting the fix for this out! 🎉I'm specifically testing out trying to type out this sequence: 안녕하세요, which was provided earlier in this thread. ![koreanoutput](https://user-images.githubusercontent.com/57155886/75596418-67dcfa80-5a45-11ea-9530-064e1cb53ba0.gif) It _seems_ to work as expected, but before I have a PR out for this fix, I'll need to make sure I haven't messed up any other IME input modes, so hang tight! 😃 One thing I would like your help on is letting me know of other sample character sequences that might possibly break the way I'm handling Korean IME! I don't know Korean at all, so having the sequence laid out in english characters like it was above with "dkssudgktpdy" (which comes out as 안녕하세요) really helped!

claunia commented

@simnalamburt commented on GitHub (Feb 29, 2020):

Wow it was very fast! I lost my chance to become hero lol

Actually there are not many corner cases in Korean IME. And your sample
video (안녕하세요) showed that it perfectly handles one of very famous corner
cases in Korean IME called “도깨비불 현상”.

If 안녕하세요 works perfectly I expect the other cases to work fine, but I’ll
share you few more samples just in case.

gksrnrdj whgdk (한국어 좋아)
To test whether aborting composition with space works fine

gksrnrdjEnterwhgdk (한국어\n좋아)
To test whether aborting composition with enter works fine

Actually there are bunch of things to test further like

Test if alternative Korean IME like 세벌식 works fine
Test if swiching IME in the middle of composition works fine (it’s very
common for Korean people)
etc

But these cases might be complicated to ask you to test so just share your
patch or make the draft PR. I have bunch of Korean and Japanese friend
developers interested in this issue and they will battle test it for you!

2020년 2월 29일 (토) 09:27, Leon Liang notifications@github.com님이 작성:

@simnalamburt commented on GitHub (Feb 29, 2020): Wow it was very fast! I lost my chance to become hero lol Actually there are not many corner cases in Korean IME. And your sample video (안녕하세요) showed that it perfectly handles one of very famous corner cases in Korean IME called “도깨비불 현상”. If 안녕하세요 works perfectly I expect the other cases to work fine, but I’ll share you few more samples just in case. gksrnrdj whgdk (한국어 좋아) To test whether aborting composition with space works fine gksrnrdj<kbd>Enter</kbd>whgdk (`한국어\n좋아`) To test whether aborting composition with enter works fine Actually there are bunch of things to test further like - Test if alternative Korean IME like 세벌식 works fine - Test if swiching IME in the middle of composition works fine (it’s very common for Korean people) - etc But these cases might be complicated to ask you to test so just share your patch or make the draft PR. I have bunch of Korean and Japanese friend developers interested in this issue and they will battle test it for you! 2020년 2월 29일 (토) 09:27, Leon Liang <notifications@github.com>님이 작성:

claunia commented

@simnalamburt commented on GitHub (Mar 2, 2020):

I tried to reproduce the scenario that you described, but I'm having trouble. I typed ㅎㅏㄴㄱ and I got 2 textUpdate event instead of 0 after typing "한".

# Typed ㅎ
_compositionStartedHandler() called.
_textUpdatingHandler() called.
Text:  ㅎ
Range: [0, 0]

# Typed ㅏ
_textUpdatingHandler() called.
Text:  하
Range: [0, 1]

# Typed ㄴ
_textUpdatingHandler() called.
Text:  한
Range: [0, 1]

# Typed ㄱ
_compositionCompletedHandler() called.
_SendAndClearText() called.
inputBuffer was L"한" and cleared.
_compositionStartedHandler() called.
_textUpdatingHandler() called.
Text:  ㄱ
Range: [0, 0]
_textUpdatingHandler() called.
Text:  ㄱ
Range: [0, 0]
_compositionCompletedHandler() called.
_SendAndClearText() called.
inputBuffer was L"ㄱㄱ" and cleared.

Is there anything that I misunderstood, or did something changed with 31c9d19a72 (diff-7708ccd4133d008adca4935827f7ddb7)?

https://github.com/simnalamburt/terminal/commit/acf74bc8ad947 this is a patch that I used for tracing.

@simnalamburt commented on GitHub (Mar 2, 2020): I tried to reproduce the scenario that you described, but I'm having trouble. I typed <kbd>ㅎ</kbd><kbd>ㅏ</kbd><kbd>ㄴ</kbd><kbd>ㄱ</kbd> and I got 2 textUpdate event instead of 0 after typing "한". ``` # Typed ㅎ _compositionStartedHandler() called. _textUpdatingHandler() called. Text: ㅎ Range: [0, 0] # Typed ㅏ _textUpdatingHandler() called. Text: 하 Range: [0, 1] # Typed ㄴ _textUpdatingHandler() called. Text: 한 Range: [0, 1] # Typed ㄱ _compositionCompletedHandler() called. _SendAndClearText() called. inputBuffer was L"한" and cleared. _compositionStartedHandler() called. _textUpdatingHandler() called. Text: ㄱ Range: [0, 0] _textUpdatingHandler() called. Text: ㄱ Range: [0, 0] _compositionCompletedHandler() called. _SendAndClearText() called. inputBuffer was L"ㄱㄱ" and cleared. ``` Is there anything that I misunderstood, or did something changed with https://github.com/microsoft/terminal/commit/31c9d19a72ef5fce19ab480c0a5e407064e15941#diff-7708ccd4133d008adca4935827f7ddb7? https://github.com/simnalamburt/terminal/commit/acf74bc8ad947 this is a patch that I used for tracing.

claunia commented

@leonMSFT commented on GitHub (Mar 2, 2020):

That's really weird! I pulled your branch from your fork (and the branch called patch-4226) and tried to do the same thing you were doing and this is what I'm getting:

After pressing ㄱ, as you can see, _textUpdatingHandler, _compositionCompletedHandler and _SendAndClearText are called in sequence, and another _compositionCompletedHandler is called afterwards. I'm not seeing the two extra _textUpdatingHandler calls that you're seeing though. 😢

@leonMSFT commented on GitHub (Mar 2, 2020): That's really weird! I pulled your branch from your fork (and the branch called patch-4226) and tried to do the same thing you were doing and this is what I'm getting: ![testingdebug1](https://user-images.githubusercontent.com/57155886/75725663-ca2c3a00-5c95-11ea-8437-4664c3e279d2.gif) After pressing <kbd>ㄱ</kbd>, as you can see, `_textUpdatingHandler`, `_compositionCompletedHandler` and `_SendAndClearText` are called in sequence, and another `_compositionCompletedHandler` is called afterwards. I'm not seeing the two extra `_textUpdatingHandler` calls that you're seeing though. 😢

claunia commented

@simnalamburt commented on GitHub (Mar 3, 2020):

That's strange. My issue (2 text update event) is being reproduced consistently on two computers, mine and PC of @guswns0528. I wonder what the difference is.

My Windows specifications:

Windows Terminal version: https://github.com/simnalamburt/terminal/commit/acf74bc8ad947
Windows Edition: Windows 10 Pro
Windows Version: 1909
Windows OS build: 18363.657
Processor type: 64-bit operating system, x64-based processor
Windows display language: English (United States)
Default app language, Default input language: 한국어

@simnalamburt commented on GitHub (Mar 3, 2020): That's strange. My issue (2 text update event) is being reproduced consistently on two computers, mine and PC of @guswns0528. I wonder what the difference is. My Windows specifications: - Windows Terminal version: https://github.com/simnalamburt/terminal/commit/acf74bc8ad947 - Windows Edition: Windows 10 Pro - Windows Version: 1909 - Windows OS build: 18363.657 - Processor type: 64-bit operating system, x64-based processor - Windows display language: English (United States) - Default app language, Default input language: 한국어

claunia commented