Clear-Host fails to clear the first line #7037

claunia commented

@DHowett-MSFT commented on GitHub (Mar 24, 2020):

This is really strange. Would you mind running a quick test for me?

Can you do this from Powershell Core outside of Terminal? To reproduce the same conditions as Terminal runs in, you will need to run this first (it will delete and temporarily disable your scrollback, but this is an important part of the repro!)

$Host.UI.RawUI.BufferSize = $Host.UI.RawUI.WindowSize

@DHowett-MSFT commented on GitHub (Mar 24, 2020): This is really strange. Would you mind running a quick test for me? Can you do this from Powershell Core _outside_ of Terminal? To reproduce the same conditions as Terminal runs in, you will need to run this first (it will delete and temporarily disable your scrollback, but this is an important part of the repro!) ```powershell $Host.UI.RawUI.BufferSize = $Host.UI.RawUI.WindowSize ```

claunia commented

@brantburnett commented on GitHub (Mar 24, 2020):

@DHowett

I performed the following steps (version of coranavirus-tracker-cli on npmjs.com has some issues with upstream API sources, so I pulled latest master):

Pulled latest source from https://github.com/sagarkarira/coronavirus-tracker-cli (b5f2f7e26bb98df0e47f7ebd3dcb6899d30d9409)
npm install
npm ./bin/index.js followed by cls in Windows Terminal to confirm the problem is still present
Switched to running PowerShell Core 7.0.0 directly, and ran $Host.UI.RawUI.BufferSize = $Host.UI.RawUI.WindowSize
npm ./bin/index.js followed by cls

The problem did not appear when running directly in PowerShell Core, only in Windows Terminal hosted PowerShell Core.

@brantburnett commented on GitHub (Mar 24, 2020): @DHowett I performed the following steps (version of coranavirus-tracker-cli on npmjs.com has some issues with upstream API sources, so I pulled latest master): 1. Pulled latest source from https://github.com/sagarkarira/coronavirus-tracker-cli (b5f2f7e26bb98df0e47f7ebd3dcb6899d30d9409) 2. `npm install` 3. `npm ./bin/index.js` followed by `cls` in Windows Terminal to confirm the problem is still present 4. Switched to running PowerShell Core 7.0.0 directly, and ran `$Host.UI.RawUI.BufferSize = $Host.UI.RawUI.WindowSize` 5. `npm ./bin/index.js` followed by `cls` The problem did not appear when running directly in PowerShell Core, only in Windows Terminal hosted PowerShell Core.

claunia commented

@DHowett-MSFT commented on GitHub (Mar 27, 2020):

Yeah, alright, I can definitely reproduce this. Gotta fix for v1.

Notes to future debuggers:

When I run clear in powershell, this is what I get out of the pty.

␛[m␛[25l␛[94m␍␊␛[H(␛[m␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␛[1;2H␛[?25h␛[97mdhowett-sl␛[94m)␣␛[95m~␣␛[92m%␣

Weirdly enough, all those ESC [ K are coming through, but we don't end up clearing the line the prompt ends up on.

@DHowett-MSFT commented on GitHub (Mar 27, 2020): Yeah, alright, I can definitely reproduce this. Gotta fix for v1. Notes to future debuggers: When I run clear in powershell, this is what I get out of the pty. `␛[m␛[25l␛[94m␍␊␛[H(␛[m␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␍␊␛[K␛[1;2H␛[?25h␛[97mdhowett-sl␛[94m)␣␛[95m~␣␛[92m%␣` Weirdly enough, _all those `ESC [ K` are coming through_, but we don't end up clearing the line the prompt ends up on.

claunia commented

@DHowett-MSFT commented on GitHub (Mar 27, 2020):

␛[H(␛[m␍␊ -- we're moving home, then cr, lf, and emitting one fewer EL than the entire screen. This seems to be related to the VT renderer when we circle the buffer.

@DHowett-MSFT commented on GitHub (Mar 27, 2020): `␛[H(␛[m␍␊` -- we're moving home, then _cr, lf_, and emitting one fewer EL than the entire screen. This seems to be related to the VT renderer when we circle the buffer.

claunia commented

@DHowett-MSFT commented on GitHub (Mar 31, 2020):

Alright, this is also doable with some alt buffer trickery. It looks like we're not clearing the top line when [something something scrollback] happens.

From PowerShell 7:

"`e[?1049h`e#8" + ("`n" * $Host.UI.RawUI.WindowSize.Height) + "`e[?1049l"

enter the alt buffer
emit DECALN to fill the screen with EEEEEE...
push some of them off the screen with enough newlines to fill the screen height
exit the alt buffer

Critically, when we redraw PowerShell 7.0.0 we don't EL the rest of that line (!)

(note the missing ␛[K which should follow the 0)

@DHowett-MSFT commented on GitHub (Mar 31, 2020): Alright, this is also doable with some alt buffer trickery. It looks like we're not clearing the top line when [something something scrollback] happens. From PowerShell 7: ``` "`e[?1049h`e#8" + ("`n" * $Host.UI.RawUI.WindowSize.Height) + "`e[?1049l" ``` 1. enter the alt buffer 2. emit `DECALN` to fill the screen with `EEEEEE...` 3. push some of them off the screen with enough newlines to fill the screen height 4. exit the alt buffer ![image](https://user-images.githubusercontent.com/14316954/78068219-df57bf80-734c-11ea-8dfa-fc328ed9ae89.png) Critically, when we redraw `PowerShell 7.0.0` we don't `EL` the rest of that line (!) ![image](https://user-images.githubusercontent.com/14316954/78068261-f1d1f900-734c-11ea-9912-a199b1b0cd0e.png) (note the missing `␛[K` which should follow the `0`)

claunia commented

@DHowett-MSFT commented on GitHub (Mar 31, 2020):

VtEngine_TraceInvalidateScroll  scrollDelta: "(X:0, Y:-1)"
VtEngine_TraceInvalidate        invalidated: "(L:0, T:0, R:171, B:41) [W:171, H:41]"
VtEngine_TraceStartPaint        cursorMoved: true
                                invalidated: [whole screen],
                                lastView:    "(L:0, T:0, R:171, B:41) [W:171, H:41]"
                                quickReturn: false
                                scrollDelta: "(X:0, Y:-1)
VtEngine_TraceLastText          lastText:    "(X:0, Y:40)
VtEngine_TraceMoveCursor        cursorPos:   "(X:0, Y:40)
                                lastText:    "(X:0, Y:40)"
VtEngine_TraceString            seq:         ^J
VtEngine_TraceMoveCursor        cursorPos:   "(X:0, Y:0)"
                                lastText:    "(X:0, Y:40)"
VtEngine_TraceString            seq:         ^[[H
VtEngine_TraceString            seq:         PowerShellSPC7.0.0
VtEngine_TraceMoveCursor        cursorPos:   "(X:0, Y:1)"
                                lastText:    "(X:16, Y:0)"
VtEngine_TraceString            seq:         ^M^J
VtEngine_TraceString            seq:         CopyrightSPC(c)SPCMicrosoftSPCCorporation.SPCAllSPCrightsSPCreserved.

that scroll delta bit scares me

@DHowett-MSFT commented on GitHub (Mar 31, 2020): ``` VtEngine_TraceInvalidateScroll scrollDelta: "(X:0, Y:-1)" VtEngine_TraceInvalidate invalidated: "(L:0, T:0, R:171, B:41) [W:171, H:41]" VtEngine_TraceStartPaint cursorMoved: true invalidated: [whole screen], lastView: "(L:0, T:0, R:171, B:41) [W:171, H:41]" quickReturn: false scrollDelta: "(X:0, Y:-1) VtEngine_TraceLastText lastText: "(X:0, Y:40) VtEngine_TraceMoveCursor cursorPos: "(X:0, Y:40) lastText: "(X:0, Y:40)" VtEngine_TraceString seq: ^J VtEngine_TraceMoveCursor cursorPos: "(X:0, Y:0)" lastText: "(X:0, Y:40)" VtEngine_TraceString seq: ^[[H VtEngine_TraceString seq: PowerShellSPC7.0.0 VtEngine_TraceMoveCursor cursorPos: "(X:0, Y:1)" lastText: "(X:16, Y:0)" VtEngine_TraceString seq: ^M^J VtEngine_TraceString seq: CopyrightSPC(c)SPCMicrosoftSPCCorporation.SPCAllSPCrightsSPCreserved. ``` that scroll delta bit scares me

claunia commented

@DHowett-MSFT commented on GitHub (Mar 31, 2020):

Okay, nailed it down. This is happening because _newBottomLine is true, even though we're rendering at 0,0 and we shouldn't care about the new bottom line.
Perhaps we had an assumption that when something scrolled, it was always triggering a draw into one of the new bottom lines.

This concept is invalidated by a full screen scroll-and-redraw.

We need to determine whether we're writing to the "new" bottom line before we optimize around its contents already being empty. 😄

Open PR #5181 touches the code that sets _newBottomLine.

@DHowett-MSFT commented on GitHub (Mar 31, 2020): Okay, nailed it down. This is happening because `_newBottomLine` is true, _even though_ we're rendering at 0,0 and we shouldn't care about the new bottom line. Perhaps we had an assumption that when something scrolled, it was always triggering a draw into one of the new bottom lines. This concept is invalidated by a full screen scroll-and-redraw. We need to determine whether we're writing to the "new" bottom line before we optimize around its contents already being empty. :smile: Open PR #5181 touches the code that sets `_newBottomLine`.

claunia commented

@DHowett-MSFT commented on GitHub (Mar 31, 2020):

Moving the cursor clears the _newBottomLine state, but we only move the cursor after we do all the determinations about whether to clear the lines based on its state.

@DHowett-MSFT commented on GitHub (Mar 31, 2020): Moving the cursor clears the `_newBottomLine` state, but we only move the cursor after we do all the determinations about whether to clear the lines based on its state.

claunia commented

@zadjii-msft commented on GitHub (Apr 1, 2020):

Okay so #5181 almost fixes this too:

Note the one mysterious 'E' immediately following the prompt that's left untouched.

If you leave more spaces in the middle of the line, you'll get more 'E's

@zadjii-msft commented on GitHub (Apr 1, 2020): Okay so #5181 _almost_ fixes this too: ![image](https://user-images.githubusercontent.com/18356694/78186109-cb39be00-7431-11ea-9784-3580998df794.png) Note the one mysterious 'E' immediately following the prompt that's left untouched. If you leave more spaces in the middle of the line, you'll get more 'E's ![image](https://user-images.githubusercontent.com/18356694/78186280-16ec6780-7432-11ea-9dc2-ab296127c102.png)

claunia commented

@zadjii-msft commented on GitHub (Apr 2, 2020):

Oh my goodness gracious. You wanna know something crazy? This won't repro without PSReadline as well.
With:

Without:

The trick is that the 'E's will only appear in spaces after the end of one run of text, before the next one starts. PSReadline will color the input as blue-on-default, but pure powershell won't. I've incorporated this into the test I'm building for #5113.

@zadjii-msft commented on GitHub (Apr 2, 2020): Oh my goodness gracious. You wanna know something crazy? This won't repro without PSReadline as well. With: ![image](https://user-images.githubusercontent.com/18356694/78262549-58c6ed80-74c6-11ea-9f76-f10deffc9998.png) Without: ![image](https://user-images.githubusercontent.com/18356694/78262561-5d8ba180-74c6-11ea-984c-e3badf0f865f.png) The trick is that the 'E's will only appear in spaces after the end of one run of text, before the next one starts. PSReadline will color the input as blue-on-default, but pure powershell won't. I've incorporated this into the test I'm building for #5113.

claunia commented