Add support for horizontal margin sequences #19408

Closed
opened 2026-01-31 06:42:23 +00:00 by claunia · 14 comments
Owner

Originally created by @j4james on GitHub (Feb 19, 2023).

Description of the new feature/enhancement

The DECSLRM escape sequence lets you set left and right margins, so you can wrap your output within a given horizontal range, and also limit the scrolling within those boundaries. This is useful for apps like multiplexers, where you can have two panes side by side, and you need to be able to scroll the one side independently of the other.

Proposed technical implementation details (optional)

There are actually two sequences we need to implement for this. The first is DECLRMM (Left Right Margin Mode), without which the margin functionality won't be active. It's disabled by default because you can't use horizontal margins at the same time as double-width line attributes.

The main sequence is DECSLRM (Set Left Right Margins), which works similarly to the DECSTBM sequence (Set Top Bottom Margins) which we already support. But there a bunch of operations we then need to update to take those margins into account - cursor movement, insert and delete ops, text output, etc.

I should also note that the DECSLRM sequence clashes with the ANSISYSSC sequence which we already support. But the way most modern terminals deal with that is to disable ANSISYSSC when the DECLRMM mode is enabled. So by default CSI s is ANSISYSSC (as we have it now), but with DECLRMM enabled it's interpreted as DECSLRM.

Originally created by @j4james on GitHub (Feb 19, 2023). # Description of the new feature/enhancement The `DECSLRM` escape sequence lets you set left and right margins, so you can wrap your output within a given horizontal range, and also limit the scrolling within those boundaries. This is useful for apps like multiplexers, where you can have two panes side by side, and you need to be able to scroll the one side independently of the other. # Proposed technical implementation details (optional) There are actually two sequences we need to implement for this. The first is `DECLRMM` (Left Right Margin Mode), without which the margin functionality won't be active. It's disabled by default because you can't use horizontal margins at the same time as double-width line attributes. The main sequence is `DECSLRM` (Set Left Right Margins), which works similarly to the `DECSTBM` sequence (Set Top Bottom Margins) which we already support. But there a bunch of operations we then need to update to take those margins into account - cursor movement, insert and delete ops, text output, etc. I should also note that the `DECSLRM` sequence clashes with the `ANSISYSSC` sequence which we already support. But the way most modern terminals deal with that is to disable `ANSISYSSC` when the `DECLRMM` mode is enabled. So by default `CSI s` is `ANSISYSSC` (as we have it now), but with `DECLRMM` enabled it's interpreted as `DECSLRM`.
Author
Owner

@j4james commented on GitHub (Feb 19, 2023):

This is another one of those sequences which has lots of edge cases that aren't always clearly specified, and which everyone handles differently. So before I submit a PR for this, it would be ideal if we could get some of those aspects clarified by testing on a real DEC terminal.

To that end I've created a Python script with a series of test cases here:
https://gist.github.com/j4james/2eae626e461175cfcc29a304e066ae87

@KalleOlaviNiemitalo and @jerch, I'd be very grateful if either of you would be willing to give that a run on your VT420 or VT525. It's setup as an automated test, so you shouldn't need to do anything other than run it, and the results will be logged in a file named margins.log.

Note that it could take a while, because it uses DECRQCRA queries to read the screen content when checking the output. But it outputs the results of each test as they're executed, so you should be able to tell whether it's frozen for some reason.

I should also mention that it highlights the results with reverse video if they've "failed", but that's just based on my best guess of how things should work. I wouldn't be surprised if the real terminals behave differently from my expectations.

Also note that some of the operations were only supported on the VT510 and above, so you can definitely expect to see failures for those if testing on a VT420. I think that includes HPA, HPR, VPA, VPR, CNL, CPL, CHT, and CBT.

So @KalleOlaviNiemitalo, if the test is running really slowly, and you want to save some time, feel free to comment those ones out. You'll find all the test groups listed at the bottom of the script, so for example, you can comment out the test_hpa() call to skip that particular group.

@j4james commented on GitHub (Feb 19, 2023): This is another one of those sequences which has lots of edge cases that aren't always clearly specified, and which everyone handles differently. So before I submit a PR for this, it would be ideal if we could get some of those aspects clarified by testing on a real DEC terminal. To that end I've created a Python script with a series of test cases here: https://gist.github.com/j4james/2eae626e461175cfcc29a304e066ae87 @KalleOlaviNiemitalo and @jerch, I'd be very grateful if either of you would be willing to give that a run on your VT420 or VT525. It's setup as an automated test, so you shouldn't need to do anything other than run it, and the results will be logged in a file named `margins.log`. Note that it could take a while, because it uses `DECRQCRA` queries to read the screen content when checking the output. But it outputs the results of each test as they're executed, so you should be able to tell whether it's frozen for some reason. I should also mention that it highlights the results with reverse video if they've "failed", but that's just based on my best guess of how things should work. I wouldn't be surprised if the real terminals behave differently from my expectations. Also note that some of the operations were only supported on the VT510 and above, so you can definitely expect to see failures for those if testing on a VT420. I think that includes `HPA`, `HPR`, `VPA`, `VPR`, `CNL`, `CPL`, `CHT`, and `CBT`. So @KalleOlaviNiemitalo, if the test is running really slowly, and you want to save some time, feel free to comment those ones out. You'll find all the test groups listed at the bottom of the script, so for example, you can comment out the `test_hpa()` call to skip that particular group.
Author
Owner

@j4james commented on GitHub (Mar 6, 2023):

I had some spare time this weekend and was able to make a serial cable to connect VT520 to Linux (RPi). Then I downloaded your .py script and ran it. It runs pretty long, so I had to cancel it at some point (esp. when it was filling up the screen with letters and doing some text cutting), but I did notice output in reverse meaning (according to the opening comments in the source code) that some assumptions were not met. Anyways, I'm not exactly sure how to share the results with you. I can take a video on my phone and then send you a link.

@al20878: Thanks for giving it try. I probably should have explained that it switches to a second page for some of the tests, where it fills the screen with characters and then performs various operations on that content. When each test is finished, it switches back to the initial page to output the results. On a slow connection it's probably going to seem like you're on the second test page for most of the time, and you may not even notice the results page until the very end.

Anyway, it's not important that you see the results, and you don't need to take a video, since it writes everything to a margins.log file. Even if you stop the test early, there should at least be a partial log. If you could upload that here I can see if it's doing something meaningful.

@j4james commented on GitHub (Mar 6, 2023): > I had some spare time this weekend and was able to make a serial cable to connect VT520 to Linux (RPi). Then I downloaded your .py script and ran it. It runs pretty long, so I had to cancel it at some point (esp. when it was filling up the screen with letters and doing some text cutting), but I did notice output in reverse meaning (according to the opening comments in the source code) that some assumptions were not met. Anyways, I'm not exactly sure how to share the results with you. I can take a video on my phone and then send you a link. @al20878: Thanks for giving it try. I probably should have explained that it switches to a second page for some of the tests, where it fills the screen with characters and then performs various operations on that content. When each test is finished, it switches back to the initial page to output the results. On a slow connection it's probably going to seem like you're on the second test page for most of the time, and you may not even notice the results page until the very end. Anyway, it's not important that you see the results, and you don't need to take a video, since it writes everything to a `margins.log` file. Even if you stop the test early, there should at least be a partial log. If you could upload that here I can see if it's doing something meaningful.
Author
Owner

@al20878 commented on GitHub (Mar 6, 2023):

No problem, @j4james , I'll just leave it run on VT420 and then submit the logfile (the connection speed is 9600 baud, not too bad, but it'll take some time to finish).

@al20878 commented on GitHub (Mar 6, 2023): No problem, @j4james , I'll just leave it run on VT420 and then submit the logfile (the connection speed is 9600 baud, not too bad, but it'll take some time to finish).
Author
Owner

@al20878 commented on GitHub (Mar 6, 2023):

It finished in 7 and a half minutes
margins.vt420.log

@al20878 commented on GitHub (Mar 6, 2023): It finished in 7 and a half minutes [margins.vt420.log](https://github.com/microsoft/terminal/files/10899539/margins.vt420.log)
Author
Owner

@al20878 commented on GitHub (Mar 6, 2023):

VT520 finished the same script in 8:56 (same 9600 baud), the results are here:
margins.vt520.log

@al20878 commented on GitHub (Mar 6, 2023): VT520 finished the same script in 8:56 (same 9600 baud), the results are here: [margins.vt520.log](https://github.com/microsoft/terminal/files/10899670/margins.vt520.log)
Author
Owner

@al20878 commented on GitHub (Mar 6, 2023):

VT525 has done it in 5:56, but I noticed it wasn't getting any characters back from the terminal, unlike previously, as everything was coming in as asterisks ('*'). All three terminals are real H/W and were reset to factory (default) settings.
margins.vt525.log

@al20878 commented on GitHub (Mar 6, 2023): VT525 has done it in 5:56, but I noticed it wasn't getting any characters back from the terminal, unlike previously, as everything was coming in as asterisks ('*'). All three terminals are real H/W and were reset to factory (default) settings. [margins.vt525.log](https://github.com/microsoft/terminal/files/10899840/margins.vt525.log)
Author
Owner

@al20878 commented on GitHub (Mar 6, 2023):

Let me know if you need anything else!

@al20878 commented on GitHub (Mar 6, 2023): Let me know if you need anything else!
Author
Owner

@j4james commented on GitHub (Mar 6, 2023):

@al20878 This is brilliant! Thank you so much! There are quite a lot of operations where the results weren't what I expected, but that's exactly why I wanted to run these tests. It's great to have these details confirmed.

VT525 has done it in 5:56, but I noticed it wasn't getting any characters back from the terminal

That's interesting. I'm using the DECRQCRA checksum operation to "read" the screen, but the details of the checksum algorithm aren't actually documented anywhere. I've been assuming it was one of two commonly used formats I've seen in other terminal emulators, one of which I know was derived from a real VT520. But I'm guessing the VT525 algorithm is different (possibly because it covers things like color attributes), so that would be an interesting subject for a future investigation.

@j4james commented on GitHub (Mar 6, 2023): @al20878 This is brilliant! Thank you so much! There are quite a lot of operations where the results weren't what I expected, but that's exactly why I wanted to run these tests. It's great to have these details confirmed. > VT525 has done it in 5:56, but I noticed it wasn't getting any characters back from the terminal That's interesting. I'm using the [DECRQCRA](https://vt100.net/docs/vt510-rm/DECRQCRA.html) checksum operation to "read" the screen, but the details of the checksum algorithm aren't actually documented anywhere. I've been assuming it was one of two commonly used formats I've seen in other terminal emulators, one of which I know was derived from a real VT520. But I'm guessing the VT525 algorithm is different (possibly because it covers things like color attributes), so that would be an interesting subject for a future investigation.
Author
Owner

@j4james commented on GitHub (Mar 8, 2023):

@al20878 Now that I've had a chance to look at all the results in detail, the only thing I'm unsure about is the handling of DECOM (origin mode). And I think the problem is with the way my test script is evaluating the results. I'm relying on certain behavior of the DECRC (restore cursor) operation, which I suspect might have changed in later DEC terminals.

So when you have a chance, I've got another little script (linked here) that I'm hoping you'll test for me. All it does is write 4 digits to the screen, while altering the margins and saving and restoring the cursor position with DECSC and DECRC. There's a short pause after each digit, so you can see what's happening (just on the off chance they end up overwriting each other).

On a VT100 and VT240 (tested on MAME), the output looks like this:

3







1

                                       24

My theory is that the later DEC terminals do something different, though. My guess is the 4 will be output several rows up (line 3 to be exact).

@j4james commented on GitHub (Mar 8, 2023): @al20878 Now that I've had a chance to look at all the results in detail, the only thing I'm unsure about is the handling of `DECOM` (origin mode). And I think the problem is with the way my test script is evaluating the results. I'm relying on certain behavior of the `DECRC` (restore cursor) operation, which I suspect might have changed in later DEC terminals. So when you have a chance, I've got another little script (linked [here](https://gist.github.com/j4james/6eef941cd763c05340c76fd021995e33)) that I'm hoping you'll test for me. All it does is write 4 digits to the screen, while altering the margins and saving and restoring the cursor position with `DECSC` and `DECRC`. There's a short pause after each digit, so you can see what's happening (just on the off chance they end up overwriting each other). On a VT100 and VT240 (tested on MAME), the output looks like this: ``` 3 1 24 ``` My theory is that the later DEC terminals do something different, though. My guess is the `4` will be output several rows up (line 3 to be exact).
Author
Owner

@al20878 commented on GitHub (Mar 8, 2023):

Sure, I'll run the script on the real terminals. So far I ran it in PuTTY, and I saw this:

3







1

                                       24








pi@raspberrypi:~$





@al20878 commented on GitHub (Mar 8, 2023): Sure, I'll run the script on the real terminals. So far I ran it in PuTTY, and I saw this: ``` 3 1 24 pi@raspberrypi:~$ ```
Author
Owner

@al20878 commented on GitHub (Mar 8, 2023):

Obviously, I would not be able to copy-paste the same from the terminals, but I can compare what would they show. I need to do a little cable reconf to reattach them in the proper order (now that I have the right cable adapter, it won't be a problem at all, and I'll do it shortly). Stay tuned

@al20878 commented on GitHub (Mar 8, 2023): Obviously, I would not be able to copy-paste the same from the terminals, but I can compare what would they show. I need to do a little cable reconf to reattach them in the proper order (now that I have the right cable adapter, it won't be a problem at all, and I'll do it shortly). Stay tuned
Author
Owner

@al20878 commented on GitHub (Mar 8, 2023):

@j4james as the saying goes, a picture is worth a thousand words, I'm attaching the "screenshots" of your script's output on 5 DEC terminals (VT320, VT330, VT420, VT520, and VT525, in order):
vt320
vt330
vt420
vt520
vt525
Let me know if you need anything else!

@al20878 commented on GitHub (Mar 8, 2023): @j4james as the saying goes, a picture is worth a thousand words, I'm attaching the "screenshots" of your script's output on 5 DEC terminals (VT320, VT330, VT420, VT520, and VT525, in order): ![vt320](https://user-images.githubusercontent.com/44822487/223612474-847915d0-1dab-4699-927a-f9839d180bd8.jpg) ![vt330](https://user-images.githubusercontent.com/44822487/223612479-d76c736b-a6e0-45b6-bf69-7dcefd8f0dcb.jpg) ![vt420](https://user-images.githubusercontent.com/44822487/223612573-3e8a4f87-14ec-4d4a-a764-9d9dbcddf39f.jpg) ![vt520](https://user-images.githubusercontent.com/44822487/223612590-44a31951-30f5-4a63-a8c4-5e5aa018114f.jpg) ![vt525](https://user-images.githubusercontent.com/44822487/223612607-41054808-ff26-4745-b165-080e6407515b.jpg) Let me know if you need anything else!
Author
Owner

@j4james commented on GitHub (Mar 8, 2023):

@al20878 This is great, thanks! I'm so glad you tested on the VT320 and VT330 as well, so now we know exactly when the new behavior was introduced. I'm now inclined to update the Windows Terminal implementation to match the VT420/VT5xx devices by default, because that's seems to be the more "correct" behavior. In the long term we can always have an option for more strict compatibility with the lower level devices for edge cases like this.

Let me know if you need anything else!

There are a number of other operations I'd like to get a better understanding of, so if you don't mind doing this sort of thing, I'd love to ping you from time to time when I've got something ready to test. But feel free to ignore me if you're busy or if you lose interest.

@j4james commented on GitHub (Mar 8, 2023): @al20878 This is great, thanks! I'm so glad you tested on the VT320 and VT330 as well, so now we know exactly when the new behavior was introduced. I'm now inclined to update the Windows Terminal implementation to match the VT420/VT5xx devices by default, because that's seems to be the more "correct" behavior. In the long term we can always have an option for more strict compatibility with the lower level devices for edge cases like this. > Let me know if you need anything else! There are a number of other operations I'd like to get a better understanding of, so if you don't mind doing this sort of thing, I'd love to ping you from time to time when I've got something ready to test. But feel free to ignore me if you're busy or if you lose interest.
Author
Owner

@al20878 commented on GitHub (Mar 9, 2023):

@j4james I'm happy to help!

to ping you from time to time when I've got something ready to test

Sure, reach out any time!

@al20878 commented on GitHub (Mar 9, 2023): @j4james I'm happy to help! > to ping you from time to time when I've got something ready to test Sure, reach out any time!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#19408