[MEGATHREAD] AtlasEngine bugs and missing features (experimental.useAtlasEngine) #13676

Open
opened 2026-01-31 03:49:20 +00:00 by claunia · 8 comments
Owner

Originally created by @zadjii-msft on GitHub (Apr 29, 2021).

Bugs

  • Desktop stutters on Nvidia GPUs due to G-SYNC switching on/off when two MPO planes are active (MSFT:37283316)
  • Zooming (font changes in general) are much slower than DxEngine
    👉 Doesn't occur with a local dev build. What's wrong with our release builds? 🤔
  • Atlas acts like IntenseIsBold is always on
  • Remaining gridlines (left, right, top, bottom, double underline)
  • Missing cursor when experimental text rendering engine is enabled in HDR enabled monitor #12507
  • Changing transparency from the command palette doesn't change the transparency until committed / pressing enter

Missing features

  • ClearType
  • Links not underlined with experimental.useAtlasEngine enabled #11871
  • A more conservative GPU memory usage
  • Shrinking the backing texture if possible
  • Double-height and double-width line support
  • Exited process, code 3221225786 (#13853)

New features

Cleanup

  • Deduplicate font fallback selection during UpdateFont(). Both DxEngine and AtlasEngine fall back to Consolas, etc. separately.

Things we think were fixed by this renderer...

...Or at least, this renderer got out of the way so that these can be fixed now.

  • #800 Ligatures do not render under cursor column in Vim
  • #3075 Processing large data is extremely slow (in VMware)
  • #3515 Extremely sluggish behavior when in full screen on a 4K monitor
  • #4974 [v0.10] When two characters are ligatured, the first one becomes invisible
  • #6738 Some of the extended Unicode characters are rendered with incorrect size and placement
  • #6974 Extremely lower performance when displaying too many box-shading glyphs
  • #7147 Support renderer fast presents
  • #7927 Frames and special characters from bitmap TTF font are not visible
  • #10462 Extremely slow performance when processing virtual terminal sequences
  • #13527 Unicode Box drawings characters not rendered correctly (disappear) with Japanese fonts
  • MSFT-38159826

Not really a bug

Originally created by @zadjii-msft on GitHub (Apr 29, 2021). # Bugs - [ ] Desktop stutters on Nvidia GPUs due to G-SYNC switching on/off when two MPO planes are active (MSFT:37283316) - [x] Zooming (font changes in general) are much slower than DxEngine 👉 Doesn't occur with a local dev build. What's wrong with our release builds? 🤔 - [x] Atlas acts like `IntenseIsBold` is always on - [x] Remaining gridlines (left, right, top, bottom, double underline) - [x] Missing cursor when experimental text rendering engine is enabled in HDR enabled monitor #12507 - [x] Changing transparency from the command palette doesn't change the transparency until committed / pressing enter # Missing features - [x] ClearType - [x] Links not underlined with experimental.useAtlasEngine enabled #11871 - [x] A more conservative GPU memory usage - [ ] Shrinking the backing texture if possible - [x] Double-height and double-width line support - [x] #13853 # New features - [ ] Raster fonts - [x] Soft fonts - [x] Investigate alternative ways to handle box drawing/block elements #5897 * Closes: experimental.useAtlasEngine breaks draw-box rendering #12512 * Possibly also #6161 - [x] Arrow (➜) character in prompt is rendered small #6864 # Cleanup - [x] Deduplicate font fallback selection during `UpdateFont()`. Both `DxEngine` and `AtlasEngine` fall back to Consolas, etc. separately. # Things we think were fixed by this renderer... ...Or at least, this renderer got out of the way so that these can be fixed now. - [x] #800 Ligatures do not render under cursor column in Vim - [x] #3075 Processing large data is extremely slow (in VMware) - [x] #3515 Extremely sluggish behavior when in full screen on a 4K monitor - [x] #4974 [v0.10] When two characters are ligatured, the first one becomes invisible - [x] #6738 Some of the extended Unicode characters are rendered with incorrect size and placement - [x] #6974 Extremely lower performance when displaying too many box-shading glyphs - [x] #7147 Support renderer fast presents - [x] #7927 Frames and special characters from bitmap TTF font are not visible - [ ] #10462 Extremely slow performance when processing virtual terminal sequences - [x] #13527 Unicode Box drawings characters not rendered correctly (disappear) with Japanese fonts - [x] MSFT-38159826 # Not _really_ a bug - [x] #12349
claunia added the Product-TerminalPriority-1Issue-ScenarioArea-AtlasEngine labels 2026-01-31 03:49:20 +00:00
Author
Owner

@DHowett commented on GitHub (Feb 1, 2022):

  • Atlas acts like IntenseIsBold is always on
@DHowett commented on GitHub (Feb 1, 2022): * [ ] Atlas acts like `IntenseIsBold` is always on
Author
Owner

@o-sdn-o commented on GitHub (May 5, 2024):

Alpha blending is broken in grayscale AA mode (dark moire around glyphs with high RGB-contrast between foreground and background). Apparently the transition between linear/sRGB is missing somewhere.

My setup: Current main branch. Windows 10.

pwsh:

"🥵🦚`e[38;2;255;0;0;48;2;0;0;255mtest`e[38;2;0;255;255mtest`e[m"

Grayscale AA mode:
image
image

Cleartype AA mode:
image

Expected grayscale AA mode blending:
image
image

@o-sdn-o commented on GitHub (May 5, 2024): Alpha blending is broken in grayscale AA mode (dark moire around glyphs with high RGB-contrast between foreground and background). Apparently the transition between linear/sRGB is missing somewhere. My setup: Current main branch. Windows 10. pwsh: ``` "🥵🦚`e[38;2;255;0;0;48;2;0;0;255mtest`e[38;2;0;255;255mtest`e[m" ``` Grayscale AA mode: ![image](https://github.com/microsoft/terminal/assets/11535558/daf44968-6d22-443a-a1b1-c01aba6c35bc) ![image](https://github.com/microsoft/terminal/assets/11535558/13d9a90f-a158-4de7-af7a-6da67e69ddd8) Cleartype AA mode: ![image](https://github.com/microsoft/terminal/assets/11535558/8345f363-690e-4d23-bdee-037ae2db0027) Expected grayscale AA mode blending: ![image](https://github.com/microsoft/terminal/assets/11535558/ccefb6a8-b8db-4b11-91e5-39e130460256) ![image](https://github.com/microsoft/terminal/assets/11535558/6e799803-cde4-4d7a-b0d3-1f314d6e13f8)
Author
Owner

@lhecker commented on GitHub (May 6, 2024):

Your issue is definitely weird. The blue text background in your two screenshots is #120efd and not #0000ff. Why's that? The red "test" text is also #ff0302 and not a clean #ff0000. Are you sure your screenshot wasn't chroma subsampled (I mean like YUV 4:2:2), or something similar?

In any case, I've tested this with Visual TrueType just now. First I've reset my ClearType settings so that I get a clean 1.8 gamma:

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Control Panel\Desktop]
"FontSmoothing"="2"
"FontSmoothingGamma"=dword:00000000
"FontSmoothingOrientation"=dword:00000001
"FontSmoothingType"=dword:00000002

[-HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics]

[-HKEY_LOCAL_MACHINE\Software\Microsoft\Avalon.Graphics]

Then I configured VTT like so:

  • Display > Size to 18pt
  • Display > Resolution to 144 dpi (this matches your screenshots)
  • Display > General Options: Here I made sure only "Follow gasp settings" and "ClearType gray" are checked at the end
  • Tools > Options: Set "Gamma correction" to 1800

This got me this:
image

which is almost identical to what BackendD3D renders for me (if you zoom into the "t"):
image

@lhecker commented on GitHub (May 6, 2024): Your issue is definitely weird. The blue text background in your two screenshots is `#120efd` and not `#0000ff`. Why's that? The red "test" text is also `#ff0302` and not a clean `#ff0000`. Are you sure your screenshot wasn't chroma subsampled (I mean like YUV 4:2:2), or something similar? In any case, I've tested this with Visual TrueType just now. First I've reset my ClearType settings so that I get a clean 1.8 gamma: ``` Windows Registry Editor Version 5.00 [HKEY_CURRENT_USER\Control Panel\Desktop] "FontSmoothing"="2" "FontSmoothingGamma"=dword:00000000 "FontSmoothingOrientation"=dword:00000001 "FontSmoothingType"=dword:00000002 [-HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics] [-HKEY_LOCAL_MACHINE\Software\Microsoft\Avalon.Graphics] ``` Then I configured VTT like so: * Display > Size to 18pt * Display > Resolution to 144 dpi (this matches your screenshots) * Display > General Options: Here I made sure only "Follow gasp settings" and "ClearType gray" are checked at the end * Tools > Options: Set "Gamma correction" to 1800 This got me this: ![image](https://github.com/microsoft/terminal/assets/2256941/e8ce03c5-3d80-40da-b396-ba68bf311042) which is almost identical to what BackendD3D renders for me (if you zoom into the "t"): ![image](https://github.com/microsoft/terminal/assets/2256941/7a76e1f4-8dde-478c-9946-e73c61c87a29)
Author
Owner

@o-sdn-o commented on GitHub (May 6, 2024):

The blue text background in your two screenshots is #120efd and not #0000ff. Why's that? The red "test" text is also #ff0302 and not a clean #ff0000. Are you sure your screenshot wasn't chroma subsampled (I mean like YUV 4:2:2), or something similar?

The screenshots were taken via RDP, perhaps there is some kind of post-processing.

Here is a screenshot from localhost:
image

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics]

[HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics\DISPLAY1]
"PixelStructure"=dword:00000002
"GammaLevel"=dword:00000898
"ClearTypeLevel"=dword:00000064
"EnhancedContrastLevel"=dword:00000032
"GrayscaleEnhancedContrastLevel"=dword:00000064
"TextContrastLevel"=dword:00000000

[HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics\DISPLAY3]
"PixelStructure"=dword:00000002
"GammaLevel"=dword:00000898
"ClearTypeLevel"=dword:00000064
"EnhancedContrastLevel"=dword:00000032
"GrayscaleEnhancedContrastLevel"=dword:00000064
"TextContrastLevel"=dword:00000000

My gamma is 2.2 (GammaLevel=2200), since sRGB-gamma is 2.2 on my local monitors with sRGB mode enabled (PHL234E5, ATI HD 3870).

This got me this...

If you convert your image to shades of gray, you will see overdarkening that, in theory, should not be there:

image

VTT does not take into account gamma correction with grayscale AA blending (or do it wrong):
image

Off: VTT has a memory leak: +100 MB of RAM on every (back and forth) window resizing step with "Segoe UI Emoji" open.

Off2: VTT does not allow the gamma correction value to be set to 2250, even though the error message states that the valid range is 1050-2250.

Here is my test image (expected grayscale blending):

image

Off3: Github breaks colors on pasted screenshots. Pay attention to the red "test" - in both screenshots it is the same image. At low resolution, compression changes color.

  • image
  • Original:
    image
@o-sdn-o commented on GitHub (May 6, 2024): > The blue text background in your two screenshots is #120efd and not #0000ff. Why's that? The red "test" text is also #ff0302 and not a clean #ff0000. Are you sure your screenshot wasn't chroma subsampled (I mean like YUV 4:2:2), or something similar? The screenshots were taken via RDP, perhaps there is some kind of post-processing. Here is a screenshot from localhost: ![image](https://github.com/microsoft/terminal/assets/11535558/adf205ed-4429-439b-ae03-2eac40d5b570) ``` Windows Registry Editor Version 5.00 [HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics] [HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics\DISPLAY1] "PixelStructure"=dword:00000002 "GammaLevel"=dword:00000898 "ClearTypeLevel"=dword:00000064 "EnhancedContrastLevel"=dword:00000032 "GrayscaleEnhancedContrastLevel"=dword:00000064 "TextContrastLevel"=dword:00000000 [HKEY_CURRENT_USER\Software\Microsoft\Avalon.Graphics\DISPLAY3] "PixelStructure"=dword:00000002 "GammaLevel"=dword:00000898 "ClearTypeLevel"=dword:00000064 "EnhancedContrastLevel"=dword:00000032 "GrayscaleEnhancedContrastLevel"=dword:00000064 "TextContrastLevel"=dword:00000000 ``` My gamma is 2.2 (GammaLevel=2200), since sRGB-gamma is 2.2 on my local monitors with sRGB mode enabled (PHL234E5, ATI HD 3870). > This got me this... If you convert your image to shades of gray, you will see overdarkening that, in theory, should not be there: ![image](https://github.com/microsoft/terminal/assets/11535558/595e5dae-db7b-4608-bf93-a22c6e5cdf7c) VTT does not take into account gamma correction with grayscale AA blending (or do it wrong): ![image](https://github.com/microsoft/terminal/assets/11535558/40a06be2-970c-4d72-bd89-f62fb1f54267) Off: VTT has a memory leak: +100 MB of RAM on every (back and forth) window resizing step with "Segoe UI Emoji" open. Off2: VTT does not allow the gamma correction value to be set to 2250, even though the error message states that the valid range is 1050-2250. Here is my test image (expected grayscale blending): ![image](https://github.com/microsoft/terminal/assets/11535558/9ec39f30-d5b5-42ac-bc9e-f2e0e3ec8ae7) Off3: Github breaks colors on pasted screenshots. Pay attention to the red "test" - in both screenshots it is the same image. At low resolution, compression changes color. - ![image](https://github.com/microsoft/terminal/assets/11535558/67c509a2-6ebe-4a2c-adb5-be056f92a14f) - Original: ![image](https://github.com/directvt/vtm/assets/11535558/18b01064-978c-434c-ae8d-03c83dcfeb04)
Author
Owner
@o-sdn-o commented on GitHub (May 6, 2024): ![image](https://github.com/microsoft/terminal/assets/11535558/299ad34a-fb77-4e5a-9217-5c714c09cebd) https://en.wikipedia.org/wiki/SRGB https://en.wikipedia.org/wiki/Alpha_compositing
Author
Owner

@o-sdn-o commented on GitHub (May 6, 2024):

I found that the deprecated DxRenderer (if I understand it correctly) GDI-renderer (with UseDx=0) (thanks to @tusharsnx) is not affected by this issue:

image
image

Make sure system-wide cleartype is disabled:
image

@o-sdn-o commented on GitHub (May 6, 2024): I found that the deprecated ~~DxRenderer (if I understand it correctly)~~ GDI-renderer (with `UseDx=0`) (thanks to @tusharsnx) is not affected by this issue: ![image](https://github.com/microsoft/terminal/assets/11535558/d22801e1-6ed7-4d5e-8c1e-1f77a36afd2b) ![image](https://github.com/microsoft/terminal/assets/11535558/f7d90661-d7ea-40f5-8409-1aec82b32822) Make sure system-wide cleartype is disabled: ![image](https://github.com/microsoft/terminal/assets/11535558/815b7ed2-6fe4-43f8-aca2-abb5f6c69db0)
Author
Owner

@tusharsnx commented on GitHub (May 6, 2024):

@o-sdn-o Nah, that's the GDI renderer (unless you have used useDX from the registry).

I do see similar result in conhost as you

Conhost with system-wide Cleartype off:

image

Windows Terminal with Grayscale AA:

image

Windows Terminal with ClearType AA:

image

@tusharsnx commented on GitHub (May 6, 2024): @o-sdn-o Nah, that's the [GDI renderer](https://github.com/microsoft/terminal/tree/3996806503bd573aa59a27d1ee8e98bd1d135b8b/src/renderer/gdi) (unless you have used `useDX` from the registry). <details> <summary>I do see similar result in conhost as you</summary> Conhost with system-wide Cleartype off: ![image](https://github.com/microsoft/terminal/assets/55626797/6282d926-ad4f-43ac-9e05-52866a163f26) Windows Terminal with Grayscale AA: ![image](https://github.com/microsoft/terminal/assets/55626797/a53d37c5-20a3-4440-9c6c-842fc928a8b6) Windows Terminal with ClearType AA: ![image](https://github.com/microsoft/terminal/assets/55626797/7f54251d-19b8-484a-84f0-73517896cbfe) </details>
Author
Owner

@ChGen commented on GitHub (Feb 3, 2025):

I regularly observe high Shared GPU memory usage (According to Task Manager values) over long period of time with 7 - 12 tabs. It could easily go more than 1GB of VRAM, higher than Firefox browser and VSCode, for example, which looks like a leak or inefficiency.
Windows 11 23H2 and 24H2. Intel UHD730/Iris XE graphics. Intel GPU drivers versions 31.0.* and 32.*.
I tried cleaning all tabs history and minimising/resizing window, but it doesn't affect shared GPU memory usage.
For 16gb laptops this is quite subtle memory leak, since main 16gb memory is shared with GPU.
Windows Terminal version 1.21.3231.0, but it affects older too.

@ChGen commented on GitHub (Feb 3, 2025): I regularly observe high `Shared GPU memory` usage (According to Task Manager values) over long period of time with 7 - 12 tabs. It could easily go more than _1GB of VRAM_, higher than Firefox browser and VSCode, for example, which looks like a leak or inefficiency. Windows 11 23H2 and 24H2. Intel UHD730/Iris XE graphics. Intel GPU drivers versions 31.0.* and 32.*. I tried cleaning all tabs history and minimising/resizing window, but it doesn't affect shared GPU memory usage. For 16gb laptops this is quite subtle memory leak, since main 16gb memory is shared with GPU. Windows Terminal version 1.21.3231.0, but it affects older too.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#13676