[PR #14959] [MERGED] Rewrite AtlasEngine to allow arbitrary overhangs #30323

Open
opened 2026-01-31 09:40:06 +00:00 by claunia · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/microsoft/terminal/pull/14959
Author: @lhecker
Created: 3/6/2023
Status: Merged
Merged: 4/26/2023
Merged by: @microsoft-github-policy-service[bot]

Base: mainHead: dev/lhecker/atlas-engine-remastered


📝 Commits (10+)

  • f24a9ea A minor AtlasEngine refactoring
  • 6ba233a Fix transparency, scrolling, dirty rects
  • 7ddddfe Improve performance, Fix OOM when drawing whitespace
  • 1eafcd4 Fix glyph rounding error, Fix custom shaders
  • 01e596c Adapter selection, Overlapping gridlines, QuadInstance simplification
  • c270284 Better dirty rect tracking and partial rerendering (WIP)
  • 339b892 Mostly fix dirty rects, Reduce memory/PCIe usage
  • d44974a Merge remote-tracking branch 'origin/main' into dev/lhecker/atlas-engine-remastered
  • 694daa7 Fix dirty area calculation, Add ATLAS_DEBUG_SHOW_DIRTY
  • badbd49 Finally fix broken rendering in BackendD3D

📊 Changes

34 files changed (+6162 additions, -3605 deletions)

View changed files

📝 .github/actions/spelling/allow/apis.txt (+1 -0)
📝 .github/actions/spelling/expect/expect.txt (+1 -0)
📝 NOTICE.md (+25 -0)
oss/stb/LICENSE (+37 -0)
oss/stb/MAINTAINER_README.md (+4 -0)
oss/stb/cgmanifest.json (+15 -0)
oss/stb/stb_rect_pack.h (+623 -0)
📝 src/interactivity/win32/window.hpp (+6 -1)
📝 src/renderer/atlas/AtlasEngine.api.cpp (+152 -127)
📝 src/renderer/atlas/AtlasEngine.cpp (+629 -1295)
📝 src/renderer/atlas/AtlasEngine.h (+49 -1016)
📝 src/renderer/atlas/AtlasEngine.r.cpp (+272 -952)
src/renderer/atlas/Backend.cpp (+109 -0)
src/renderer/atlas/Backend.h (+85 -0)
src/renderer/atlas/BackendD2D.cpp (+684 -0)
src/renderer/atlas/BackendD2D.h (+68 -0)
src/renderer/atlas/BackendD3D.cpp (+2065 -0)
src/renderer/atlas/BackendD3D.h (+311 -0)
📝 src/renderer/atlas/DWriteTextAnalysis.cpp (+6 -2)
📝 src/renderer/atlas/DWriteTextAnalysis.h (+3 -14)

...and 14 more files

📄 Description

This is practically a from scratch rewrite of AtlasEngine.

The initial approach used a very classic monospace text renderer, where
the viewport is subdivided into cells and each cell is assigned one
glyph texture, just like how real terminals used to work.
While we knew that it would have problems with overly large glyphs,
like those found in less often used languages, we didn't expect the
absolutely massive number of fonts that this approach would break.
For one, the assumption that monospace fonts are actually mostly
monospace has turned out to be a complete lie and we can't force users
to use better designed fonts. But more importantly, we can't just
design an entire Unicode fallback font collection from scratch where
every major glyph is monospace either. This is especially problematic
for vertical overhangs which are extremely difficult to handle in a
way that outperforms the much simpler alternative approach:
Just implementing a bog-standard, modern, quad-based text renderer.

Such an approach is both, less code and runs faster due to a less
complex CPU-side. The text shaping engine (in our case DirectWrite)
has to resolve text into glyph indices anyways, so using them directly
for text rendering allows reduces the effort of turning it back into
text ranges and hashing those. It's memory overhead is also reduced,
because we can now break up long ligatures into their individual glyphs.
Especially on AMD APUs I found this approach to run much faster.

A list of issues I think are either obsolete (and could be closed)
or resolved with this PR in combination with #14255:

Closes #6864
Closes #6974
Closes #8993
Closes #9940
Closes #10128
Closes #12537
Closes #13064
Closes #13527
Closes #13662
Closes #13700
Closes #13989
Closes #14022
Closes #14057
Closes #14094
Closes #14098
Closes #14117
Closes #14533
Closes #14877
Closes #9381

PR Checklist

  • Enabling software rendering enables D2D mode
  • Both D2D and D3D:
    • Background appears correctly
    • Text appears correctly
      • Cascadia Code Regular
      • Cascadia Code Bold
      • Cascadia Code Italic
      • Cascadia Code block chars leave (almost) no gaps
      • Terminus TTF at 13.5pt leaves no gaps between block chars
      • "`u{e0b2}`u{e0b0}" in Fira Code Nerd Font forms a square
    • Cursor appears correctly
      • Legacy small/medium/large
      • Vertical bar
      • Underscore
      • Empty box
      • Full box
      • Double underscore
    • Changing the cursor color works
    • Selection appears correctly
    • Scrolling in various manners always renders correctly
    • Changing the text antialising mode works
    • Semi-transparent backgrounds work
    • Scroll-zooming the font size works
    • Double-size characters work
    • Resizing while text is printing works
    • DWM +Heatmap_ShowDirtyRegions shows that only the cursor
      region is dirty when it's blinking
  • D2D
    • Margins are filled with background color
      They're filled with the neighboring's cell background color for
      convenience, as D2D doesn't support D3D11_TEXTURE_ADDRESS_BORDER
  • D3D
    • Margins are filled with background color
    • Soft fonts work
    • Custom shaders enable continous redraw if time constant is used
    • Retro shader appears correctly
    • Resizing while a custom shader is running works

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/microsoft/terminal/pull/14959 **Author:** [@lhecker](https://github.com/lhecker) **Created:** 3/6/2023 **Status:** ✅ Merged **Merged:** 4/26/2023 **Merged by:** [@microsoft-github-policy-service[bot]](https://github.com/apps/microsoft-github-policy-service) **Base:** `main` ← **Head:** `dev/lhecker/atlas-engine-remastered` --- ### 📝 Commits (10+) - [`f24a9ea`](https://github.com/microsoft/terminal/commit/f24a9ea8de333cd57eae62d0e936444bcefd3bd6) A minor AtlasEngine refactoring - [`6ba233a`](https://github.com/microsoft/terminal/commit/6ba233a27fed720b6e3aae51de1391969dc72b74) Fix transparency, scrolling, dirty rects - [`7ddddfe`](https://github.com/microsoft/terminal/commit/7ddddfe1611707518d960942325b28a8109ae693) Improve performance, Fix OOM when drawing whitespace - [`1eafcd4`](https://github.com/microsoft/terminal/commit/1eafcd4dd1356df347ccd0b9244601bb61376ca1) Fix glyph rounding error, Fix custom shaders - [`01e596c`](https://github.com/microsoft/terminal/commit/01e596ca7ce85b1add48b7b8abba0f8b0fd1e287) Adapter selection, Overlapping gridlines, QuadInstance simplification - [`c270284`](https://github.com/microsoft/terminal/commit/c270284b174a501801b950c7af3696b7d2fd3b2d) Better dirty rect tracking and partial rerendering (WIP) - [`339b892`](https://github.com/microsoft/terminal/commit/339b8924723b55d141aa7234402fff5765964d8d) Mostly fix dirty rects, Reduce memory/PCIe usage - [`d44974a`](https://github.com/microsoft/terminal/commit/d44974a92af3281ed9eeff5e70639f9cac6a386a) Merge remote-tracking branch 'origin/main' into dev/lhecker/atlas-engine-remastered - [`694daa7`](https://github.com/microsoft/terminal/commit/694daa7f24e07abd7b6709c984dbc0c3374a36ce) Fix dirty area calculation, Add ATLAS_DEBUG_SHOW_DIRTY - [`badbd49`](https://github.com/microsoft/terminal/commit/badbd49e2ccf7cc6845e7a155f46f8dcabd4c77d) Finally fix broken rendering in BackendD3D ### 📊 Changes **34 files changed** (+6162 additions, -3605 deletions) <details> <summary>View changed files</summary> 📝 `.github/actions/spelling/allow/apis.txt` (+1 -0) 📝 `.github/actions/spelling/expect/expect.txt` (+1 -0) 📝 `NOTICE.md` (+25 -0) ➕ `oss/stb/LICENSE` (+37 -0) ➕ `oss/stb/MAINTAINER_README.md` (+4 -0) ➕ `oss/stb/cgmanifest.json` (+15 -0) ➕ `oss/stb/stb_rect_pack.h` (+623 -0) 📝 `src/interactivity/win32/window.hpp` (+6 -1) 📝 `src/renderer/atlas/AtlasEngine.api.cpp` (+152 -127) 📝 `src/renderer/atlas/AtlasEngine.cpp` (+629 -1295) 📝 `src/renderer/atlas/AtlasEngine.h` (+49 -1016) 📝 `src/renderer/atlas/AtlasEngine.r.cpp` (+272 -952) ➕ `src/renderer/atlas/Backend.cpp` (+109 -0) ➕ `src/renderer/atlas/Backend.h` (+85 -0) ➕ `src/renderer/atlas/BackendD2D.cpp` (+684 -0) ➕ `src/renderer/atlas/BackendD2D.h` (+68 -0) ➕ `src/renderer/atlas/BackendD3D.cpp` (+2065 -0) ➕ `src/renderer/atlas/BackendD3D.h` (+311 -0) 📝 `src/renderer/atlas/DWriteTextAnalysis.cpp` (+6 -2) 📝 `src/renderer/atlas/DWriteTextAnalysis.h` (+3 -14) _...and 14 more files_ </details> ### 📄 Description This is practically a from scratch rewrite of AtlasEngine. The initial approach used a very classic monospace text renderer, where the viewport is subdivided into cells and each cell is assigned one glyph texture, just like how real terminals used to work. While we knew that it would have problems with overly large glyphs, like those found in less often used languages, we didn't expect the absolutely massive number of fonts that this approach would break. For one, the assumption that monospace fonts are actually mostly monospace has turned out to be a complete lie and we can't force users to use better designed fonts. But more importantly, we can't just design an entire Unicode fallback font collection from scratch where every major glyph is monospace either. This is especially problematic for vertical overhangs which are extremely difficult to handle in a way that outperforms the much simpler alternative approach: Just implementing a bog-standard, modern, quad-based text renderer. Such an approach is both, less code and runs faster due to a less complex CPU-side. The text shaping engine (in our case DirectWrite) has to resolve text into glyph indices anyways, so using them directly for text rendering allows reduces the effort of turning it back into text ranges and hashing those. It's memory overhead is also reduced, because we can now break up long ligatures into their individual glyphs. Especially on AMD APUs I found this approach to run much faster. A list of issues I think are either obsolete (and could be closed) or resolved with this PR in combination with #14255: Closes #6864 Closes #6974 Closes #8993 Closes #9940 Closes #10128 Closes #12537 Closes #13064 Closes #13527 Closes #13662 Closes #13700 Closes #13989 Closes #14022 Closes #14057 Closes #14094 Closes #14098 Closes #14117 Closes #14533 Closes #14877 Closes #9381 ## PR Checklist * Enabling software rendering enables D2D mode ✅ * Both D2D and D3D: * Background appears correctly ✅✅ * Text appears correctly * Cascadia Code Regular ✅✅ * Cascadia Code Bold ✅✅ * Cascadia Code Italic ✅✅ * Cascadia Code block chars leave (almost) no gaps ✅✅ * Terminus TTF at 13.5pt leaves no gaps between block chars ✅✅ * ``"`u{e0b2}`u{e0b0}"`` in Fira Code Nerd Font forms a square ✅✅ * Cursor appears correctly * Legacy small/medium/large ✅✅ * Vertical bar ✅✅ * Underscore ✅✅ * Empty box ✅✅ * Full box ✅✅ * Double underscore ✅✅ * Changing the cursor color works ✅✅ * Selection appears correctly ✅✅ * Scrolling in various manners always renders correctly ✅✅ * Changing the text antialising mode works ✅✅ * Semi-transparent backgrounds work ✅✅ * Scroll-zooming the font size works ✅✅ * Double-size characters work ✅✅ * Resizing while text is printing works ✅✅ * DWM `+Heatmap_ShowDirtyRegions` shows that only the cursor region is dirty when it's blinking ✅✅ * D2D * Margins are filled with background color ❌ They're filled with the neighboring's cell background color for convenience, as D2D doesn't support `D3D11_TEXTURE_ADDRESS_BORDER` * D3D * Margins are filled with background color ✅ * Soft fonts work ✅ * Custom shaders enable continous redraw if time constant is used ✅ * Retro shader appears correctly ✅ * Resizing while a custom shader is running works ✅ --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-31 09:40:06 +00:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#30323