[PR #2368] [MERGED] Replace CodepointWidthDetector's runtime table with a static one #24902

Open
opened 2026-01-31 09:06:01 +00:00 by claunia · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/microsoft/terminal/pull/2368
Author: @DHowett-MSFT
Created: 8/8/2019
Status: ✅ Merged
Merged: 8/16/2019
Merged by: @DHowett-MSFT

Base: master ← Head: dev/duhowett/codepoint


📝 Commits (2)

  • 50dcfe1 Replace CodepointWidthDetector's runtime table with a static one
  • 9dbfa7f Update src/types/CodepointWidthDetector.cpp

📊 Changes

3 files changed (+318 additions, -1207 deletions)

View changed files

📝 src/host/ut_host/CodepointWidthDetectorTests.cpp (+0 -20)
📝 src/types/CodepointWidthDetector.cpp (+317 -1110)
📝 src/types/inc/CodepointWidthDetector.hpp (+1 -77)

📄 Description

This commit replaces CodepointWidthDetector's
dynamically-generated map with a static constexpr one that's compiled
into the binary.

It also almost totally removes the notion of an Invalid width. We
definitely had gaps in our character coverage where we'd report a
character as invalid, but we'd then flatten that down to Narrow when
asked. By combining the not-present state and the narrow state, we get
to save a significant chunk of data.

I've tested this by feeding it all 0x10FFFF codepoints (and then some)
and making sure they 100% match the old code's outputs.

Metric Then Now
disk space 56k (.text) 3k (.rdata)
runtime memory (allocations) 1088 0
runtime memory (bytes) 51k ~0
memory behavior not shared fully shared
lookup time ~31ns ~9ns
first hit penalty ~170000ns 0ns
lines of code 1088 285
clarity extreme slightly worse

I also took a moment and cleaned up a stray boolean that we didn't need.


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/microsoft/terminal/pull/2368 **Author:** [@DHowett-MSFT](https://github.com/DHowett-MSFT) **Created:** 8/8/2019 **Status:** ✅ Merged **Merged:** 8/16/2019 **Merged by:** [@DHowett-MSFT](https://github.com/DHowett-MSFT) **Base:** `master` ← **Head:** `dev/duhowett/codepoint` --- ### 📝 Commits (2) - [`50dcfe1`](https://github.com/microsoft/terminal/commit/50dcfe19192e947ed74438b780323ce576d988b4) Replace CodepointWidthDetector's runtime table with a static one - [`9dbfa7f`](https://github.com/microsoft/terminal/commit/9dbfa7f2fd199de8df7c0355cc03baed561e7e9f) Update src/types/CodepointWidthDetector.cpp ### 📊 Changes **3 files changed** (+318 additions, -1207 deletions) <details> <summary>View changed files</summary> 📝 `src/host/ut_host/CodepointWidthDetectorTests.cpp` (+0 -20) 📝 `src/types/CodepointWidthDetector.cpp` (+317 -1110) 📝 `src/types/inc/CodepointWidthDetector.hpp` (+1 -77) </details> ### 📄 Description This commit replaces CodepointWidthDetector's dynamically-generated map with a static constexpr one that's compiled into the binary. It also almost totally removes the notion of an `Invalid` width. We definitely had gaps in our character coverage where we'd report a character as invalid, but we'd then flatten that down to `Narrow` when asked. By combining the not-present state and the narrow state, we get to save a significant chunk of data. I've tested this by feeding it all 0x10FFFF codepoints (and then some) and making sure they 100% match the old code's outputs. | Metric | Then | Now | |------------------------------|---------------|----------------| | disk space | 56k (`.text`) | 3k (`.rdata`) | | runtime memory (allocations) | 1088 | 0 | | runtime memory (bytes) | 51k | ~0 | | memory behavior | not shared | fully shared | | lookup time | ~31ns | ~9ns | | first hit penalty | ~170000ns | 0ns | | lines of code | 1088 | 285 | | clarity | extreme | slightly worse | I also took a moment and cleaned up a stray boolean that we didn't need. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-31 09:06:01 +00:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#24902