[PR #15098] Fix accuracy bugs around float/double/int conversions #30415

Open
opened 2026-01-31 09:40:42 +00:00 by claunia · 0 comments
Owner

Original Pull Request: https://github.com/microsoft/terminal/pull/15098

State: closed
Merged: Yes


I noticed this bug while resizing my window on my 150% scale display.
Every 3 "snaps" of the window size, it would fail to resize the text
buffer. I found that this occurs, because we convert the swap chain
size from a float into a double, which converts my 597.333313 height
into 597.33331298828125, which then multiplied by 1.5 results in
895.999969482421875. If you just cast this to an integer, it'll
result in a height of 895px instead of the expected 896px.

This PR addresses the issue in two ways:

  • Replace casts to integers with lrint or floor, etc.
  • Remove many of the redundant double <> float conversions.

PR Checklist

  • Resizing my window always resizes the text buffer
**Original Pull Request:** https://github.com/microsoft/terminal/pull/15098 **State:** closed **Merged:** Yes --- I noticed this bug while resizing my window on my 150% scale display. Every 3 "snaps" of the window size, it would fail to resize the text buffer. I found that this occurs, because we convert the swap chain size from a float into a double, which converts my 597.333313 height into 597.33331298828125, which then multiplied by 1.5 results in 895.999969482421875. If you just cast this to an integer, it'll result in a height of 895px instead of the expected 896px. This PR addresses the issue in two ways: * Replace casts to integers with `lrint` or `floor`, etc. * Remove many of the redundant double <> float conversions. ## PR Checklist * Resizing my window always resizes the text buffer ✅
claunia added the pull-request label 2026-01-31 09:40:42 +00:00
Sign in to join this conversation.
No Label pull-request
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#30415