starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

[PR #6242] [CLOSED] Feature: mjpeg streaming for terminal control background #26615

New Issue
Open
opened 2026-01-31 09:17:08 +00:00 by claunia · 0 comments
claunia commented 2026-01-31 09:17:08 +00:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/microsoft/terminal/pull/6242
Author: @sipsorcery
Created: 5/28/2020
Status: ❌ Closed

Base: main ← Head: mjpeg

📝 Commits (3)

  • 5de4da6 Mjpeg streaming for terminal control background.
  • 591465e Executed runformat tool.
  • 9c67742 Added spelling whitelist file.

📊 Changes

9 files changed (+887 additions, -12 deletions)

View changed files

📝 .github/actions/spell-check/dictionary/names.txt (+1 -0)
➕ .github/actions/spell-check/whitelist/eacbc106c4603f6128d726ec0fdcb1cbfc6cab94.txt (+55 -0)
📝 src/cascadia/TerminalControl/TermControl.cpp (+72 -1)
📝 src/cascadia/TerminalControl/TermControl.h (+6 -0)
📝 src/cascadia/TerminalControl/TerminalControl.vcxproj (+4 -6)
📝 src/cascadia/TerminalControl/TerminalControl.vcxproj.filters (+7 -5)
➕ src/cascadia/TerminalControl/mjpeg.h (+472 -0)
➕ src/cascadia/TerminalControl/rtpsocket.cpp (+181 -0)
➕ src/cascadia/TerminalControl/rtpsocket.h (+89 -0)

📄 Description

This PR adds a rudimentary mjpeg receiver to allow the terminal control background to display a stream.

Update:

Motivation for this feature

The example screenshots and video are a little bit gimmicky but there could be some very useful scenarios for the ability to employ an information rich terminal background:

  • Network monitoring tools could cause the background to display results, e.g. a ping could cause a host in a network map to change colour,

  • Quick video calls (my own goal). For developers who spend a lot of time in a console it's advantageous to avoid context switches to other applications for minor tasks. If a quick video call could be done "in-terminal" that could be a small efficiency gain,

  • Infrequent graphical notifications. A video stream of 1 frame/second or less could be used to show notifications for configurable events. Again this could be a context switch saving for people that permanently have a terminal open. GitHub notifications would be a good use case. I do 90% of my git interactions from the command line. If the results of commit CI jobs were automatically displayed in the terminal background (that I keep open solely for git interactions) it could save me reading an email and visiting a web page.

It's perhaps easier to illustrate than to explain. The two screenshots below show stills of two different streams. Small mp4 captures are here and here (the second one's a bit spooky).

terminal_mjpeg_test_pattern

terminal_mjpeg_max

To test out the feature the steps are:

  • Set the backgroundImage to a file path with ;mjpeg:1234 appended. The 1234 represents a loopback port that the mjpeg receiving socket will listen on. For example:

"backgroundImage": "C:/Users/aaron/Pictures/gravatar_400x400.png;mjpeg:10010"

  • Start a source stream to the listening port. For example:

ffmpeg -re -f lavfi -i testsrc=size=1280x720:rate=10 -vf "drawtext=fontfile=/Windows/Fonts/arial.ttf: expansion=normal: text='%{localtime}': r=25: x=1100: y=50: fontcolor=black" -vcodec mjpeg -pix_fmt yuvj420p -huffman 0 -f rtp rtp://127.0.0.1:10010

Known Issues:

  • Only one terminal control can listen on a port at a time so starting a second terminal with the same backgroundImage setting will not be able to create a listening socket.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/microsoft/terminal/pull/6242 **Author:** [@sipsorcery](https://github.com/sipsorcery) **Created:** 5/28/2020 **Status:** ❌ Closed **Base:** `main` ← **Head:** `mjpeg` --- ### 📝 Commits (3) - [`5de4da6`](https://github.com/microsoft/terminal/commit/5de4da6cf8ff238f143e6132653bd74aaafde4fb) Mjpeg streaming for terminal control background. - [`591465e`](https://github.com/microsoft/terminal/commit/591465ef359ea29860bc95328fab3a7e5fe7c802) Executed runformat tool. - [`9c67742`](https://github.com/microsoft/terminal/commit/9c67742e0698c991de13eac4326833e48448acc7) Added spelling whitelist file. ### 📊 Changes **9 files changed** (+887 additions, -12 deletions) <details> <summary>View changed files</summary> 📝 `.github/actions/spell-check/dictionary/names.txt` (+1 -0) ➕ `.github/actions/spell-check/whitelist/eacbc106c4603f6128d726ec0fdcb1cbfc6cab94.txt` (+55 -0) 📝 `src/cascadia/TerminalControl/TermControl.cpp` (+72 -1) 📝 `src/cascadia/TerminalControl/TermControl.h` (+6 -0) 📝 `src/cascadia/TerminalControl/TerminalControl.vcxproj` (+4 -6) 📝 `src/cascadia/TerminalControl/TerminalControl.vcxproj.filters` (+7 -5) ➕ `src/cascadia/TerminalControl/mjpeg.h` (+472 -0) ➕ `src/cascadia/TerminalControl/rtpsocket.cpp` (+181 -0) ➕ `src/cascadia/TerminalControl/rtpsocket.h` (+89 -0) </details> ### 📄 Description This PR adds a rudimentary mjpeg receiver to allow the terminal control background to display a stream. **Update:** Motivation for this feature The example screenshots and video are a little bit gimmicky but there could be some very useful scenarios for the ability to employ an information rich terminal background: - Network monitoring tools could cause the background to display results, e.g. a ping could cause a host in a network map to change colour, - Quick video calls (my own goal). For developers who spend a lot of time in a console it's advantageous to avoid context switches to other applications for minor tasks. If a quick video call could be done "in-terminal" that could be a small efficiency gain, - Infrequent graphical notifications. A video stream of 1 frame/second or less could be used to show notifications for configurable events. Again this could be a context switch saving for people that permanently have a terminal open. GitHub notifications would be a good use case. I do 90% of my git interactions from the command line. If the results of commit CI jobs were automatically displayed in the terminal background (that I keep open solely for git interactions) it could save me reading an email and visiting a web page. It's perhaps easier to illustrate than to explain. The two screenshots below show stills of two different streams. Small mp4 captures are [here](https://github.com/sipsorcery/sipsorcery-mediafiles/raw/master/terminal/terminal_mjpeg1.mp4) and [here](https://github.com/sipsorcery/sipsorcery-mediafiles/raw/master/terminal/terminal_mjpeg2.mp4) (the second one's a bit spooky). ![terminal_mjpeg_test_pattern](https://user-images.githubusercontent.com/197660/83198189-9fd40680-a136-11ea-9f83-1cbaf2a88446.png) ![terminal_mjpeg_max](https://user-images.githubusercontent.com/197660/83198191-a2366080-a136-11ea-9f51-faa5568c8fd7.png) To test out the feature the steps are: - Set the `backgroundImage` to a file path with `;mjpeg:1234` appended. The `1234` represents a loopback port that the mjpeg receiving socket will listen on. For example: `"backgroundImage": "C:/Users/aaron/Pictures/gravatar_400x400.png;mjpeg:10010"` - Start a source stream to the listening port. For example: `ffmpeg -re -f lavfi -i testsrc=size=1280x720:rate=10 -vf "drawtext=fontfile=/Windows/Fonts/arial.ttf: expansion=normal: text='%{localtime}': r=25: x=1100: y=50: fontcolor=black" -vcodec mjpeg -pix_fmt yuvj420p -huffman 0 -f rtp rtp://127.0.0.1:10010` Known Issues: - Only one terminal control can listen on a port at a time so starting a second terminal with the same `backgroundImage` setting will not be able to create a listening socket. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
claunia added the pull-request label 2026-01-31 09:17:08 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#26615
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?