starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-04 05:35:20 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

Why do we need idl files? #3019

New Issue
Closed
opened 2026-01-30 23:10:58 +00:00 by claunia · 10 comments
claunia commented 2026-01-30 23:10:58 +00:00
Owner
Copy Link

Originally created by @mcpiroman on GitHub (Jul 31, 2019).

Originally assigned to: @bitcrazed on GitHub.

It doesn't look as there is any IPC going on, so interfaces could be done as well in c++, right? It's quite a pain to work, especially rebuild, with idl.

Originally created by @mcpiroman on GitHub (Jul 31, 2019). Originally assigned to: @bitcrazed on GitHub. It doesn't look as there is any IPC going on, so interfaces could be done as well in c++, right? It's quite a pain to work, especially rebuild, with idl.
claunia added the Needs-TriageNeeds-Tag-FixIssue-Docs labels 2026-01-30 23:10:58 +00:00
claunia commented 2026-01-30 23:10:59 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jul 31, 2019):

That's just how c++/WinRT works. In some future, we imagine shipping the TerminalControl as a nuget package that anyone could use to embed in their application. In that hypothetical future, c++/WinRT is going to be incredibly helpful to us, for enabling us to easily create C# projections of our types. So someone could write a C# application that uses the terminal, without needing a crazy Pinvoke projection.

By making the boundaries between our libraries well-defined in idl, that means that it's extra easy for someone else to swap out an implementation with one written in another language. For example, maybe someone in their hypothetical terminal application wants to create their own connection type, that they've written in C#. By defining the interface in idl, it makes it possible for us to consume the code they authored easily.

Much of the XAML code is pretty heavily dependent up on working with c++winrt anyways, so we need idl files to author XAML-y types even if we're not doing any IPCs.

@zadjii-msft commented on GitHub (Jul 31, 2019): That's just how c++/WinRT works. In some future, we imagine shipping the TerminalControl as a nuget package that anyone could use to embed in their application. In that hypothetical future, c++/WinRT is going to be incredibly helpful to us, for enabling us to easily create C# projections of our types. So someone could write a C# application that uses the terminal, without needing a crazy Pinvoke projection. By making the boundaries between our libraries well-defined in idl, that means that it's extra easy for someone else to swap out an implementation with one written in another language. For example, maybe someone in their hypothetical terminal application wants to create their own connection type, that they've written in C#. By defining the interface in idl, it makes it possible for us to consume the code they authored easily. Much of the XAML code is pretty heavily dependent up on working with c++winrt anyways, so we need idl files to author XAML-y types even if we're not doing any IPCs.
claunia commented 2026-01-30 23:10:59 +00:00
Author
Owner
Copy Link

@mcpiroman commented on GitHub (Jul 31, 2019):

So, just to speed up developement, could we convert these idls that are not XAML related to c++? They'd be pure virutal classes (interfaces), with a note that they might become idl one day, when that's actualy needed. Maybe also enforce a rule that these files can only expose winrt types (e.g. hstring instead of std::string).
Working with idls is far more painful than with c++, but I don't know if it's the right way to fix that.

@mcpiroman commented on GitHub (Jul 31, 2019): So, just to speed up developement, could we convert these idls that are not XAML related to c++? They'd be pure virutal classes (interfaces), with a note that they might become idl one day, when that's actualy needed. Maybe also enforce a rule that these files can only expose winrt types (e.g. hstring instead of std::string). Working with idls is far more painful than with c++, but I don't know if it's the right way to fix that.
claunia commented 2026-01-30 23:10:59 +00:00
Author
Owner
Copy Link

@zadjii-msft commented on GitHub (Jul 31, 2019):

I don't know why we'd do that. Seems to me like it'd be more work to convert the classes to pure c++ types, then back to winrt types than the productivity lost due to dealing with idl files.

What in particular about working with idl's is so painful? I'm sure @Scottj1s, @LarryOsterman and @kennykerr would be super interested.

I'll admit that I only started working with c++winrt and idl's just about a year ago, and while I'm not good at it now, I'm certainly a lot better than I used to be. It's one of those tech stacks that as you learn it, it'll be just as easy as working with pure c++.

@zadjii-msft commented on GitHub (Jul 31, 2019): I don't know why we'd do that. Seems to me like it'd be more work to convert the classes to pure c++ types, then _back_ to winrt types than the productivity lost due to dealing with idl files. What in particular about working with idl's is so painful? I'm sure @Scottj1s, @LarryOsterman and @kennykerr would be super interested. I'll admit that I only started working with c++winrt and idl's just about a year ago, and while I'm not good at it now, I'm certainly a lot better than I used to be. It's one of those tech stacks that as you learn it, it'll be just as easy as working with pure c++.
claunia commented 2026-01-30 23:10:59 +00:00
Author
Owner
Copy Link

@mcpiroman commented on GitHub (Jul 31, 2019):

Yea, syntax-wise they're very nice and clean. And they're based on C#, that I like a lot.

I ment the integration and build process:

  • When the file is changed (physically changed, like added a space), I have to delete some folders, usually Generated Files and obj, but this doesn't work all the time. And if it does, the build is much longer than it would be for c++. Alternatively I can rebuild whole solution.
  • Intelisense works only when these files are changed, but not always. Sometimes it happend that certain names were highlighted red even after rebuild for a while.
  • If there is an error across c++ and idl, it's hardly possible to deduct that error from the error message(s) the VS gives (not that they're particuarly clean in c++). They go into autogenerated files and something like base.h.
  • Debugging across the idl <-> c++ boundary is also not so fun, as you have go across few frames of autogenerated thunk-mess code. Same goes to stacktrace.

But since you seem not to have problems with this, I might be missing something.

@mcpiroman commented on GitHub (Jul 31, 2019): Yea, syntax-wise they're very nice and clean. And they're based on C#, that I like a lot. I ment the integration and build process: - When the file is changed (physically changed, like added a space), I have to delete some folders, usually `Generated Files` and `obj`, but this doesn't work all the time. And if it does, the build is much longer than it would be for c++. Alternatively I can rebuild whole solution. - Intelisense works only when these files are changed, but not always. Sometimes it happend that certain names were highlighted red even after rebuild for a while. - If there is an error across c++ and idl, it's hardly possible to deduct that error from the error message(s) the VS gives (not that they're particuarly clean in c++). They go into autogenerated files and something like `base.h`. - Debugging across the idl <-> c++ boundary is also not so fun, as you have go across few frames of autogenerated thunk-mess code. Same goes to stacktrace. But since you seem not to have problems with this, I might be missing something.
claunia commented 2026-01-30 23:11:00 +00:00
Author
Owner
Copy Link

@LarryOsterman commented on GitHub (Jul 31, 2019):

FIrst off: Thank you for the comment about the syntax - I worked hard to make it clean and straightforward :).

Mike also left out one really important aspect of IDL (well, he touched on it, but I think it needs amplifying): C++ doesn't define a mechanism for stable binary contracts across binaries.

That means that if your component is built of multiple binaries (DLLs (or shared objects if you're a *nix person), you need to define an interoperability pattern for your DLLs. It turns out that this is true for ALL the interfaces into and out of a DLL: The only native language that has a standardized pattern is C, and C doesn't support modern constructs like C++ classes, which is a huge implementation barrier.

And before you ask: Nope, C++ doesn't have any cross binary interoperable construct. It's true that by convention, most of the existing C++ compilers use similar calling conventions, but it's not guaranteed by the standard - it's just because the compiler vendors decided to choose the same binary contract. This is especially true when you deal with C++ constructs like exceptions - there is no stable binary contract for exceptions, even across different versions of the same compiler. So whenever an exception is thrown within a DLL, it must be caught within the DLL and turned into some construct which is stable across binaries (like an HRESULT or status code). This is also why marking a function "noexcept" causes the C++ runtime to abort() your process if an exception crosses a DLL boundary.

By careful design, the binary contract defined by IDL's is based on C structures containing pointers to functions and that binary contract IS stable over both time and across compiler versions (because the C standard lays out rules of how structs are laid out and the calling convention of the processor define how parameters to functions are laid out. And because it's C, tricky constructs like "this" aren't an issue).

So if your component has ANY DLLs, it becomes incredibly helpful to define your types in terms of COM/WinRT classes - that allows you to keep the C++ features most developers desire and still have binary interoperability.

And of course, WinRT enables cross language projections (as Mike mentioned above). That means that you can write console apps in C# as well as C++.

Ok, enough ranting about stability of binary contracts (you can tell it's a subject I care about a lot :)). Now to your other issues...

It appears that many/most of your issues are related to the fact that VS doesn't really treat IDL (and the C++/WinRT authoring experience) as a first class citizen, especially as compared to C++ or C#. I know that the C++/WinRT team has been working tirelessly with the VS team to improve the authoring experience, @Scottj1s is the primary author of that experience and may have more information.

@LarryOsterman commented on GitHub (Jul 31, 2019): FIrst off: Thank you for the comment about the syntax - I worked hard to make it clean and straightforward :). Mike also left out one really important aspect of IDL (well, he touched on it, but I think it needs amplifying): C++ doesn't define a mechanism for stable binary contracts across binaries. That means that if your component is built of multiple binaries (DLLs (or shared objects if you're a *nix person), you need to define an interoperability pattern for your DLLs. It turns out that this is true for ALL the interfaces into and out of a DLL: The only native language that has a standardized pattern is C, and C doesn't support modern constructs like C++ classes, which is a huge implementation barrier. And before you ask: Nope, C++ doesn't have any cross binary interoperable construct. It's true that by convention, most of the existing C++ compilers use similar calling conventions, but it's not guaranteed by the standard - it's just because the compiler vendors decided to choose the same binary contract. This is **especially** true when you deal with C++ constructs like exceptions - there is no stable binary contract for exceptions, even across different versions of the same compiler. So whenever an exception is thrown within a DLL, it *must* be caught within the DLL and turned into some construct which is stable across binaries (like an HRESULT or status code). This is also why marking a function "noexcept" causes the C++ runtime to abort() your process if an exception crosses a DLL boundary. By careful design, the binary contract defined by IDL's is based on C structures containing pointers to functions and that binary contract IS stable over both time and across compiler versions (because the C standard lays out rules of how structs are laid out and the calling convention of the processor define how parameters to functions are laid out. And because it's C, tricky constructs like "this" aren't an issue). So if your component has ANY DLLs, it becomes incredibly helpful to define your types in terms of COM/WinRT classes - that allows you to keep the C++ features most developers desire and still have binary interoperability. And of course, WinRT enables cross language projections (as Mike mentioned above). That means that you can write console apps in C# as well as C++. Ok, enough ranting about stability of binary contracts (you can tell it's a subject I care about a lot :)). Now to your other issues... It appears that many/most of your issues are related to the fact that VS doesn't really treat IDL (and the C++/WinRT authoring experience) as a first class citizen, especially as compared to C++ or C#. I know that the C++/WinRT team has been working tirelessly with the VS team to improve the authoring experience, @Scottj1s is the primary author of that experience and may have more information.
claunia commented 2026-01-30 23:11:00 +00:00
Author
Owner
Copy Link

@mcpiroman commented on GitHub (Jul 31, 2019):

So, do we need dlls, now?

(And isn't it that c++ doesn't define abi only beyond the things shared with C?)

@mcpiroman commented on GitHub (Jul 31, 2019): So, do we need dlls, now? (And isn't it that c++ doesn't define abi only beyond the things shared with C?)
claunia commented 2026-01-30 23:11:00 +00:00
Author
Owner
Copy Link

@LarryOsterman commented on GitHub (Jul 31, 2019):

"Do we need DLLs now"? That depends on how much pain and suffering you want to inflict on your customers. Let me tell you a little story about what happens without DLLs. It's ancient history, so I think I can tell it now (there are countless other examples of this, especially in the *nix/OSS world btw).

About a decade ago, Microsoft was informed of a rather nasty security vulnerability in some code. Normally this isn't a huge deal. We recompile the offending binary with the fix for the security vulnerability, drop it onto Windows Update and we're good to go. But this vulnerability was different. It was in a template library which Microsoft shipped for more than a decade before anyone noticed the vulnerability.

And that template library was embedded in literally hundreds of thousands of customer applications. ALL of which had to be either patched or blacklisted before we could be confident that the vulnerability was fixed. Just finding and fixing the vulnerable Microsoft applications took years and was a huge effort. Fortunately the finder was patient and understood the magnitude of the problem and we were able to apply appropriate mitigations. But it was a massive undertaking.

shared objects provide an extremely convenient servicing boundary - if there's a bug/vulnerability in a DLL/so, you can recompile the DLL and push it out via your platform's servicing mechanisms. Now they're not perfect - last weeks hubub about VLC is a great example of how not correctly servicing bugs can wreak havoc on customers. And if you want a really fun time, look up libpng or openssl security vulnerability to see what happens when a security vulnerability is found in a component distributed as a lib embedded in a binary (it's not pretty because every component which included the vulnerable code needs to be patched).

So sure, the terminal team could have decided to ship their code simply as OSS and compile their code directly into customer applications (there's a tricky bit here because they'd have to figure out how to get their code compiling with every possible customer compiler and build environment choice, but it's solvable - OSS projects have done this for years). But for servicing reasons alone, shared objects are a huge win for customer reliability.

And you're right - C++ does define a standardized binary contract. But it happens to only cover the constructs that C++ shares with C. So you're still limited to C constructs if you want interoperability.

Btw, the interoperability story is even worse if you distribute your code via a lib. There is no guarantee of interoperabilty between different versions of the same compiler, much less between different compilers (compiler authors attempt to maintain binary compatibility across time, but they DO break their internal contracts on occasion (simple example: changing name decoration algorithms breaks interoperability, even if it doesn't break the binary contract).

@LarryOsterman commented on GitHub (Jul 31, 2019): "Do we need DLLs now"? That depends on how much pain and suffering you want to inflict on your customers. Let me tell you a little story about what happens without DLLs. It's ancient history, so I think I can tell it now (there are **countless** other examples of this, especially in the *nix/OSS world btw). About a decade ago, Microsoft was informed of a rather nasty security vulnerability in some code. Normally this isn't a huge deal. We recompile the offending binary with the fix for the security vulnerability, drop it onto Windows Update and we're good to go. But this vulnerability was different. It was in a template library which Microsoft shipped for more than a decade before anyone noticed the vulnerability. And that template library was embedded in literally hundreds of thousands of customer applications. *ALL* of which had to be either patched or blacklisted before we could be confident that the vulnerability was fixed. Just finding and fixing the vulnerable Microsoft applications took years and was a huge effort. Fortunately the finder was patient and understood the magnitude of the problem and we were able to apply appropriate mitigations. But it was a massive undertaking. shared objects provide an extremely convenient servicing boundary - if there's a bug/vulnerability in a DLL/so, you can recompile the DLL and push it out via your platform's servicing mechanisms. Now they're not perfect - last weeks hubub about VLC is a great example of how not correctly servicing bugs can wreak havoc on customers. And if you want a really fun time, look up libpng or openssl security vulnerability to see what happens when a security vulnerability is found in a component distributed as a lib embedded in a binary (it's not pretty because every component which included the vulnerable code needs to be patched). So sure, the terminal team could have decided to ship their code simply as OSS and compile their code directly into customer applications (there's a tricky bit here because they'd have to figure out how to get their code compiling with every possible customer compiler and build environment choice, but it's solvable - OSS projects have done this for years). But for servicing reasons alone, shared objects are a huge win for customer reliability. And you're right - C++ does define a standardized binary contract. But it happens to only cover the constructs that C++ shares with C. So you're still limited to C constructs if you want interoperability. Btw, the interoperability story is even worse if you distribute your code via a lib. There is no guarantee of interoperabilty between different versions of the same compiler, much less between different compilers (compiler authors attempt to maintain binary compatibility across time, but they DO break their internal contracts on occasion (simple example: changing name decoration algorithms breaks interoperability, even if it doesn't break the binary contract).
claunia commented 2026-01-30 23:11:00 +00:00
Author
Owner
Copy Link

@mcpiroman commented on GitHub (Jul 31, 2019):

Yes, of course, I know the story about shared vs static vs code libraries (I guess).

But my point is that the terminal is now in alpha stage. If someone want's to try it he is best of building it himself, and eventually, there are preview releases on windows store (and I'm not sure which way to acquire it is actually more challenging :P). So just for the time no one outside the terminal's development team is using this code (and that's probably quite a long time) or using different compiler versions or even shipping the application with a proclamation that it won't break anything, we could theoretically move things around to allow easier and quicker development, especially in the prototyping phase where interfaces like idl often changes, which goes to things above. That's why I said now.

Also I'm glad to hear the VS team is working on better integration with idl's.

@mcpiroman commented on GitHub (Jul 31, 2019): Yes, of course, I know the story about shared vs static vs code libraries (I guess). But my point is that the terminal is now in alpha stage. If someone want's to try it he is best of building it himself, and eventually, there are preview releases on windows store (and I'm not sure which way to acquire it is actually more challenging :P). So just for the time no one outside the terminal's development team is using this code (and that's probably quite a long time) or using different compiler versions or even shipping the application with a proclamation that it won't break anything, we could theoretically move things around to allow easier and quicker development, especially in the prototyping phase where interfaces like idl often changes, which goes to things above. That's why I said `now`. Also I'm glad to hear the VS team is working on better integration with idl's.
claunia commented 2026-01-30 23:11:01 +00:00
Author
Owner
Copy Link

@LarryOsterman commented on GitHub (Jul 31, 2019):

I'm not on the Terminal team (heck, I'm no longer in Windows for that matter), so I cannot speak for them and their architecture. However IMHO it's far better to build your design up-front to meet the long-term goals of the feature rather than add those architectural changes in later.

And I'm not sure it's a correct statement that "the VS team is working on better integration with IDLs". What I said was that the C++/WinRT team is working with the VS team to improve the integration. Not quite the same thing.

@LarryOsterman commented on GitHub (Jul 31, 2019): I'm not on the Terminal team (heck, I'm no longer in Windows for that matter), so I cannot speak for them and their architecture. However IMHO it's far better to build your design up-front to meet the long-term goals of the feature rather than add those architectural changes in later. And I'm not sure it's a correct statement that "the VS team is working on better integration with IDLs". What I said was that the C++/WinRT team is working with the VS team to improve the integration. Not quite the same thing.
claunia commented 2026-01-30 23:11:01 +00:00
Author
Owner
Copy Link

@bitcrazed commented on GitHub (Jul 31, 2019):

@mcpiroman Jumping in with a few thoughts re. some of the points you raise above:

"If someone want's to try it [they are] best off building it [themselves]"

Not true - if someone is interested in just using the Terminal, it's likely best that they install the preview builds from the Store. This way they'll automatically receive updates every time we push a new build. Note: We'll be pushing builds more often and more regularly - we'll post an update on this news shortly.

"or using different compiler versions"

The Terminal is not a stand-alone project. It shares many of its core components with Windows' Console and the code-base also includes the command-line plumbing that's built into Windows itself. Thus, we don't have the luxury of arbitrarily choosing which compiler to use. Nor would we even if we could - doing so would likely introduce many unknown and undiscovered issues that are hard to find and likely harder to fix.

Alas, the C++ standards deliberately avoid defining the mechanics of ABI/calling-convention/exception-passing/etc. While this has allowed great flexibility for individual compiler/tooling creators/vendors, it has resulted in many inconsistencies and incompatibilities between different C++ implementations. And because of the latter, we end up with IDL being required to describe classes and types in a manner that can be serialized into C in order to provide some amount of smooth projection/interop between C++ implementations, and other platforms, languages, and tools like .NETetc.

Until there's a universally adopted standard for a C++ runtime, we'll have to use IDL to ensure that we can interop with C++ from other languages and platforms, and using C++ code built using other toolchains.

Closing this issue for now as there's no further action to take here.

@bitcrazed commented on GitHub (Jul 31, 2019): @mcpiroman Jumping in with a few thoughts re. some of the points you raise above: > "If someone want's to try it [they are] best off building it [themselves]" Not true - if someone is interested in just _using_ the Terminal, it's likely best that they [install the preview builds from the Store](https://github.com/microsoft/terminal#microsoft-store). This way they'll automatically receive updates every time we push a new build. Note: We'll be pushing builds more often and more regularly - we'll post an update on this news shortly. > "or using different compiler versions" The Terminal is not a stand-alone project. It shares many of its core components with Windows' Console and the code-base also includes the command-line plumbing that's built into Windows itself. Thus, we don't have the luxury of arbitrarily choosing which compiler to use. Nor would we even if we could - doing so would likely introduce many unknown and undiscovered issues that are hard to find and likely harder to fix. Alas, the C++ standards deliberately avoid defining the mechanics of ABI/calling-convention/exception-passing/etc. While this has allowed great flexibility for individual compiler/tooling creators/vendors, it has resulted in many inconsistencies and incompatibilities between different C++ implementations. And because of the latter, we end up with IDL being required to describe classes and types in a manner that can be serialized into C in order to provide some amount of smooth projection/interop between C++ implementations, and other platforms, languages, and tools like .NETetc. Until there's a universally adopted standard for a C++ runtime, we'll have to use IDL to ensure that we can interop with C++ from other languages and platforms, and using C++ code built using other toolchains. Closing this issue for now as there's no further action to take here.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label Issue-Docs Needs-Tag-Fix Needs-Triage
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#3019
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?