wap packaging: .xbf and .pri generation is not working for us #2549

Closed
opened 2026-01-30 22:58:06 +00:00 by claunia · 13 comments
Owner

Originally created by @DHowett-MSFT on GitHub (Jul 4, 2019).

Originally assigned to: @DHowett-MSFT on GitHub.

The WAP packaging project, CascadiaPackage, consumes most of its important dependencies transitively. It depends on WindowsTerminal and Host.EXE.

By default, wapproj puts all files collected from a project into a subdirectory named after that project, even if those files were originally from a nested dependency. This includes the .exes.

We've suppressed that behavior so that WindowsTerminal and conhost can live in the appx package root next to their DLLs.

We must also do this so that the .winmd -> AppX Manifest generator works right -- if we don't, it will insert references like this:

<InProcessServer>
  <Path>Whatever.dll</Path>
</InProcessServer>

when our directory structure looks like this:

WindowsTerminal/Whatever.dll
WindowsTerminal/Whatever.xbf

This will fail validation (and activation)! This cannot be fixed, so we stomp the directory for DLL and EXE files.

eae920e5f9/src/cascadia/CascadiaPackage/CascadiaPackage.wapproj (L300-L310)

Well, we're also stomping the directory for XBF files. Therefore, MinMaxCloseControl.xbf and App.xbf end up in the appx root.

This works, because we've overridden their resource locator paths:

eae920e5f9/src/cascadia/TerminalApp/MinMaxCloseControl.cpp (L17-L18)

That works great when deployed from Visual Studio. However, when the project is deployed by MSBuild (like in our CI), those loose XBF files are missing. If you look in one of the PRI files, it says that the path is TerminalApp/MinMaxCloseControl.xbf.

Well, that never existed on disk (see above: wapproj would put it in WindowsTerminal/MinMaxCloseControl.xbf because it was a transitive dependency.)

It gets worse. That path would actually work because the XBF file has been merged into the PRI file(!). However, the PRI files from wapproj transitive dependencies do not get merged into the global resources.pri.

If the transitive dependency PRI files can be merged into resources.pri, we can skip overriding the resource loader and remove the loose XBF files from our package.

/cc @metathinker, who originally reported a crash from the CI builds.

Originally created by @DHowett-MSFT on GitHub (Jul 4, 2019). Originally assigned to: @DHowett-MSFT on GitHub. The WAP packaging project, CascadiaPackage, consumes most of its important dependencies transitively. It depends on WindowsTerminal and Host.EXE. By default, _wapproj_ puts all files collected from a project into a subdirectory _named after_ that project, even if those files were originally from a nested dependency. This includes the `.exe`s. We've suppressed that behavior so that `WindowsTerminal` and `conhost` can live in the appx package root next to their DLLs. **We must also do this** so that the `.winmd` -> AppX Manifest generator works right -- if we don't, it will insert references like this: ```xml <InProcessServer> <Path>Whatever.dll</Path> </InProcessServer> ``` when our directory structure looks like this: ``` WindowsTerminal/Whatever.dll WindowsTerminal/Whatever.xbf ``` This will fail validation (and activation)! This cannot be fixed, so we stomp the directory for DLL and EXE files. https://github.com/microsoft/terminal/blob/eae920e5f931d7efd33cc3e1bfb12f8c683b001b/src/cascadia/CascadiaPackage/CascadiaPackage.wapproj#L300-L310 Well, we're also stomping the directory for XBF files. Therefore, `MinMaxCloseControl.xbf` and `App.xbf` end up in the appx root. This works, because we've overridden their resource locator paths: https://github.com/microsoft/terminal/blob/eae920e5f931d7efd33cc3e1bfb12f8c683b001b/src/cascadia/TerminalApp/MinMaxCloseControl.cpp#L17-L18 That works great when deployed from Visual Studio. However, _when the project is deployed by MSBuild_ (like in our CI), those loose XBF files are missing. If you look in one of the _PRI_ files, it says that the path is `TerminalApp/MinMaxCloseControl.xbf`. Well, **that never existed on disk** (see above: _wapproj_ would put it in `WindowsTerminal/MinMaxCloseControl.xbf` because it was a transitive dependency.) It gets worse. That path would _actually work_ because the XBF file has been merged _into_ the PRI file(!). However, the PRI files from _wapproj_ transitive dependencies _do not get merged into the global `resources.pri`_. If the transitive dependency PRI files can be merged into `resources.pri`, we can skip overriding the resource loader _and_ remove the loose XBF files from our package. /cc @metathinker, who originally reported a crash from the CI builds.
Author
Owner

@DHowett-MSFT commented on GitHub (Jul 4, 2019):

Oh, also, this results in a crash when you use builds from Azure DevOps or MSBuild. Yeah.

@DHowett-MSFT commented on GitHub (Jul 4, 2019): Oh, also, this results in a crash when you use builds from Azure DevOps or MSBuild. Yeah.
Author
Owner

@metathinker commented on GitHub (Jul 4, 2019):

Previous (and occasionally silly) discussion as part of another issue thread: https://github.com/microsoft/terminal/issues/1364#issuecomment-508292860

As Dustin notes above, .appx packages and loose-file app package layouts with this defect will cause Terminal to crash on startup; the precise nature of the crash is discussed in the other thread.

@metathinker commented on GitHub (Jul 4, 2019): Previous (and occasionally silly) discussion as part of another issue thread: https://github.com/microsoft/terminal/issues/1364#issuecomment-508292860 As Dustin notes above, .appx packages and loose-file app package layouts with this defect will cause Terminal to crash on startup; the precise nature of the crash is discussed in the other thread.
Author
Owner

@DHowett-MSFT commented on GitHub (Jul 4, 2019):

I have a terrible idea. I'm going to use my holiday and try to replace the WAP packaging project with a native UWP project. Instead of going from EXE to Centennial (which is apparently very hard), why not go from UWP to Centennial (which will probably expose hundreds of other issues, but solve this set of a hundred issues 😁)

@DHowett-MSFT commented on GitHub (Jul 4, 2019): I have a terrible idea. I'm going to use my holiday and try to replace the WAP packaging project with a _native UWP project_. Instead of going from EXE to Centennial (which is apparently very hard), why not go from UWP to Centennial (which will probably expose hundreds of other issues, but solve this set of a hundred issues 😁)
Author
Owner

@DHowett-MSFT commented on GitHub (Jul 4, 2019):

Everything is terrible! 😄

feature WAP UAP
transitive dependencies ❌
(this issue)
✔️
PRI merging ❌
(this issue)
✔️
dependency resources ❌
(this issue)
✔️
simpler project structure ❌
(one extra project)
✔️
isn't a weird .NET project type ❌
(this makes debugging annoying)
✔️
partially works today ✔️ ❌
(new uncharted unknown)
depends on VCLibs.Desktop ✔️ ❌
(this seems hard)
intended for this purpose ✔️ ❌
(throws a warning)
can be debugged
(with some hoops)
✔️ ❌
(application thinks it isn't deployed)
Project not selected to build for this solution configuration, even after it builds.
@DHowett-MSFT commented on GitHub (Jul 4, 2019): Everything is terrible! :smile: |feature|WAP|UAP| |-|:-:|:-:| |transitive dependencies|❌<br>(this issue)|✔️| |PRI merging|❌<br>(this issue)|✔️| |dependency resources|❌<br>(this issue)|✔️| |simpler project structure|❌<br>(one extra project)|✔️| |isn't a weird .NET project type|❌<br>(this makes debugging annoying)|✔️| |partially works today|✔️|❌<br>(new uncharted unknown)| |depends on `VCLibs.Desktop`|✔️|❌<br>(this seems hard)| |intended for this purpose|✔️|❌<br>(throws a warning)| |can be debugged<br>(with some hoops)|✔️|❌<br>(application thinks it isn't deployed)<br>_Project not selected to build for this solution configuration_, even after it _builds_. |
Author
Owner

@mdtauk commented on GitHub (Jul 4, 2019):

WinUI and XAML Desktop apps - may change all this. I hope you are sharing your findings and thoughts with that team.

@mdtauk commented on GitHub (Jul 4, 2019): WinUI and XAML Desktop apps - may change all this. I hope you are sharing your findings and thoughts with that team.
Author
Owner

@DHowett-MSFT commented on GitHub (Jul 4, 2019):

@mdtauk not only are we sharing our findings and thoughts with that team, we report to the same engineering manager as that team. 😄
Unfortunately, that team doesn't own the packaging rules shipped with Visual Studio.

@DHowett-MSFT commented on GitHub (Jul 4, 2019): @mdtauk not only are we sharing our findings and thoughts with that team, we report to the same engineering manager as that team. :smile: Unfortunately, that team doesn't own the packaging rules shipped with Visual Studio.
Author
Owner

@mdtauk commented on GitHub (Jul 4, 2019):

@DHowett-MSFT I can only imagine that the VS team, would be developing new compilation and packaging processes, to match the requirements of the frameworks which are to come

@mdtauk commented on GitHub (Jul 4, 2019): @DHowett-MSFT I can only imagine that the VS team, would be developing new compilation and packaging processes, to match the requirements of the frameworks which are to come
Author
Owner

@zadjii-msft commented on GitHub (Jul 4, 2019):

IIRC the wsl distro launchers don't use a wapproj, maybe we can use their structure


From: Martin Anderson notifications@github.com
Sent: Thursday, July 4, 2019 5:08:28 PM
To: microsoft/terminal
Cc: Subscribed
Subject: Re: [microsoft/terminal] wap packaging: .xbf and .pri generation is a house of horrors (#1816)

@DHowett-MSFThttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDHowett-MSFT&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105194204&sdata=f8MCJ4c9S4VwRbtg8gQ%2F%2BLL2IEhkzahq2CUSc51KMNU%3D&reserved=0 I can only imagine that the VS team, would be developing new compilation and packaging processes, to match the requirements of the frameworks which are to come

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fterminal%2Fissues%2F1816%3Femail_source%3Dnotifications%26email_token%3DAEMBTVR2KAKDEOJZGEKJ67DP5ZYFZA5CNFSM4H5QYKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZIF45I%23issuecomment-508583541&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105204192&sdata=o22iE5vxqwP64M2d6wXqvUjTmawoU2BJ4eVRrYfxkRs%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMBTVXXP5UH4UJM2JSPWYLP5ZYFZANCNFSM4H5QYKJQ&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105204192&sdata=tRktrG6IpsrCFMqbYUu0lVGudegtVzSjW3jQTd1fOok%3D&reserved=0.

@zadjii-msft commented on GitHub (Jul 4, 2019): IIRC the wsl distro launchers don't use a wapproj, maybe we can use their structure ________________________________ From: Martin Anderson <notifications@github.com> Sent: Thursday, July 4, 2019 5:08:28 PM To: microsoft/terminal Cc: Subscribed Subject: Re: [microsoft/terminal] wap packaging: .xbf and .pri generation is a house of horrors (#1816) @DHowett-MSFT<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDHowett-MSFT&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105194204&sdata=f8MCJ4c9S4VwRbtg8gQ%2F%2BLL2IEhkzahq2CUSc51KMNU%3D&reserved=0> I can only imagine that the VS team, would be developing new compilation and packaging processes, to match the requirements of the frameworks which are to come — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fterminal%2Fissues%2F1816%3Femail_source%3Dnotifications%26email_token%3DAEMBTVR2KAKDEOJZGEKJ67DP5ZYFZA5CNFSM4H5QYKJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZIF45I%23issuecomment-508583541&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105204192&sdata=o22iE5vxqwP64M2d6wXqvUjTmawoU2BJ4eVRrYfxkRs%3D&reserved=0>, or mute the thread<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEMBTVXXP5UH4UJM2JSPWYLP5ZYFZANCNFSM4H5QYKJQ&data=02%7C01%7CMike.Griese%40microsoft.com%7C6253ebac2a284ad2f84408d700cc24f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636978749105204192&sdata=tRktrG6IpsrCFMqbYUu0lVGudegtVzSjW3jQTd1fOok%3D&reserved=0>.
Author
Owner

@DHowett-MSFT commented on GitHub (Jul 4, 2019):

feature WAP
If you add the dependencies directly to the wap project,
they fail with new and interesting linker errors
✔️

Yeah, if you add TerminalConnection as a direct dependency to wapproj, it has to re-link it during deployment, and it refuses to link in ConTypes.lib.

@DHowett-MSFT commented on GitHub (Jul 4, 2019): |feature|WAP| |-|-| |If you add the dependencies directly to the wap project,<br>**they fail with new and interesting _linker_ errors**|✔️| Yeah, if you add TerminalConnection as a direct dependency to _wapproj_, it has to re-link it during deployment, and it refuses to link in ConTypes.lib.
Author
Owner

@DHowett-MSFT commented on GitHub (Jul 12, 2019):

/cc @ocalvo for info

@DHowett-MSFT commented on GitHub (Jul 12, 2019): /cc @ocalvo for info
Author
Owner

@ocalvo commented on GitHub (Jul 12, 2019):

/cc @azchohfi for info

@ocalvo commented on GitHub (Jul 12, 2019): /cc @azchohfi for info
Author
Owner

@metathinker commented on GitHub (Jul 18, 2019):

I can't reopen issues after they have been closed, so paging @DHowett-MSFT:

It looks like the issue wasn't fixed or was reintroduced - I built the current master commit 988fe0ba60 and installed the loose-file AppX layout. The crash on startup is back, and occurs at the same place. This time, the resource that failed to load is: ms-appx:///TerminalApp/TerminalPage.xaml

@metathinker commented on GitHub (Jul 18, 2019): I can't reopen issues after they have been closed, so paging @DHowett-MSFT: It looks like the issue wasn't fixed or was reintroduced - I built the current `master` commit 988fe0ba608a3c99f42f2245ce680e535ce6e97a and installed the loose-file AppX layout. The crash on startup is back, and occurs at the same place. This time, the resource that failed to load is: `ms-appx:///TerminalApp/TerminalPage.xaml`
Author
Owner

@DHowett-MSFT commented on GitHub (Jul 18, 2019):

Yuuuup! #2018 to fix :)

@DHowett-MSFT commented on GitHub (Jul 18, 2019): Yuuuup! #2018 to fix :)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#2549