This pipeline runs on our agents, rather than OneBranch's, and doesn't
use all of the OneBranch machinery--which is only required for producing
a vpack that we check into Windows. Since Canary will never be a vpack,
we don't need to worry.
It runs at about twice the speed _and_ we control the build images!
This pull request also adds support for the "Terrapin Retrieval Tool,"
which will allow us to move away from having vcpkg contact remote
servers directly to download source code (and which may become mandatory
even in our OneBranch pipelines.)
Our last build failed because it tried to pass "10621.0" off as a
uint64. I didn't know it had to be a single number... so let's use the
3-component equivalent (which would have been 1.24.260303001)
Two main changes:
- the build was failing for ARM64 because the build agents run out of
memory downloading artifacts; this is a known issue that nobody cares to
fix (and it pertains to using the x64 emulated agent host on native
arm64 machines)
- deleting the PDBs shrinks the build output by 2500MB :P
- we had to switch to a new type of identity for publishing nuget
packages to our cross-org feed
I've verified that we get counts _and_ can even produce final PGO count
packages for both architectures!
This will allow us to publish vpacks without making the build fail
waiting for us to *merge* those vpacks into Windows. It also gives us
better control over when and where the vpack update gets merged.
Due to an unexpected decision on behalf of the Azure Artifacts folks,
the default view for a feed with upstream sources reports all packages,
even if they are not actually populated into the feed.
This results in (uncontrolled) 401 errors whenever a new package appears
upstream, because the feed tells our users and our build system that it
is available, but fails when the download actually begins because it is
not allowed to "write" the upstream version to the feed.
When we first transitioned to the `R1` network isolation
environment--which required us to _not_ contact
powershellgallery.com--we had to give up on the `AzureFileCopy` task.
Since then, the `AzurePowerShell` task has also become somewhat broken
while OneBranch fixed the issue that prevented us from using
`AzureFileCopy`.
In short, we now need to:
- Install the Azure PowerShell modules directly from our own feed (which
satisfies the network constraint)
- ... using `PSResourceGet` ...
- ... for Windows PowerShell 5.1 ...
- which is used by `AzureFileCopy` to later perform authentication.
- Various spelling fixes
- Refresh metadata (including dictionaries)
- Upgrade to v0.0.25
## Validation Steps Performed
- check-spelling has been automatically testing this repository for a
while now on a daily basis to ensure that it works fairly reliably:
https://github.com/check-spelling-sandbox/autotest-check-spelling/actions/workflows/microsoft-terminal-spelling2.yml
Specific in-code fixes:
- winget
- whereas
- tl;dr
- set up
- otherwise,
- more,
- macbook
- its
- invalid
- in order to
- if
- if the
- for this tab,...
- fall back
- course,
- cch
- aspect
- archaeologists
- an
- all at once
- a
- `...`
- ; otherwise,
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
This required removing connections during the build to `nuget.org` and
`powershellgallery.com`.
The NuGet Tool task was downloading nuget from `nuget.org`
unconditionally.
The `AzureFileCopy` task was downloading `Az.Accounts` from
`powershellgallery.com` unconditionally.
Both of these tasks have better options nowadays.
Tested and passed in OneBranch on 2025-04-01.
PackageES is deprecated by known scourge-on-earth OneBranch, and is now
the cause of some non-compliance.
I got permission from them to open-source it, so that's coming next.
For now, we can just depend on a package based on our code based on
theirs.
Tested and working for C++ (DLL, EXE), C#, NuGet and MSIX.
Same justification as #17749.
We will revert this when either OneBranch Custom Pools become
fit-for-purpose or they upgrade to VS 17.11. Or the heat death of the
universe.
Thanks to a string of compiler bugs, we had to use an older container
image that shipped with VS 17.9.
Unfortunately, that container image is falling further and further out
of date. The build agents don't cache it any longer, so they spend 30-45
minutes of every build pulling it from the registry.
With the changes to ConPTY in #17510 removing the need for til::bitmap,
we no longer need to work around the compiler bugs it exposed.
Furthermore, 17.10.6+ has a much more robust and presumably "working"
compiler.
`nuget restore` actually runs through MSBuild! However, #15855 added a
dependency from our project on a system-installed _or locally detected_
`vcpkg.targets` (or `.props`).
Our build runs `nuget restore` before finding or installing vcpkg, so
the rules in our project file would try to import vcpkg before it had
been found (or installed).
On build agents with vcpkg installed via the VS workload, this was fine:
we would import the one that came with VS and go on our merry way. On
build agents where it needs to be installed locally, it could not be
imported.
The fix in this PR is to install/bootstrap vcpkg before running nuget.
I tried to isolate the vcpkg rules to only run _in the absence of
nuget_, but that didn't work.
This pull request removes the following vendored open source, in favor
of getting it from vcpkg:
- CLI11 2.4
- jsoncpp 1.9
- fmt 7.1.3
- gsl 3.1 (not vendored, but submoduled--arguably worse!)
Now that Visual Studio 2022 includes a built-in workload for vcpkg, the
onboarding process is much smoother. Terminal should only require the
vcpkg workload.
I've added some build rules that detect vcpkg via VS and via the user's
environment before falling back to a location in the source tree. The CI
pipeline will fall back to installing and bootstrapping vcpkg in
dep/vcpkg if necessary.
Some OSS has not been (and will not be) migrated:
- wyhash: ours is included directly in til/hash
- pcg_random: we have a stripped down copy compared to vcpkg
- stb_rect: vcpkg only ships *all of STB*; ours is a stripped down copy
- chromium numerics: vcpkg does not ship Chromium, especially not this
tiny fraction of Chromium
- dynamic_bitset and libpopcnt: removing in #17510
- interval_tree: no vcpkg equivalent
To support the needs of the inbox Windows build, I've split up our vcpkg
manifest into dependencies for all projects and dependencies just for
Terminal. To support this, we now offer a `terminal` feature. The vcpkg
rules in `common.build.pre.props` are set up to turn it on, whereas the
build rules we eventually write for the OS will not be.
Most of the work is concentrated in `common.build.pre.props`.
We have to run in an older OneBranch Windows container image due to
compiler bugs.
This change prevents us from having to wait for the container image to
download for build legs that _aren't_ using the compiler.
This allows us to remove the dependency on the `Terminal.Internal`
repository.
I have also added some parameters to the build pipeline to ease testing.
This also updates the localization pipeline to check in translations for
the PDPs.
Right now, the primary source for PDPs is the Terminal.Internal
repository. They are submitted from there, and pulled back in as though
they were destined for the internal repo. We rename them on disk prior
to loc check-in to pretend they live in this repo.
Once I submit a change request to the Touchdown team to update the paths
in their backend, I will follow up with another pull request that
updates the remaining build steps to account for that.
This centralized all our ESRP calls in one file, which will make it
easier in the future when we are invariable required to change how we
call it again.
This required me to push a bunch more parameters through the build
pipeline, but it gave me the opportunity to define them as variables
that can be set at queue time.
This is required for us to move off Entra ID Application identity.
(cherry picked from commit 2e7c3fa313)
This was approved in #16957, so I will merge with one signoff.
Work is ongoing to remove individually-authenticated service accounts
from some pipelines. This moves us closer to that goal.
Tested in Nightly 2403.28002.
