starred/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2026-02-03 21:25:34 +00:00
Code Issues 19.0k Packages Projects Releases 20 Wiki Activity

[PR #15808] Rewrite the entire Azure DevOps build system #30742

New Issue
Open
opened 2026-01-31 09:42:41 +00:00 by claunia · 0 comments
claunia commented 2026-01-31 09:42:41 +00:00
Owner
Copy Link

Original Pull Request: https://github.com/microsoft/terminal/pull/15808

State: closed
Merged: Yes

This pull request rewrites the entire Azure DevOps build system.

The guiding principles behind this rewrite are:

  • No pipeline definitions should contain steps (or tasks) directly.
  • All jobs should be in template files.
  • Any set of steps that is reused across multiple jobs must be in
    template files.
  • All artifact names can be customized (via a property called
    artifactStem on all templates that produce or consume artifacts).
  • No compilation happens outside of the "Build" phase, to consolidate
    the production and indexing of PDBs.
  • Building the project produces a bin directory. That bin
    directory is therefore the primary currency of the build. Jobs will
    either produce or consume bin if they want to do anything with the
    build outputs.
  • All step and job templates are named with step or job first,
    which disambiguates them in the templates directory.
  • Most jobs can be run on different pools, so that we can put
    expensive jobs on expensive build agents and cheap jobs on cheap
    build agents. Some jobs handle pool selection on their own, however.

Our original build pipelines used the VSBuild task all over the
place. This resulted in Terminal being built in myriad ways, different
for every pipeline. There was an attempt at standardization early on,
where ci.yml consumed jobs and steps templates... but when
release.yml was added, all of that went out the window.

The new pipelines are consistent and focus on a small, well-defined set
of jobs:

  • job-build-project
    • This is the big one!
    • Takes a list of build configurations and platforms.
    • Produces an artifact named build-PLATFORM-CONFIG for the entire
      matrix of possibilities.
    • Optionally signs the output and produces a bill of materials.
    • Admittedly has a lot going on.
  • job-build-package-wpf
    • Takes a list of build configurations and platforms.
    • Consumes the build- artifact for every config/platform
      possibility, plus one for "Any CPU" (hardcoded; this is where the
      .NET code builds)
    • Produces one wpf-nupkg-CONFIG for each configuration, merging
      all platforms.
    • Optionally signs the output and produces a bill of materials.
  • job-merge-msix-into-bundle
    • Takes a list of build configurations and platforms.
    • Consumes the build- artifact for every config/platform
    • Produces one appxbundle-CONFIG for each configuration, merging
      all platforms for that config into one msixbundle.
    • Optionally signs the output and produces a bill of materials.
  • job-package-conpty
    • Takes a list of build configurations and platforms.
    • Consumes the build- artifact for every config/platform
    • Produces one conpty-nupkg-CONFIG for each configuration, merging
      all platforms.
    • Optionally signs the output and produces a bill of materials.
  • job-test-project
    • Takes one build config and one platform.
    • Consumes build-PLATFORM-CONFIG
    • Selects its own pools (hardcoded) because it knows about
      architectures and must choose the right agent arch.
    • Runs tests (directly on the build agent).
  • job-run-pgo-tests
    • Just like the above, but runs tests where IsPgo is true
    • Collects all of the PGO counts and publishes a pgc-intermediates
      artifact for that platform and configuration.
  • job-pgo-merge-pgd
    • Takes one build config and multiple platforms.
    • Consumes build-$platform-CONFIG for each platform.
    • Consumes pgc-intermediates-$platform-CONFIG for each platform.
    • Merges the pgc files into pgd files
    • Produces a new pgd- artifact.
  • job-pgo-build-nuget-and-publish
    • Consumes the pgd- artifact from above.
    • Packs it into a nupkg and publishes it.
  • job-submit-windows-vpack
    • Only expected to run against Release.
    • Consumes the appxbundle-CONFIG artifact.
    • Publishes it to a vpack for Windows to consume.
  • job-check-code-format
    • Does not use artifacts. Runs clang-format.
  • job-index-github-codenav
    • Does not use artifacts.

Fuzz submission is broken due to changes in the onefuzz client.

I have removed the compliance and security build because it is no longer
supported.

Finally, this pull request has some additional benefits:

  • I've expanded the PGO build phase to cover ARM64!
  • We can remove everything Helix-related except the WTT parser
    • We no longer depend on Helix submission or Helix pools
  • The WPF control's inner DLLs are now codesigned (#15404)
  • Symbols for the WPF control, both .NET and C++, are published
    alongside all other symbols.
  • The files we submit to ESRP for signing are batched up into a single
    step1

Closes #11874
Closes #11974
Closes #15404

  1. This will have to change if we want to sign the individual
    per-architecture .appx files before bundling so that they can be
    directly installed. ↩︎

**Original Pull Request:** https://github.com/microsoft/terminal/pull/15808 **State:** closed **Merged:** Yes --- This pull request rewrites the entire Azure DevOps build system. The guiding principles behind this rewrite are: - No pipeline definitions should contain steps (or tasks) directly. - All jobs should be in template files. - Any set of steps that is reused across multiple jobs must be in template files. - All artifact names can be customized (via a property called `artifactStem` on all templates that produce or consume artifacts). - No compilation happens outside of the "Build" phase, to consolidate the production and indexing of PDBs. - **Building the project produces a `bin` directory.** That `bin` directory is therefore the primary currency of the build. Jobs will either produce or consume `bin` if they want to do anything with the build outputs. - All step and job templates are named with `step` or `job` _first_, which disambiguates them in the templates directory. - Most jobs can be run on different `pool`s, so that we can put expensive jobs on expensive build agents and cheap jobs on cheap build agents. Some jobs handle pool selection on their own, however. Our original build pipelines used the `VSBuild` task _all over the place._ This resulted in Terminal being built in myriad ways, different for every pipeline. There was an attempt at standardization early on, where `ci.yml` consumed jobs and steps templates... but when `release.yml` was added, all of that went out the window. The new pipelines are consistent and focus on a small, well-defined set of jobs: - `job-build-project` - This is the big one! - Takes a list of build configurations and platforms. - Produces an artifact named `build-PLATFORM-CONFIG` for the entire matrix of possibilities. - Optionally signs the output and produces a bill of materials. - Admittedly has a lot going on. - `job-build-package-wpf` - Takes a list of build configurations and platforms. - Consumes the `build-` artifact for every config/platform possibility, plus one for "Any CPU" (hardcoded; this is where the .NET code builds) - Produces one `wpf-nupkg-CONFIG` for each configuration, merging all platforms. - Optionally signs the output and produces a bill of materials. - `job-merge-msix-into-bundle` - Takes a list of build configurations and platforms. - Consumes the `build-` artifact for every config/platform - Produces one `appxbundle-CONFIG` for each configuration, merging all platforms for that config into one `msixbundle`. - Optionally signs the output and produces a bill of materials. - `job-package-conpty` - Takes a list of build configurations and platforms. - Consumes the `build-` artifact for every config/platform - Produces one `conpty-nupkg-CONFIG` for each configuration, merging all platforms. - Optionally signs the output and produces a bill of materials. - `job-test-project` - Takes **one** build config and **one** platform. - Consumes `build-PLATFORM-CONFIG` - Selects its own pools (hardcoded) because it knows about architectures and must choose the right agent arch. - Runs tests (directly on the build agent). - `job-run-pgo-tests` - Just like the above, but runs tests where `IsPgo` is `true` - Collects all of the PGO counts and publishes a `pgc-intermediates` artifact for that platform and configuration. - `job-pgo-merge-pgd` - Takes **one** build config and multiple platforms. - Consumes `build-$platform-CONFIG` for each platform. - Consumes `pgc-intermediates-$platform-CONFIG` for each platform. - Merges the `pgc` files into `pgd` files - Produces a new `pgd-` artifact. - `job-pgo-build-nuget-and-publish` - Consumes the `pgd-` artifact from above. - Packs it into a `nupkg` and publishes it. - `job-submit-windows-vpack` - Only expected to run against `Release`. - Consumes the `appxbundle-CONFIG` artifact. - Publishes it to a vpack for Windows to consume. - `job-check-code-format` - Does not use artifacts. Runs `clang-format`. - `job-index-github-codenav` - Does not use artifacts. Fuzz submission is broken due to changes in the `onefuzz` client. I have removed the compliance and security build because it is no longer supported. Finally, this pull request has some additional benefits: - I've expanded the PGO build phase to cover ARM64! - We can remove everything Helix-related except the WTT parser - We no longer depend on Helix submission or Helix pools - The WPF control's inner DLLs are now codesigned (#15404) - Symbols for the WPF control, both .NET and C++, are published alongside all other symbols. - The files we submit to ESRP for signing are batched up into a single step[^1] Closes #11874 Closes #11974 Closes #15404 [^1]: This will have to change if we want to sign the individual per-architecture `.appx` files before bundling so that they can be directly installed.
claunia added the pull-request label 2026-01-31 09:42:41 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
⛺ Reserved
A11yCO
A11yMAS
A11ySev1
A11ySev2
A11ySev3
A11yTTValidated
A11yUsable
A11yVoiceAccess
A11yWCAG
Area-Accessibility
Area-AtlasEngine
Area-AzureShell
Area-Build
Area-Build
Area-Chat
Area-CmdPal
Area-CodeHealth
Area-Commandline
Area-CookedRead
Area-DefApp
Area-Extensibility
Area-Fonts
Area-GroupPolicy
Area-i18n
Area-Input
Area-Interaction
Area-Interop
Area-Localization
Area-Output
Area-Performance
Area-Portable
Area-Quality
Area-Remoting
Area-Rendering
Area-Schema
Area-Server
Area-Settings
Area-SettingsUI
Area-ShellExtension
Area-ShellExtension
Area-ShellExtension
Area-Suggestions
Area-Suggestions
Area-TerminalConnection
Area-TerminalControl
Area-Theming
Area-UserInterface
Area-VT
Area-Windowing
Area-WPFControl
AutoMerge
Blocking-Ingestion
Culprit-Centennial
Culprit-WinUI
Disability-All
Disability-Blind
Disability-LowVision
Disability-Mobility
External-Blocked-WinUI3
Fixed
Gathering-Data
good first issue
HCL-E+D
HCL-WindowsTerminal
Help Wanted
Impact-Compatibility
Impact-Compliance
Impact-Correctness
Impact-Visual
In-PR
InclusionBacklog
InclusionBacklog-Windows TerminalWin32
InclusionCommitted-202206
Issue-Bug
Issue-Docs
Issue-Feature
Issue-Feature
Issue-Question
Issue-Samples
Issue-Scenario
Issue-Task
Needs-Attention
Needs-Author-Feedback
Needs-Bisect
Needs-Discussion
Needs-Repro
Needs-Tag-Fix
Needs-Tag-Fix
Needs-Triage
No-Recent-Activity
Priority-0
Priority-1
Priority-2
Priority-3
Product-Cmd.exe
Product-Colortool
Product-Colortool
Product-Colortool
Product-Conhost
Product-Conpty
Product-Meta
Product-Powershell
Product-Terminal
Product-WSL
pull-request
Mirrored from GitHub Pull Request
 Resolution-Answered
Resolution-By-Design
Resolution-Duplicate
Resolution-External
Resolution-Fix-Available
Resolution-Fix-Committed
Resolution-No-Repro
Resolution-Won't-Fix
Severity-Blocking
Severity-Crash
Severity-DataLoss
spam
this-will-be-a-breaking-change
Tracking-External
WindowsTerminal_Win32
Work-Item
zAskModeBug
zInbox-Bug
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#30742
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?