Use sccache on CI to reduce the build time #11517

Open
opened 2026-01-31 02:49:59 +00:00 by claunia · 0 comments
Owner

Originally created by @skyline75489 on GitHub (Nov 21, 2020).

Description of the new feature/enhancement

I just found this amazing project sccache by Mozilla, which is a improved version of ccache and can be used to speed up the build.

I found it in the PyTorch CircleCI run like this one.

Per the documentation of sccache:

sccache is a ccache-like compiler caching tool. It is used as a compiler wrapper and avoids compilation when possible, storing cached results either on local disk or in one of several cloud storage backends.

sccache includes support for caching the compilation of C/C++ code, Rust, as well as NVIDIA's CUDA using nvcc.
sccache also provides icecream-style distributed compilation

Proposed technical implementation details (optional)

By simply storing cached compilation result on cloud storage, I think we can speed up the compilation dramatically. And there's more - distributed compilation!

I'm not sure Azure DevOps builtin sccache support, though. That would be a problem.

Originally created by @skyline75489 on GitHub (Nov 21, 2020). <!-- 🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨 I ACKNOWLEDGE THE FOLLOWING BEFORE PROCEEDING: 1. If I delete this entire template and go my own path, the core team may close my issue without further explanation or engagement. 2. If I list multiple bugs/concerns in this one issue, the core team may close my issue without further explanation or engagement. 3. If I write an issue that has many duplicates, the core team may close my issue without further explanation or engagement (and without necessarily spending time to find the exact duplicate ID number). 4. If I leave the title incomplete when filing the issue, the core team may close my issue without further explanation or engagement. 5. If I file something completely blank in the body, the core team may close my issue without further explanation or engagement. All good? Then proceed! --> # Description of the new feature/enhancement I just found this amazing project [sccache](https://github.com/mozilla/sccache) by Mozilla, which is a improved version of ccache and can be used to speed up the build. I found it in the PyTorch CircleCI run like [this one](https://app.circleci.com/pipelines/github/pytorch/pytorch/241977/workflows/d0eb985d-94c9-428c-9c2e-e58bb8217fcf/jobs/9103296). Per the documentation of sccache: >sccache is a ccache-like compiler caching tool. It is used as a compiler wrapper and avoids compilation when possible, storing cached results either on local disk or in one of several cloud storage backends. > >sccache includes support for caching the compilation of C/C++ code, Rust, as well as NVIDIA's CUDA using nvcc. >sccache also provides icecream-style distributed compilation <!-- A clear and concise description of what the problem is that the new feature would solve. Describe why and how a user would use this new functionality (if applicable). --> # Proposed technical implementation details (optional) By simply storing cached compilation result on cloud storage, I think we can speed up the compilation dramatically. And there's more - distributed compilation! I'm not sure Azure DevOps builtin sccache support, though. That would be a problem. <!-- A clear and concise description of what you want to happen. -->
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/terminal#11517