Skip to content

Commit 96a6bd7

Browse files
authored
Runs-on for linux-build-lib and linux-test (2X faster CI) (#20107)
## Which issue does this PR close? Related to #13813 Thanks to the infra team and @gmcdonald specifically, we now have the ability to use more powerful AWS-provided runners in our CI 🥳 DataFusion has one of the largest runtimes across Apache projects - that's why we're bringing those runners here first. Since we're first to test this, I think it's reasonable to do a gradual transition, so I updated the two most frequently failing actions to be hosted in AWS. The plan is to test that everything works fine and then transition the remaining actions. ## What changes are included in this PR? If the org is `apache`, we'll now use ASF-provisioned runners in the ASF infra AWS account. Forks will not have access to those runners, so they will fall back to GitHub-provisioned ones. ## Are these changes tested? Yes. - Forks in non-apache orgs: blaginin#10 - Apache org: apache/datafusion-sandbox#163 ## Are there any user-facing changes? No
1 parent 39da29f commit 96a6bd7

File tree

2 files changed

+10
-2
lines changed

2 files changed

+10
-2
lines changed

.github/workflows/rust.yml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,11 @@ jobs:
4545
# Check crate compiles and base cargo check passes
4646
linux-build-lib:
4747
name: linux build test
48-
runs-on: ubuntu-latest
48+
runs-on: ${{ github.repository_owner == 'apache' && format('runs-on={0},family=m7a,cpu=16,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }}
4949
container:
5050
image: amd64/rust
5151
steps:
52+
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
5253
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
5354
- name: Setup Rust toolchain
5455
uses: ./.github/actions/setup-builder
@@ -266,12 +267,13 @@ jobs:
266267
linux-test:
267268
name: cargo test (amd64)
268269
needs: linux-build-lib
269-
runs-on: ubuntu-latest
270+
runs-on: ${{ github.repository_owner == 'apache' && format('runs-on={0},family=m7a,cpu=16,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }}
270271
container:
271272
image: amd64/rust
272273
volumes:
273274
- /usr/local:/host/usr/local
274275
steps:
276+
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
275277
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
276278
with:
277279
submodules: true

docs/source/contributor-guide/index.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,3 +199,9 @@ Please understand the reviewing capacity is **very limited** for the project, so
199199
### Better ways to contribute than an “AI dump”
200200

201201
It's recommended to write a high-quality issue with a clear problem statement and a minimal, reproducible example. This can make it easier for others to contribute.
202+
203+
### CI Runners
204+
205+
We use [Runs-On](https://runs-on.com/) for some actions in the main repository, which run in the ASF AWS account to speed up CI time. In forks, these actions run on the default GitHub runners since forks do not have access to ASF infrastructure.
206+
207+
We also use standard GitHub runners for some actions in the main repository; these are also runnable in forks.

0 commit comments

Comments
 (0)