Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ on: [push, pull_request]

jobs:
build:
name: Main Build
name: Main Build (java ${{ matrix.java }}, shard ${{ matrix.shard }})
runs-on: ubuntu-latest
if: github.repository == 'apache/drill'
timeout-minutes: 150
Expand All @@ -36,6 +36,12 @@ jobs:
# Java versions to run unit tests (Jetty 12 requires Java 17+)
java: [ '17', '21', '25' ]
profile: ['default-hadoop']
# exec/java-exec holds ~31 minutes of the suite in 577 classes, which dominates
# the build. Splitting it three ways by class name brings the longest shard to
# about 11 minutes. Boundaries were chosen from measured per-class times; the
# patterns are exhaustive, so a newly added test always lands in exactly one
# shard rather than being silently skipped.
shard: [ '1', '2', '3' ]
fail-fast: false
steps:
- name: Checkout
Expand All @@ -57,9 +63,36 @@ jobs:
mkswap /tmp/swapfile
swapon /tmp/swapfile
"
# Which slice of the suite this job runs. The patterns are spelled out one prefix
# at a time because surefire does NOT support character-class globs:
# -Dtest='Test[A-F]*' silently matches nothing and the build passes green having
# run no tests. Only '*' wildcards and comma separated lists work, so each letter
# gets its own entry, in both cases (TestforBaseTestInheritance has a lowercase f).
#
# Shard 3 repeats surefire's other default includes (*Test, *Tests, *TestCase) to
# pick up classes not named Test*. Verified exhaustive against every test
# class in the repo. TestFrameworkTest matches shards 1 and 3 and runs twice —
# harmless, and far safer than a gap that skips tests without saying so.
- name: Select test shard
id: shard
run: |
case "${{ matrix.shard }}" in
1) echo 'pattern=TestA*,Testa*,TestB*,Testb*,TestC*,Testc*,TestD*,Testd*,TestE*,Teste*,TestF*,Testf*' >> "$GITHUB_OUTPUT" ;;
2) echo 'pattern=TestG*,Testg*,TestH*,Testh*,TestI*,Testi*,TestJ*,Testj*,TestK*,Testk*,TestL*,Testl*,TestM*,Testm*,TestN*,Testn*,TestO*,Testo*,TestP*,Testp*' >> "$GITHUB_OUTPUT" ;;
3) echo 'pattern=TestQ*,Testq*,TestR*,Testr*,TestS*,Tests*,TestT*,Testt*,TestU*,Testu*,TestV*,Testv*,TestW*,Testw*,TestX*,Testx*,TestY*,Testy*,TestZ*,Testz*,*Test,*Tests,*TestCase,Test0*,Test1*,Test2*,Test3*,Test4*,Test5*,Test6*,Test7*,Test8*,Test9*,Test_*' >> "$GITHUB_OUTPUT" ;;
esac

# -DforkCount=2: several of the slowest classes are wall-clock bound rather than
# CPU bound — TestTimedCallable spends 105 seconds asleep, and the RPC, SSL and
# graceful-shutdown tests wait on timeouts — so a second fork overlaps that idle
# time at little extra cost. Raise further only with memory headroom: the vector
# tests fail with OutOfMemoryException under pressure.
- name: Build and test
run: |
mvn -P${{ matrix.profile }} install --batch-mode --no-transfer-progress \
-DforkCount=2 \
-Dsurefire.failIfNoSpecifiedTests=false \
-Dtest='${{ steps.shard.outputs.pattern }}' \
-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
- name: Remove swap space
run : |
Expand Down
Loading