Conversation
|
✅ No conflicts with other open PRs targeting |
There was a problem hiding this comment.
Pull request overview
Risk Rating: MEDIUM
This PR updates multiple GitHub Actions workflows to prefer spot runners for most runs, while forcing on-demand (non-spot) runners for merge-queue and release-related events/refs to avoid blocking critical pipelines.
Changes:
- Updated many
runs-on/ runner-label strings to dynamically setspot=co(default) vsspot=falseformerge_group, release branches, release tags, andreleaseevents. - Updated several jq-generated per-test runner labels to parameterize
spotvia a bash variable. - Added a composite action intended to centralize the spot decision logic.
Scrupulous human review needed:
- Verify the release/merge-queue detection logic matches the repo’s actual release and tagging flows (
merge_group,refs/heads/release/…,refs/tags/v…,releaseevent), since misclassification can materially impact runner cost and pipeline latency. - Confirm the runner label parser backing
runs-on=.../spot=...treatsspot=falseandspot=coexactly as intended across all instance families used here.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/integration-tests.yml | Makes builder and compile runners switch spot based on event/ref conditions. |
| .github/workflows/go-mod-cache.yml | Changes cache jobs from fixed spot=false to conditional spot behavior. |
| .github/workflows/docker-build.yml | Makes Docker build runner labels conditionally use spot vs non-spot. |
| .github/workflows/cre-wf-caching-test.yml | Switches CRE caching test runner to conditional spot behavior. |
| .github/workflows/cre-system-tests.yaml | Parameterizes jq-generated runner labels with conditional spot. |
| .github/workflows/cre-soak-memory-leak.yml | Switches soak test runner to conditional spot behavior. |
| .github/workflows/cre-regression-system-tests.yaml | Parameterizes regression runner labels with conditional spot. |
| .github/workflows/cre-local-env-tests.yaml | Switches local env test jobs to conditional spot behavior. |
| .github/workflows/codeql.yml | Makes CodeQL Go runner label conditional on event/ref. |
| .github/workflows/ci-deployments.yml | Switches deployment job runners to conditional spot behavior. |
| .github/workflows/ci-core.yml | Switches multiple core CI runner labels to conditional spot behavior. |
| .github/workflows/ccip-system-tests.yaml | Parameterizes CCIP system test runner labels with conditional spot. |
| .github/actions/get-runner-spot/action.yml | Adds a composite action to compute spot settings (currently not wired into workflows). |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Suppressed comments (1)
.github/workflows/integration-tests.yml:251
build-chainlinkruns two matrix jobs (core + plugins) that can execute concurrently, butSH_BUILDER_RUNNERis now the same label for both (based only ongithub.run_id). This reintroduces runner contention/serialization that the previous-core/-pluginssuffixes avoided (the earlier comment explicitly called this out). Consider restoring per-matrix unique labels (e.g.,runs-on=${{ github.run_id }}-core/...and...-plugins/...) or otherwise including a matrix-specific suffix in theruns-on=label.
GH_BUILDER_RUNNER: ubuntu22.04-8cores-32GB
SH_BUILDER_RUNNER: runs-on=${{ github.run_id }}/cpu=16/ram=64/family=m7i+m8i/${{ steps.spot-config.outputs.spot_param }}/extras=s3-cache+tmpfs
run: |
if [[ "${OPT_OUT}" == "true" ]]; then
echo "builder-runner-label=${GH_BUILDER_RUNNER}" | tee -a "$GITHUB_OUTPUT"
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
.github/workflows/integration-tests.yml:288
- Both matrix entries use the same
needs.labels.outputs.builder-runner-labelvalue. If the self-hosted runner provisioning expects a unique label per job (as the previous-core/-pluginssuffixes implied), this can cause runner contention/serialization or runner-stealing between the core and plugins image builds.
- name: ""
runner: ${{ needs.labels.outputs.builder-runner-label || 'ubuntu22.04-8cores-32GB' }}
dockerfile: core/chainlink.Dockerfile
tag-suffix: ""
cache-scope: core
any-should-run: >-
${{
github.event_name == 'workflow_dispatch' ||
needs.changes.outputs.general-changes == 'true' ||
needs.changes.outputs.core-changes == 'true' ||
needs.changes.outputs.cre-changes == 'true' ||
needs.changes.outputs.ccip-changes == 'true' ||
needs.labels.outputs.run-e2e-tests-label-found == 'true'
}}
- name: (plugins)
runner: ${{ needs.labels.outputs.builder-runner-label || 'ubuntu22.04-8cores-32GB' }}
dockerfile: plugins/chainlink.Dockerfile
.github/workflows/codeql.yml:119
- This changes the self-hosted Go CodeQL runner instance family to
c6i, which is unrelated to the PR's stated goal (spot gating) and differs from thec7*families used elsewhere for similar 32cpu/64GB runners. If unintentional, it could reduce performance/capacity for CodeQL runs.
env:
OPT_OUT: ${{ steps.pr-labels.outputs.check-label-found || 'false' }}
SH_GO_RUNNER: runs-on=${{ github.run_id }}/cpu=32/ram=64/family=c6i/${{ steps.spot-config.outputs.spot_param }}/extras=s3-cache+tmpfs
GH_GO_RUNNER: ubuntu24.04-8cores-32GB
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.
Suppressed comments (3)
.github/workflows/integration-in-memory-tests.yml:52
actions/checkoutin thechangesjob can end up checking out the repository default branch when neitherinputs.chainlink_versionnorinputs.cl_refis set (both are optional). For PR/merge_group events that meansadvanced-triggersmay evaluate the wrong code revision, and it contradicts thechainlink_versioninput description (“defaults to github.sha”).
- name: Checkout the repo
uses: actions/checkout@v7
with:
persist-credentials: false
repository: smartcontractkit/chainlink
ref: ${{ inputs.chainlink_version || inputs.cl_ref }}
.github/workflows/integration-tests.yml:253
SH_BUILDER_RUNNERis now a single label shared by both the core and plugins build matrix entries. This removes the previous per-image isolation and can cause the two builds to contend for the same ephemeral self-hosted runner (serialization) or conflict over the workspace/caches, increasing CI latency and flakiness.
OPT_OUT: ${{ steps.label-runs-on-opt-out.outputs.check-label-found || 'false' }}
GH_BUILDER_RUNNER: ubuntu22.04-8cores-32GB
SH_BUILDER_RUNNER: runs-on=${{ github.run_id }}/cpu=16/ram=64/family=m7i+m8i/${{ steps.spot-config.outputs.spot_param }}/extras=s3-cache+tmpfs
run: |
if [[ "${OPT_OUT}" == "true" ]]; then
echo "builder-runner-label=${GH_BUILDER_RUNNER}" | tee -a "$GITHUB_OUTPUT"
else
echo "builder-runner-label=${SH_BUILDER_RUNNER}" | tee -a "$GITHUB_OUTPUT"
.github/workflows/ci-core.yml:186
- This PR’s description is about gating
spotby event/ref, but this hunk also changes the instance family selection (e.g.,family=c7gd+c8gforgolangci). Similar family changes appear elsewhere in this workflow (unit/fuzz). If unintentional, it could change capacity/availability characteristics or break runner matching.
runs-on:
- runs-on=${{ github.run_id }}-${{ strategy.job-index }}
- cpu=16
- ram=32
- family=c7gd+c8g
- ${{ needs.filter.outputs.spot_param }}
- image=ubuntu24-full-arm64
- extras=s3-cache
|





Spot instances are great for cost savings, and should be used for PRs in most cases, but when used in releases or merge queues, they can seriously hold up development. This gates us to use
spot=falsewhen we're merging or pushing a tag.Big Changes
Small Changes
runs-onrunners to use list yaml approach instead of one-liner where possible. I think it makes it a bit easier to parse.