Skip to content

Use more reasonable timeouts for some jobs - #156097

Open
StanFromIreland wants to merge 1 commit into
python:mainfrom
StanFromIreland:workflow-timeouts
Open

Use more reasonable timeouts for some jobs#156097
StanFromIreland wants to merge 1 commit into
python:mainfrom
StanFromIreland:workflow-timeouts

Conversation

@StanFromIreland

Copy link
Copy Markdown
Member

A job that on average, takes under half a minute, does not need a 60 minute time out. As such, when something goes wrong and the job hangs we are needlessly wasting CI resources. So I propose some slightly stricter timeout-minutes. They are all still quite generous, however, giving around 2-4x the average time.

Also, drop actions/setup-python in tail-call.yml, it's not needed for anything.

@webknjaz webknjaz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please!

Here's some more context from the past:

@webknjaz

Copy link
Copy Markdown
Member

They are all still quite generous, however, giving around 2-4x the average time.

If we wanted to be really precise, we could have step-bound timeouts with specific I/O (deps installs) or test runs capped. One other thing to consider could be that when you take measurements of the old job runs, sometimes they show cache-optimized timings and when that cache gets invalidated, the time will be different. My strategy is to find the slowest non-cached run and give it some percentage on top.

- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I'd probably prefer to have this in a standalone PR, FWIW)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to include it, it's just clean up after all. I presume it was copy-pasted from the JIT workflow. CC @Fidget-Spinner (I think you created tail-call.yml?)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I dont think the tail call yanl file had any serious thought put into it in the initial impl by me. Feel free to improve it however you deem fit.

@hugovk

hugovk commented Aug 21, 2026

Copy link
Copy Markdown
Member

Do these jobs often stall such they frequently hit the timeout?

A lot of these aren't making it a huge amount of difference, like 60->40 or 30, 10->5 or 2.

The main point for timeout is that we don't fall back to the default 6 hours, and have something roughly reasonable.

I'm not sure we need to spend a lot of our own time fine-tuning and then maintaining slim timeouts, when (hopefully!) timeouts are rarely needed.

And going the other way, if something ends up running slower than normal, I don't think we should kill the job early, when it might still finish in some sort of reasonable time. We have had some Ubuntu image fetches running slower not so long ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting core review infra CI, GitHub Actions, buildbots, Dependabot, etc. skip issue skip news

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants