test: add failing test reproducing #113 (GIL held during blocking I/O) - #114
Open
monperrus wants to merge 1 commit into
Open
test: add failing test reproducing #113 (GIL held during blocking I/O)#114monperrus wants to merge 1 commit into
monperrus wants to merge 1 commit into
Conversation
…ing I/O)
cursor.execute() (and commit/rollback/fetch*/executemany/
executescript) call into blocking Rust code without releasing the
GIL. When the underlying call blocks -- e.g. waiting on sqlite's busy
handler for a lock, or (per the linked issue) on a stuck Hrana stream
-- the whole interpreter freezes, including any other Python thread.
This is why a concurrent.futures-based timeout wrapped around the
call from another thread never fires: the watcher thread's timed wait
elapses at the OS level, but it then blocks trying to reacquire the
GIL from the thread stuck inside the extension call, since that
thread never released it.
This test blocks one connection against another via sqlite's busy
handler (no remote server required) and asserts a background thread
keeps ticking during the wait. It currently fails:
AssertionError: background thread only ticked 2 times while
cursor.execute() was blocked for ~1.5s -- GIL was likely held
during the blocking call
Ref: tursodatabase#113
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a regression test for #113 that reproduces the root cause without needing a remote Hrana server.
It blocks one local connection against another via sqlite's busy handler (a real blocking call inside the extension, same code path as a stuck Hrana call) and asserts a background Python thread keeps making progress while the call is blocked. It currently fails on
main:This confirms the mechanism behind the issue:
cursor.execute()(andcommit/rollback/fetch*/executemany/executescript) call into blocking Rust code without releasing the GIL. When the underlying call hangs, the whole interpreter freezes, including any other thread -- which is also why aconcurrent.futures-based timeout wrapped around the call from another thread never actually fires (the watcher thread's OS-level timed wait elapses, but it then blocks trying to reacquire the GIL from the thread stuck inside the extension call).No behavior/source changes here, test only.