Skip to content

feat(python-sdk): move template build-context uploads onto pyqwest - #1603

Draft
mishushakov wants to merge 5 commits into
migrate-volume-to-pyqwestfrom
migrate-template-uploads-to-pyqwest
Draft

feat(python-sdk): move template build-context uploads onto pyqwest#1603
mishushakov wants to merge 5 commits into
migrate-volume-to-pyqwestfrom
migrate-template-uploads-to-pyqwest

Conversation

@mishushakov

@mishushakov mishushakov commented Jul 23, 2026

Copy link
Copy Markdown
Member

What

Stacked on #1602 (which is stacked on #1601). Migrates the template build-context uploads (streaming the build archive to S3 presigned URLs in build_api.upload_file) onto pyqwest via its httpx-compatible transport adapter.

Originally deferred from #1601 because S3 presigned URLs reject chunked transfer encoding and Content-Length framing through reqwest was unverified. Verified at the wire level (raw-socket capture server): httpx's Content-Length — derived from the spooled archive (sync) or set explicitly on the async-iterator body (async) — is forwarded by the adapter and reqwest keeps Content-Length framing for streamed bodies, no chunked fallback.

Note

Rebased onto #1601, which maps httpx.Proxy onto pyqwest's Proxy object and therefore requires pyqwest 0.8 (released, and locked in #1601). The only change here is following the rename — this PR builds its transport from proxy_to_config(...) instead of proxy_to_url(...).

How

  • e2b/template_sync/build_api.py / template_async/build_api.py: upload_file uses a one-off pyqwest transport (ApiPyqwestTransport from feat(python-sdk): move the REST API client onto pyqwest's httpx transport adapter #1601) instead of the generated client's httpx transport.
  • verify_ssl=False on the generated client is no longer honored for uploads (pyqwest has no insecure-TLS option), and http2=False is gone (S3 negotiates HTTP/1.1 via ALPN anyway).
  • The 1-hour upload timeout now bounds the entire upload rather than each socket write — arguably the intended meaning for that endpoint.

Testing

Usage example

No API changes — template builds upload their context exactly as before:

from e2b import Template

template = Template().from_image("ubuntu:22.04").copy("data/", "/data")
Template.build(template, alias="my-template")   # archive upload now goes through pyqwest

🤖 Generated with Claude Code

@cla-bot cla-bot Bot added the cla-signed label Jul 23, 2026
@changeset-bot

changeset-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ada7677

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@e2b/python-sdk Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@cursor

cursor Bot commented Jul 23, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Upload path swap plus timeout semantics change and loss of verify_ssl=False for presigned PUTs could affect large builds or TLS-debug setups; S3 Content-Length contract is covered by existing regression tests.

Overview
Template build-context uploads to S3 presigned URLs in sync and async upload_file now use a one-off pyqwest httpx transport (ApiPyqwestTransport / AsyncApiPyqwestTransport) instead of the default httpx stack. Proxy settings come from proxy_to_config on the API client’s _proxy.

Content-Length framing for streamed archive bodies is unchanged (required so S3 does not get chunked encoding). The default 1-hour upload timeout now applies to the whole request, not per socket write. verify_ssl=False on the generated client is not applied to these uploads because pyqwest has no insecure-TLS path. http2=False is removed for uploads.

Upload regression tests normalize header names to lowercase because pyqwest/hyper lowercases them where httpcore title-cased them.

Reviewed by Cursor Bugbot for commit ada7677. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Package Artifacts

Built from 7ef7a31. Download artifacts from this workflow run.

JS SDK (e2b@2.38.1-migrate-template-uploads-to-pyqwest.0):

npm install ./e2b-2.38.1-migrate-template-uploads-to-pyqwest.0.tgz

CLI (@e2b/cli@2.16.2-migrate-template-uploads-to-pyqwest.0):

npm install ./e2b-cli-2.16.2-migrate-template-uploads-to-pyqwest.0.tgz

Python SDK (e2b==2.37.0+migrate.template.uploads.to.pyqwest):

pip install ./e2b-2.37.0+migrate.template.uploads.to.pyqwest-py3-none-any.whl

@linear-code

linear-code Bot commented Jul 30, 2026

Copy link
Copy Markdown

SDK-268

@mishushakov
mishushakov force-pushed the migrate-template-uploads-to-pyqwest branch from a6fdb80 to 08cacf7 Compare July 30, 2026 16:35
@mishushakov
mishushakov changed the base branch from migrate-rest-to-pyqwest to migrate-volume-to-pyqwest July 30, 2026 16:35
@mishushakov
mishushakov force-pushed the migrate-template-uploads-to-pyqwest branch from 08cacf7 to fa0a167 Compare July 30, 2026 16:36
@mishushakov
mishushakov force-pushed the migrate-template-uploads-to-pyqwest branch from fa0a167 to ccab1da Compare July 31, 2026 14:51
mishushakov and others added 5 commits August 4, 2026 16:00
Volume/AsyncVolume file operations join the API client on the
ApiPyqwestTransport + connection-retry stack; transport caches become
process-global keyed by proxy. httpx.Proxy values reduce to plain proxy
URLs; stream_idle_timeout is accepted but a no-op at this point (bounded
by the transport in the next commits).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n a dedicated transport

reqwest's read timer keeps ticking while a request body is sent and while
waiting for the response head, so a shared read_timeout would kill slow
uploads; streamed reads get their own transport with read_timeout=60s
(JS SDK parity), uploads and unary calls stay unbounded-per-read.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ead_file

asyncio.wait_for around each read (response head and every chunk); explicit
values run on the regular transport so values above the 60s transport bound
aren't capped and 0 disables idle bounding. Sync keeps the parameter but
ignores it — it cannot interrupt a blocking read into the Rust transport.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… alias

Follows the control-plane PR in dropping the runtime import of httpx's private
`_types` module: the volume connection config and the `Volume`/`AsyncVolume`
constructors take the `ProxyTypes` alias `e2b.connection_config` now owns. The
volume client factories needed no change — they key their transports by
`ProxyConfig` and no longer name the input union at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
build_api.upload_file uses a one-off pyqwest transport; Content-Length
framing verified at the wire level for both the spooled sync archive and
the async-iterator body (S3 rejects chunked encoding). The upload-file
regression tests compare header names case-insensitively since hyper
lowercases them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mishushakov
mishushakov force-pushed the migrate-template-uploads-to-pyqwest branch from ccab1da to ada7677 Compare August 4, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant