feat: Add large command chunking

[camdecoster]

Description

Add handling of large commands such that we don't hit Chromium's 100MiB buffer size limit.

Closes #305.

Changes

• Add size check for incoming commands
• Add chunking code for large commands
• Add tests

Testing

• Be on this branch
• Create a new venv
• Install plotly, kaleido==1.2.0, choreographer, and any other required dependencies. Kaleido 1.2.0 is required because 1.3.0 added chunking code and we want to test the Choreographer chunking code.
• Save the following code and run it in the venv:

   import asyncio
   import time
   from importlib.metadata import version
   from pathlib import Path
   
   import numpy as np
   import plotly.graph_objects as go
   from plotly.io.json import to_json_plotly
   
   import kaleido
   
   MI = 1024 * 1024
   N_POINTS = 6_000_000  # ~140MiB serialized
   OUT = Path("big_figure.png")
   
   
   async def main():
       fig = go.Figure(
           go.Scatter(
               x=np.arange(N_POINTS, dtype="float64"),
               y=np.sin(np.arange(N_POINTS, dtype="float64") / 100_000.0),
               mode="lines",
           ),
       )
   
       size = len(to_json_plotly(fig, engine="orjson"))
       print(f"kaleido: {version('kaleido')}, choreographer: {version('choreographer')}")
       print(f"figure:  {N_POINTS:,} points, {size / MI:.1f} MiB serialized")
       print(f"limit:   100.0 MiB  ->  {'OVER' if size > 100 * MI else 'under'}")
   
       start = time.perf_counter()
       await kaleido.write_fig(fig, path=OUT, opts={"format": "png"})
       elapsed = time.perf_counter() - start
   
       if not OUT.is_file():
           print("\nno file produced")
           raise SystemExit(1)
       print(f"\nwrote {OUT} ({OUT.stat().st_size / 1024:.0f} KiB) in {elapsed:.1f}s")
   
   
   asyncio.run(main())

• Note that the script hangs
• Kill the script
• Install the local repo as an editable project: uv pip install -e /path/to/choreographer
• Run the script again
• Note that the script completes successfully

Notes

• This only works for calls to Runtime.callFunctionOn
• The change should be transparent to users
• With this change merged, the Kaleido chunking added in fix: exporting a large figure hangs Kaleido#442 can be removed
• Here's a flow diagram describing how this works:

flowchart TD
    A["send_command"] --> B["serialize once"]
    B --> C{"over 100MiB?"}

    C -->|no| D["write to pipe"]
    D --> R["response to caller"]

    C -->|yes| E{"Runtime.callFunctionOn<br/>with value arguments?"}
    E -->|no| F["raise MessageTooLargeError"]

    E -->|yes| G["push the serialized text into<br/>the page in 10MiB pieces"]
    G --> H["run a wrapper that rejoins them,<br/>pulls out the arguments, and<br/>applies the caller's function"]
    H --> R

Loading