Skip to content

fix(io): run blocking device calls on the io executor, not the event loop - #1199

Open
rickwierenga wants to merge 1 commit into
mainfrom
fix/ftdi-executor-offload
Open

fix(io): run blocking device calls on the io executor, not the event loop#1199
rickwierenga wants to merge 1 commit into
mainfrom
fix/ftdi-executor-offload

Conversation

@rickwierenga

Copy link
Copy Markdown
Member

FTDI.write, FTDI.read and FTDI.readline are async def but call into libftdi directly, so every transfer blocks the event loop until it returns. The ThreadPoolExecutor the object creates in setup() is never used for them. This moves them onto that worker.

Device open and close had the same shape in ftdi, hid and usb, and the port scan in serial:

  • hid.setup ran hid.enumerate() and hid.Device(path=…) on the loop; hid.stop ran device.close() there.
  • usb.setup ran device discovery, set_configuration(), endpoint resolution and the while self._read_packet() is not None buffer drain on the loop. The drain is the worst of these — an unbounded sequence of timeout-bounded blocking reads, so a device with a full buffer stalls everything.
  • serial.setup ran comports() on the loop (a sysfs/registry sweep).

Each is now a _setup_sync that runs as a single unit on the executor. Running it as one callable rather than wrapping the individual calls keeps the handle opened by the thread that later reads and writes it, and means no other coroutine can observe a half-configured device between the awaits — usb.read/write check self.dev and self.read_endpoint, which were previously set several await points apart.

Two fixes fall out of this:

  • The executors are created before the open and torn down when it fails. serial.setup created its executor at the top and then returned from the "No machines found" and "Multiple devices detected" paths without shutting it down, leaking a thread per failed setup.
  • Executor shutdown is now wait=False, cancel_futures=True. The worker is idle by the time stop() reaches it, so wait=True could only ever block the loop behind an unrelated queued read.

Behavior is otherwise unchanged: the same exceptions and messages, and the existing if len(data) != 0 guards around read/readline capture recording are kept, so capture files replay as before.

Not included: USB.ctrl_transfer is sync and does blocking libusb I/O. Offloading it means making it async, which is a breaking change, so it is left for a separate discussion.

Testing

ruff format, ruff check and mypy clean; full test suite passes (2090 passed, 3 skipped, 162 subtests). Not tested against hardware.

🤖 Generated with Claude Code

…loop

FTDI's write, read and readline called into libftdi directly from async
methods, blocking the event loop for the duration of the transfer. Move
them onto the single worker that the io object already owns.

Device open and close had the same problem in ftdi, hid and usb, and the
serial port scan in serial: enumeration, configuration and the usb read
buffer drain all ran on the loop. Each is now a _setup_sync that runs as
one unit on the executor, so the handle is opened by the thread that
later uses it and no other coroutine observes a half-configured device.

Executors are created before the open and torn down when it fails, which
also fixes a thread leak in serial's setup, where the "no machines found"
and "multiple devices detected" paths returned without shutting theirs
down. Shutdown is now non-blocking: the worker is idle by that point, so
waiting on it could only stall the loop behind an unrelated queued read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant