From dfdd55cb26999b272bb491d51dfc3f06f54dbf03 Mon Sep 17 00:00:00 2001 From: jotpalch Date: Thu, 20 Aug 2026 12:52:14 +0800 Subject: [PATCH] Add a usbdevfs fd with the synchronous ioctls New src/syscall/usbdev.c/.h: typed FD_USBDEV constructor for /dev/bus/usb/BBB/DDD (any access mode except O_PATH; the host fd is a readiness pipe for the async stage's poll), read/pread/readv/lseek serving the usbfs descriptors blob at a per-open position, fstat as char 189:minor, and the synchronous ioctl set against IOUSBDeviceInterface650 / IOUSBInterfaceInterface800: CLAIM/RELEASEINTERFACE, SETINTERFACE (implicit claim + SetAlternateInterface + pipe-map rebuild), SETCONFIGURATION (-EBUSY while anything on the device is claimed, device-wide like usb_interface_claimed: this fd's claims, another usbfs fd's claims via a lock-free per-slot mirror, and bound host drivers via one registry iterator pass), CLEAR_HALT/RESETEP (ClearPipeStallBothEnds), GETDRIVER, GET_SPEED (the enum as the return value), CONNECTINFO, DISCONNECT_CLAIM, USBDEVFS_IOCTL DISCONNECT/CONNECT, sync CONTROL (DeviceRequestTO) and sync BULK (Read/WritePipeTO). kIOReturn maps to errno per devio.c: a stall is -EPIPE, a timeout -ETIMEDOUT, a vanished device -ENODEV, and kIOReturnAborted is -EINTR with syscall_restart_forbid(), because the transfer was already on the wire and a dispatcher restart would send it twice; the eintr contract records ioret_neg_errno as 'forbids'. The open consults the model, not the hardware. stat, access and an O_PATH open of the same node are all answered from the model, so resolving the IOKit service at open time made a plain O_RDONLY the one entry point that reported ENODEV for a node the other three describe, and it put the ELFUSE_USB_FIXTURE model, whose devices have no IOKit service at all, out of reach of every assertion about this fd. The service is resolved on first use instead, and each operation that needs the wire reports -ENODEV without one. usb-sysfs models the node 0666 for the same reason: the owner it reports is the host user, the guest's uid need not equal it, and 0664 left access(W_OK) refusing what open(O_RDWR) then served. Interface arbitration is attempted, not predicted. macOS arbitrates per IOKit object and dynamically: USBInterfaceOpen answers kIOReturnExclusiveAccess exactly while a driver holds that interface open and succeeds while it is idle. So the claim attempts the open and maps what IOKit answers, rather than refusing on the presence of a driver child -- which refused work macOS grants. Measured on the attached ESP32-S3, with the same native IOKit probe run at the same moment: with nothing holding /dev/cu.usbmodem1101, USBInterfaceOpen on interface 1 (CDC data, bound to AppleUSBACMData) returns 0x00000000 and CLAIMINTERFACE 1 now returns 0 where it returned -EBUSY; with the tty held, both say ExclusiveAccess and -EBUSY. Interface 0 (CDC control) is ExclusiveAccess and -EBUSY either way. Relatedly, a user client is not a driver: IOKit publishes an AppleUSBHostInterfaceUserClient child for every USBInterfaceOpen, so taking the first child of any class reported a peer usbfs consumer as a bound host driver. GETDRIVER of an interface another usbfs fd holds now reports usbfs (measured: driver=AppleUSBHostInterfaceUserClient before, usbfs after). Kernel-fidelity gates and check order, all of it observable: - The two capability bits come from OPEN_FMODE, once, and five gates read them: (flags + 1) & O_ACCMODE, not a comparison against O_RDONLY (fs.h:3631). Access modes 0, 1 and 2 give FMODE_READ, FMODE_WRITE and both; access mode 3 gives neither, and it is reachable -- open(2) takes it and ACC_MODE(3) asks this 0666 node for read plus write, which it grants. Each gate tested the access mode itself, so all four agreed on modes 0, 1 and 2 and all four were wrong on mode 3: read and pread served the descriptors blob and every ioctl was accepted, where Linux answers -EBADF and -EPERM. Measured in a container, on a char device and on a plain file: the open succeeds and read, pread and write are all EBADF. - Every ioctl needs FMODE_WRITE and is -EPERM without it (devio.c:2605), ahead of everything else. - The whole write family is -EBADF without FMODE_WRITE and -EINVAL otherwise, the order vfs_write checks FMODE_WRITE then FMODE_CAN_WRITE. write() had this; writev, pwrite and pwritev fell through to the readiness pipe behind the fd and answered its -EBADF for a writable fd. - FIONBIO and FIOASYNC never reach this file's ioctl handler. do_vfs_ioctl answers both for every file before it calls f_op->unlocked_ioctl (fs/ioctl.c:818-822), so they also never meet the FMODE_WRITE gate above. Sent into it they came back -EPERM on a read-only fd and -ENOTTY on a writable one, while fcntl(F_SETFL, O_NONBLOCK) on the same descriptor succeeded and F_GETFL reported the flag: two entry points onto one flag, disagreeing about it. They are answered where FIOCLEX and FIONCLEX already are, which is where the kernel puts them relative to the file's own handler, so no other fd type's path changes. FIONBIO edits only the guest flag word -- the readiness pipe behind the fd stays nonblocking whatever the guest asks -- and always reports 0, as ioctl_fionbio does (fs/ioctl.c:342-363). FIOASYNC is not an unconditional 0: ioctl_fioasync consults f_op->fasync only when the request would change the FASYNC state and usbdev_file_operations declares none (devio.c:2846-2856), so asking for the state the fd already has is 0 and asking to change it is -ENOTTY. It must not reach the O_ASYNC arm in sys_ioctl either, which would arm a SIGIO watcher on a file the kernel refuses to arm at all. Measured in a container across all four access modes: FIONBIO(1) is 0 and sets O_NONBLOCK in every one, FIOASYNC(1) is ENOTTY and FIOASYNC(0) is 0. - usbfs_memory_mb is one allowance for every transfer in flight, not a per-call size cap. usbfs_increase_memory_usage charges len + sizeof(struct urb) against a module-global total and usbfs_decrease_memory_usage gives it back when the transfer settles (devio.c:145-178, charged at devio.c:1308). Read as a per-call ceiling it got both halves wrong: a single request of exactly the 16 MB allowance was accepted, because only the buffer was measured and the URB was not, and nothing accumulated across calls or across fds, so a guest holding all 32 side-table slots could keep 32 host buffers of that size alive at once. An atomic counter now carries the total, charged where Linux charges it and refunded on every exit from there down, the EFAULT and IOKit error arms included, which is why the OUT path's guest_read failure joins the single exit instead of returning from the middle. The per-URB overhead is a named constant rather than a claim about sizeof(struct urb), which is kernel-internal and config-dependent: what the boundary needs is that the charge strictly exceeds the length. Measured on the board before and after, interface 1, bulk OUT ep 0x02: len 16 MB reached the wire and timed out, and is now -ENOMEM; len 16 MB - 4096 still reaches the wire, twice in a row and again after a faulting transfer, so both refunds land. - check_ctrlrecip runs before the wLength cap (devio.c:1177-1183) and findintfep before the transfer-length checks (devio.c:1289-1298), so a request naming an endpoint or interface the device does not have is -ENOENT however long it is. - The reserved-bit test on an endpoint address (devio.c:860) lives in the one lookup BULK, CLEAR_HALT, RESETEP and the control endpoint recipient all reach, so the four cannot disagree about it, and it tests the caller's whole 32-bit word, as findintfep does. Narrowing to a byte first made ep 0x183 and 0x01000083 both name endpoint 0x83: measured on the board, all three cleared the stall on 0x83, where Linux answers -EINVAL for the outer two. - The interface-number bound is 64: claimintf refuses ifnum >= 8 * sizeof(ps->ifclaimed) and ifclaimed is an unsigned long (devio.c:75, :785). Between 64 and 32 an interface is merely absent, a question about the device, so claimed_mask is a uint64_t. The bound also applies to the number as it leaves the descriptor: bInterfaceNumber is a device-supplied byte with the whole 0..255 range behind it, and the endpoint-owner lookup reports it back as checkintf's -EINVAL (devio.c:842) rather than handing ifaces[64] an index of 200. - An altsetting above 255 is -EINVAL, after the implicit claim, the order proc_setintf decides them in (devio.c:1533-1539): bAlternateSetting is a byte, so usb_altnum_to_altsetting matches nothing (usb.c:391) and usb_set_interface refuses (message.c:1548). Narrowed into SetAlternateInterface's UInt8 instead, altsetting 256 selected setting 0 and took effect -- measured on the board. - The positional and vector write paths answer in Linux's order too: ksys_pwrite64 and do_pwritev test pos < 0 before the fd lookup, so a negative offset is -EINVAL rather than the -EBADF this fd owes for having no write op; vfs_readv and vfs_writev test the file's mode before an empty vector returns 0, so readv, preadv and pwritev2 of nothing still answer by direction; and pwritev2(RWF_APPEND) had no dispatch of its own and reached the readiness pipe, answering -ESPIPE. - Every failure the constructor reports carries the errno of what failed. ENODEV from the model -- nothing answers to this address -- is the one that becomes the -ENOENT open(2) owes for a name with nothing behind it; an allocation, a scratch-tree mkdir, or a pipe the host would not give reports itself, where all three used to be flattened into -ENOENT or -EMFILE. usbdev_build_pipe_map carries the IOReturn the same way instead of folding it into -EIO. - proc_disconnect_claim answers -EINVAL for an interface the device does not have (devio.c:2467-2469), the opposite of claimintf's -ENOENT. - USBDEVFS_IOCTL CONNECT on an interface that already has a driver is -EBUSY (devio.c:2362-2368); only a free one reaches device_attach. 2357-2361 is the DISCONNECT arm above it. - SETINTERFACE rewrites -ENOENT and -EPROTO to -EINVAL and passes every other answer through: the IOReturn SetAlternateInterface produces for an absent altsetting is not in the errno table and reached the map's default -EPROTO. - SEEK_END is -EINVAL, and the new position is range-checked with __builtin_add_overflow, which lseek(fd, INT64_MAX, SEEK_SET) followed by lseek(fd, 1, SEEK_CUR) reaches. - GET_CAPABILITIES reports 0. Every bit in that word names part of the SUBMITURB/REAPURB machinery, which is -ENOTTY here; the async stage raises the word as it lands each one. - A short bulk OUT is -EIO: WritePipeTO reports no length, and reporting the requested count would spell a partial write as a complete one. Locking and lifetime. A lookup pins its entry under usbdev_table_lock and takes the per-entry lock with the table lock released. Nesting the two meant a thread blocked on one fd's transfer held the table lock on every other thread's behalf: measured, a GET_SPEED on an unrelated fd waited 5517 ms behind a 6 s BULK, and it now returns in 0 ms, as do an open and a close of the same node. Two ioctls on ONE fd still serialize; Linux drops the device lock around the URB wait (devio.c:1219/1245, :1337/1357) and this does not, which is recorded below rather than claimed as kernel behavior. The teardown hook is handed a bare fd number after the fd-table slot is already free, so a sibling thread's open can already hold that number and two entries answer to it; matching on the number alone tore down whichever sat lower in the table, about half the time the live one. fd_alloc stamps a globally monotonic generation, so the closing entry is the one with the smaller generation. Measured with the mutation restored, three runs of 8000: 281, 313 and 371 freshly opened fds lost, and 0 with it on every run. The count is a race and varies; that it is never 0 without the tiebreak is the point. fd_alloc also publishes the fd number before the side table can bind it, and the same hook matches on that number, so a close landing in that window found nothing to tear down and left the entry, its blob and its pipe held for the life of the process. The open therefore rereads the fd's generation once its entry is findable and retires the entry itself if the close has been and gone. That leaves two windows around the bind rather than one, and the entry has to keep one identity across both. Before the bind its guest_fd is still -1, so a close finds nothing; after the bind it is findable and therefore also freeable, so a close can reap it, usbdev_unref can free the slot, and a sibling open can take that slot and bind its own live fd before the recheck runs. Two things make the identity unambiguous. The generation is fd_alloc_from's out parameter, captured inside the fd_lock section that stamped it, rather than a later read of the slot: re-derived after the slot was publishable it was whatever the number carried by then, which in the first window is a reopening thread's stamp, and two entries then held the identical pair -- the guard compared equal and retired nothing, and usbdev_acquire, which matches on that same pair, answered from whichever sat at the lower index. And the retire path claims the whole tuple the caller allocated -- used and alive, this fd number, this generation -- rather than testing that the slot is not dead: in the second window that test claimed the sibling's entry, marked it dead, freed its blob and closed its pipe, and the sibling's still-open fd answered EBADF on every read and ioctl, which is the failure the teardown hook's generation tiebreak exists to avoid, reintroduced on the other side of the same bind. What makes the tuple sufficient is that slot storage is static, so the read is always defined; that all four fields are written and read under usbdev_table_lock, so they are read as one value; and that fd_next_generation is globally monotonic, so each entry carries its own allocation's stamp and no two live entries can present the same pair. Both windows are driven by a widening hook, and what each one can show differs, so this says which is an assertion and which is a mutation. ELFUSE_USBDEV_PUBLISH_DELAY_US covers the first. A close inside it leaks a slot with the recheck deleted: 32 simultaneous opens before the race, 31 after, over 4 rounds. The generation half of that window cannot be shown the same way, because taking the stamp from the allocator removes the re-derivation rather than guarding it -- there is no longer a read to race. Moving the re-derivation back inside the widened window does show it: with the reopen naming the other bus, the fd read back the idVendor of the device it was not opened on and a slot went missing, 32 before and 31 after. Left where the pre-fix code had it, ahead of the hook, the same lane passes, so this window is guarded by the shape of the code and by that mutation, not by a lane that fails without the fix. ELFUSE_USBDEV_RETIRE_DELAY_US covers the second, and that one is a lane: with the claim narrowed back to 'not dead', the reopened fd, which was never closed, answered EBADF (errno 9) on pread and on GET_SPEED. Plumbing: FD_USBDEV in linux-wire.h (+LINUX_EPIPE/LINUX_ETIME), dispatch hooks in io.c (ioctl/read/write/writev/pwrite/pwritev/pwritev2 and the empty-vector arm), fs.c (openat constructor, lseek arm, dup refusal), fs-stat.c (fstat, ahead of the /dev/bus path stamp, which otherwise claimed every one of these fds and left the typed answer unreachable), usbdev_init in syscall_init, usb-sysfs exports usb_sysfs_device_info/node_stat. usbdev_table_lock is registered in internal.h's lock order, whose entry now describes what the code does -- table lock dropped before the entry lock, refs and dead pinning the slot across the gap -- rather than a nesting no path takes. The constructor takes its generation from the allocator rather than reading it back, which removes an fd_lock trip as well as the window, and the tuple it later tests is read under usbdev_table_lock alone, so fd_lock never nests under it. The completion-pipe fds are initialized to -1 so the merged error arm cannot close two indeterminate descriptors when pipe() itself fails. Tests. tests/test-usbdev-ioctl.c is a new hardware-free lane: the fixture's devices are modeled with no IOKit service behind them, which is exactly the shape of a device the machine cannot reach, so the file contract, the ioctl gates and the whole argument-validation surface get 92 assertions that answer the same on any host. It carries the four- thread open/read/close churn that the teardown identity needs, and prints each known gap as an XFAIL with both values. The ELFUSE_USB_FIXTURE model grew the endpoint descriptor its interface descriptor already claimed, so the endpoint-addressed paths have something to resolve against; tests.mk keeps the fixture on the test- usb-sysfs lane, which the earlier draft had removed after the constructor stopped serving modeled devices. test-usbdev-faults runs the same binary six more times, forcing one condition each, because none of the six can be provoked from a guest on a healthy host. ELFUSE_USB_FIXTURE=badifnum stands up a device whose one interface declares bInterfaceNumber 200 and carries endpoint 0x81: every byte is well formed and the range is the field's own, so nothing anyone can plug in emits it. The answer is -EINVAL on both sides of the bound, which is what checkintf answers, and what changes is that nothing reads out of bounds first -- with the bound removed and the fixture in place, UBSAN reports "index 200 out of bounds for type usbdev_iface_t[64]" at the array read and aborts the run. ELFUSE_USBDEV_OPEN_FAULT fails the model lookup and the descriptor copy with ENOMEM and the readiness pipe with ENFILE, which the guest saw as ENOENT, ENOENT and EMFILE before. ELFUSE_USBDEV_PUBLISH_DELAY_US and ELFUSE_USBDEV_RETIRE_DELAY_US widen the two windows around the side-table bind for the races above; each run asserts only its own window. All three hooks are read once and do nothing when unset, the same shape as the directory and identity lanes' hooks, and all three are recorded in docs/testing.md. The allowance needs a device whose interface can actually be claimed, which the service-less fixture has none of: the claim answers -ENODEV before any length is looked at. So that check says so and stands down here, and is covered where there is one -- on the board, and on the loopback model the async stage adds, which runs this same binary. Known gaps, each printed by that lane as an XFAIL and listed in internals.md: no disconnect gate (Linux answers -ENODEV for every ioctl once the device is gone; the answers served from the open-time model still report it, and noticing the disconnect needs an IOKit termination notification on a run loop, which is the async stage's machinery); two ioctls on one fd serialize; GETDRIVER reports the IOKit class name where Linux reports the driver name, and DISCONNECT_CLAIM's name filters compare against it; RESET clears the claimed pipes' stalls instead of re-enumerating (ReEnumerate would destroy the handles); a short bulk OUT is -EIO; the side table holds 32 open fds per process and reports -ENOMEM past that, where Linux allocates a usb_dev_state per open; USBDEVFS_IOCTL DISCONNECT of an interface another usbfs fd holds is -EBUSY where Linux releases that claim; dup() is -EBADF and fork drops the fd (fd_type_is_synthetic). Document the layer as internals.md USB Device Passthrough. Verified on the attached ESP32-S3 (303a:1001, modeled as /dev/bus/usb/002/001), every line re-run against this build: CLAIM iface2 = 0, CLAIM iface1 = 0 with the tty free and -EBUSY with it held, CLAIM iface0 = -EBUSY; GETDRIVER(0) = AppleUSBACMControl, GETDRIVER(5) = -ENODATA, GETDRIVER(2) from a second fd = usbfs; SETINTERFACE 2/9 = -EINVAL, 2/0 = 0; DISCONNECT_CLAIM iface5 = -EINVAL; USBDEVFS_IOCTL CONNECT iface1 = -EBUSY, iface5 = -EINVAL; CONTROL GET_DESCRIPTOR = 18 bytes matching the first 18 of the 116-byte read(); CLEAR_HALT 0x83 = 0, CLEAR_HALT 0x30 = -EINVAL; BULK OUT 1 byte = 1, BULK on absent ep 0x09 with len 32 MB = -ENOENT, BULK IN on the idle ep 0x83 with a 5000 ms timeout = -ETIMEDOUT after 5038 ms; GET_CAPABILITIES = 0, GET_SPEED = 2 (FULL), CONNECTINFO devnum 1. The arguments that used to be narrowed, same board, iface2 claimed: CLEAR_HALT 0x183 = -EINVAL and 0x01000083 = -EINVAL where all three used to clear the stall on 0x83 and return 0, RESETEP 0x183 = -EINVAL, CLEAR_HALT 0x83 = 0 still, BULK ep 0x102 = -EINVAL, SETINTERFACE 2 alt 256 = -EINVAL where it used to return 0 having selected setting 0, alt 7 = -EINVAL, alt 0 = 0, and CONTROL GET_STATUS with wIndex 0x0183 = 2 bytes, since check_ctrlrecip masks wIndex to its low byte before the lookup (devio.c:904) and that path must not gain the wider test. --- Makefile | 7 + docs/internals.md | 173 ++- docs/testing.md | 20 +- mk/tests.mk | 43 +- scripts/check-eintr-contract.py | 5 + src/runtime/usb-desc.h | 2 + src/runtime/usb-sysfs.c | 137 +- src/runtime/usb-sysfs.h | 31 + src/syscall/fs-stat.c | 16 + src/syscall/fs.c | 23 + src/syscall/internal.h | 17 +- src/syscall/io.c | 167 +++ src/syscall/linux-wire.h | 3 + src/syscall/syscall.c | 2 + src/syscall/usbdev.c | 2232 +++++++++++++++++++++++++++++++ src/syscall/usbdev.h | 74 + tests/test-usb-sysfs.c | 37 +- tests/test-usbdev-ioctl.c | 1219 +++++++++++++++++ 18 files changed, 4171 insertions(+), 37 deletions(-) create mode 100644 src/syscall/usbdev.c create mode 100644 src/syscall/usbdev.h create mode 100644 tests/test-usbdev-ioctl.c diff --git a/Makefile b/Makefile index 1b5b23aa..d7a55b47 100644 --- a/Makefile +++ b/Makefile @@ -71,6 +71,7 @@ SRCS := \ syscall/net-absock.c \ syscall/net-sockopt.c \ syscall/netlink.c \ + syscall/usbdev.c \ syscall/sysvipc.c \ debug/crashreport.c \ debug/gdbstub.c \ @@ -352,6 +353,12 @@ $(BUILD_DIR)/%: tests/%.c | $(BUILD_DIR) @echo " CROSS $<" $(Q)$(CROSS_COMPILE)gcc $(CROSS_TEST_CFLAGS) -o $@ $< +# test-usbdev-ioctl churns open/read/close on one usbdevfs node from four +# threads, so a close and a sibling's open contend for the same fd number. +$(BUILD_DIR)/test-usbdev-ioctl: tests/test-usbdev-ioctl.c | $(BUILD_DIR) + @echo " CROSS $< (with -lpthread)" + $(Q)$(CROSS_COMPILE)gcc $(CROSS_TEST_CFLAGS) -o $@ $< -lpthread + # test-eventfd-semaphore-contended races two blocking readers on one eventfd. $(BUILD_DIR)/test-eventfd-semaphore-contended: \ tests/test-eventfd-semaphore-contended.c | $(BUILD_DIR) diff --git a/docs/internals.md b/docs/internals.md index bf8c14ff..d3367446 100644 --- a/docs/internals.md +++ b/docs/internals.md @@ -106,6 +106,7 @@ Key files: | `src/syscall/time.c` | clocks, timers, `setitimer`, clock-ID translation | | `src/syscall/sys.c` | `uname`, `sysinfo`, `getrandom`, `prlimit64` | | `src/syscall/net.c`, `net-abi.c`, `net-absock.c`, `net-msg.c`, `net-sockopt.c`, `netlink.c` | sockets, SCM_RIGHTS, abstract Unix sockets, netlink | +| `src/syscall/usbdev.c` | usbdevfs fds over IOKit (see [USB Device Passthrough](#usb-device-passthrough)) | | `src/syscall/translate.c` | errno and shared `AT_*` flag translation | | `src/syscall/proc.c` | vCPU run loop, `wait4`, ptrace coordination, HVC #6 routing | | `src/syscall/exec.c` | `execve`: ELF reload, interpreter resolve, vCPU restart | @@ -605,6 +606,7 @@ waiter enqueue, so the compare-and-wait is a single critical section. | FUSE (sessions, file/dir state) | global `fuse_lock` + per-session `session->lock` | `src/syscall/fuse.c` | | Sysroot snapshot | `pthread_mutex` | `src/syscall/proc-state.c` | | Synthetic USB tree (scratch dirs + device model) | `usb_lock` (leaf) | `src/runtime/usb-sysfs.c` | +| usbdevfs fd side table | `usbdev_table_lock` + per-entry lock | `src/syscall/usbdev.c` | Lock ordering is documented inline in those files (`mmap_lock` is order 1, `fd_lock` is order 3, `sfd_lock` is order 5a) @@ -858,6 +860,172 @@ Validation lives in `make test-fuse-alpine`, which exercises `/dev/fuse` plus `mount("fuse")` against the staged Alpine musl sysroot fixture. +## USB Device Passthrough + +`src/syscall/usbdev.c` implements the usbdevfs character device +(`/dev/bus/usb/BBB/DDD`) on top of IOKit's `IOUSBDeviceInterface650` and +`IOUSBInterfaceInterface800` plugin interfaces, so Linux USB tools drive +real devices attached to the Mac. macOS arbitrates that access per IOKit +object and dynamically: `USBInterfaceOpen` refuses an interface exactly while +some driver holds that interface open, and grants it while that driver is +idle. An interface bound to an idle Apple class driver therefore does open -- +the ESP32-S3's CDC data interface opens whenever nothing holds +`/dev/cu.usbmodem1101` and refuses while something does -- so the layer +attempts the open and maps what IOKit answers rather than deciding from the +presence of a bound driver. + +The fd model: opening a node constructs a typed `FD_USBDEV` fd. The open +consults the model, not the hardware: `stat`, `access` and an `O_PATH` open +of the same name are all answered from the model, so requiring a live IOKit +service here would make a plain `O_RDONLY` the one entry point that reported +`ENODEV` for a node the other three describe. The service is resolved on +first use instead, and every operation that needs the wire reports `ENODEV` +when it is not there. + +Per-fd state (the IOKit device handle, claimed interfaces, and the +endpoint-to-pipe map) lives in a side table keyed by the guest fd and the fd +generation, so a concurrently closed and reallocated guest fd cannot reach +another open's state. A lookup pins its entry under `usbdev_table_lock` and +then takes the per-entry lock with the table lock released: taking the two +nested meant a thread blocked on one fd's transfer held the table lock on +every other thread's behalf, and one transfer with usbdevfs's documented +"unlimited" `timeout == 0` wedged every usbdevfs fd in the process, +`close()` included. Teardown runs from the fd-cleanup hook, which is handed +a bare fd number after the fd-table slot is already free, so a sibling +thread's open can already hold the same number; among the entries that +answer to it the closing one is the one with the smaller generation, since +`fd_alloc` stamps a globally monotonic counter. `fd_alloc` also publishes the +guest fd before the side table can bind it, so a close landing in that window +would find no entry to tear down; the open rereads the fd's generation once its +entry is findable and retires the entry itself if the close has already been +and gone. + +That leaves two windows around the bind, and the entry keeps one identity +across both. Before the bind its `guest_fd` is still -1, so a close finds +nothing; after it, the entry is findable and therefore also freeable, so a +close can reap it, the last unref can free the slot, and a sibling open can +bind its own live fd there before the recheck runs. The generation is the one +`fd_alloc_from` reports from inside the allocating `fd_lock` section, not a +later read of the slot -- read back after the slot was publishable it is +whatever the number carries by then, which in the first window is a reopening +thread's stamp, and two entries then hold the same pair. And the retire path +claims the whole tuple the caller allocated (used and alive, this fd number, +this generation) rather than testing that the slot is not dead, which in the +second window claims the sibling's entry instead. The tuple is sufficient +because slot storage is static, so the read is always defined; all four fields +are written and read under `usbdev_table_lock`, so they are read as one value; +and `fd_next_generation` is globally monotonic, so each entry carries its own +allocation's stamp and no two live entries can present the same pair. + +The entry's host fd is a pipe read end reserved for readiness signaling. +`read()` serves the descriptors blob at a per-open file position, +byte-identical to the sysfs `descriptors` attribute; `SEEK_END` is `EINVAL`, +as on Linux usbfs; `fstat` reports a character device, major 189, answered from +the entry's own identity ahead of the `/dev/bus` path stamp, the way a FUSE +descriptor is answered by the layer that owns it. The whole +write family -- `write`, `writev`, `pwrite`, `pwritev` -- is `EBADF` without +`FMODE_WRITE` and `EINVAL` otherwise, the order `vfs_write` checks +`FMODE_WRITE` and then `FMODE_CAN_WRITE`; none of them may fall through to +the readiness pipe behind the fd. The two capability bits are derived once, +the way `OPEN_FMODE` derives them -- `(flags + 1) & O_ACCMODE`, not a +comparison against `O_RDONLY` -- and the four gates that need them (read, +pread, the write family, and the ioctl surface) read that. Access mode 3 is +the case the comparison gets wrong: `open(2)` takes it and `ACC_MODE(3)` asks +this 0666 node for read plus write, which it grants, so Linux hands back a +descriptor carrying neither `FMODE_READ` nor `FMODE_WRITE` and refuses +everything on it. `dup` of the fd is refused with `EBADF` +(the side table is keyed by the guest fd and IOKit plugin handles are +process-local). + +The ioctl surface at this layer: `CLAIMINTERFACE` / `RELEASEINTERFACE`, +`SETINTERFACE`, `SETCONFIGURATION`, `CLEAR_HALT` / `RESETEP`, `GETDRIVER`, +`GET_CAPABILITIES`, `GET_SPEED`, `CONNECTINFO`, `DISCONNECT_CLAIM`, +`USBDEVFS_IOCTL` `DISCONNECT` / `CONNECT`, and the synchronous `CONTROL` +and `BULK` transfers, which bounce guest data through host buffers around +`DeviceRequestTO` and `ReadPipeTO` / `WritePipeTO`. The transfers here are +synchronous: the guest thread blocks in the ioctl for the duration of the +request. + +errno fidelity is the design rule, because libusb and friends branch on +exact values, and so is the *order* the answers are decided in, which is just +as observable: `check_ctrlrecip` runs before the `wLength` cap and +`findintfep` before the transfer-length checks, so a request naming an +endpoint or interface the device does not have is `ENOENT` however long it +is. The reserved-bit test on an endpoint address lives in the one lookup +`BULK`, `CLEAR_HALT`, `RESETEP` and the control endpoint recipient all go +through, so the four cannot disagree about it. The interface-number bound is +64, `8 * sizeof(unsigned long)` as in `claimintf`; between 64 and 32 an +interface is merely absent, which is a question about the device. + +Every argument that comes off the wire is bounded where it enters, not only +where it is used. `bInterfaceNumber` is a device-supplied byte with the whole +0..255 range behind it, and the endpoint-owner lookup reports it back as the +`EINVAL` `checkintf` reports rather than handing a 64-entry array an index of +200. An endpoint address arrives as a 32-bit word and is tested as one, since +narrowing it to a byte first resolves the transfer to an endpoint the caller +did not name; the control path is the exception Linux itself makes, masking +`wIndex` to its low byte before the lookup. An altsetting above 255 matches no +`bAlternateSetting` and is `EINVAL`, after the implicit claim, the order +`proc_setintf` decides them in. + +The other half of errno fidelity is that a failure keeps the errno of whatever +failed. `ENODEV` from the model, meaning nothing answers to this address, +becomes the `ENOENT` open(2) owes for a name with nothing behind it; every +other failure on the open path -- an allocation, a scratch-tree `mkdir`, a +pipe the host would not give -- reports itself, so a host out of memory is not +described to the guest as a missing node. + +Every ioctl needs `FMODE_WRITE` and is `EPERM` without it (Linux's gate in +`devio.c`). `FIONBIO` and `FIOASYNC` are the exception, because they are not +part of this file's ioctl set at all: `do_vfs_ioctl` answers both for every +file before it reaches `f_op->unlocked_ioctl`, so they never meet that gate. +They are answered where `FIOCLEX` and `FIONCLEX` already are. `FIONBIO` edits +only the guest flag word, the way `F_SETFL` does -- the readiness pipe behind +the fd stays nonblocking whatever the guest asks -- and always reports 0. +`FIOASYNC` reports 0 when the requested `FASYNC` state already holds and +`ENOTTY` when it would change, because `usbdev_file_operations` declares no +`.fasync`. + +Transfer memory is one allowance for everything in flight, not a per-call +size cap: Linux charges `len + sizeof(struct urb)` against a module-global +`usbfs_memory_mb` (16 MB) and refunds it when the transfer settles, so a +request of exactly the allowance never fits and concurrent transfers across +different fds contend for the same total. An atomic counter carries it here, +charged where Linux charges it and refunded on every exit including the error +arms. A claim that IOKit answers `kIOReturnExclusiveAccess` is `EBUSY`, +what Linux reports for an interface held by a kernel driver. `GETDRIVER` +names the bound Apple driver, or `usbfs` for an interface any usbfs fd on +this device holds, or reports `ENODATA`; a user-client child is not a driver, +so another usbfs consumer's `USBInterfaceOpen` is not reported as one. +Transfer errors map per `devio.c`: a stall is `EPIPE`, a timeout +`ETIMEDOUT`, a vanished device `ENODEV`. `kIOReturnAborted` maps to `EINTR` +with `syscall_restart_forbid()`, because the transfer was already on the wire +and a dispatcher restart would send it twice. + +Known gaps at this stage, each printed as an XFAIL by +`tests/test-usbdev-ioctl.c` rather than only written down here: + +- `GET_CAPABILITIES` reports 0. Every capability bit names part of the + SUBMITURB/REAPURB machinery, which answers `ENOTTY` here. +- No disconnect gate. Linux answers `ENODEV` for every ioctl once the device + is gone; the answers served from the open-time model (`GET_SPEED`, + `CONNECTINFO`, `GET_CAPABILITIES`, `read()`) still report it. Noticing the + disconnect needs an IOKit termination notification on a run loop, which is + the async stage's machinery. +- Two ioctls on one fd serialize, because the entry lock is held across the + blocking transfer. Linux drops the device lock around the URB wait. +- `GETDRIVER` reports the IOKit class name (`AppleUSBACMControl`) where Linux + reports the driver's name (`cdc_acm`), and `DISCONNECT_CLAIM`'s name filters + compare against it. +- `RESET` clears the claimed pipes' stalls and reports success instead of + re-enumerating the port, which would destroy the handles. +- A short bulk OUT is `EIO`: `WritePipeTO` reports no length, and reporting + the requested count would spell a partial write as a complete one. +- The side table holds 32 open fds per process and reports `ENOMEM` past + that; Linux allocates a `usb_dev_state` per open and has no such limit. +- `USBDEVFS_IOCTL DISCONNECT` of an interface another usbfs fd holds is + `EBUSY`; Linux releases that claim and answers 0. + ## procfs And Device Emulation `src/runtime/procemu.c` intercepts a focused set of guest-visible paths @@ -986,9 +1154,8 @@ falls back to a usbfs directory scan. The `/dev/bus/usb/BBB/DDD` nodes are 0444 placeholder files on disk (the `/dev/pts` placeholder pattern); the stat intercept reports them as character devices, major 189, minor `(bus - 1) * 128 + (dev - 1)`, and the -open intercept diverts an open away from the placeholder. Opening a node -serves the shared descriptors blob read-only; a writable open reports -`EACCES`. +open intercept diverts an open away from the placeholder into the usbdevfs +fd constructor (see [USB Device Passthrough](#usb-device-passthrough)). One layout deviation is deliberate: the `/sys/bus/usb/devices` entries are real directories, not symlinks into `/sys/devices/...`, so `realpath()` of diff --git a/docs/testing.md b/docs/testing.md index ca91bee4..1eb8fe64 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -438,11 +438,12 @@ bash tests/driver.sh -f test-proc ## Fault Injection And Recorded Rows -Three failures the directory and identity lanes have to pin cannot be provoked -from a test: a `malloc` coming back NULL part-way through a backing listing, a -`readdir` that fails part-way through a directory elfuse itself materialized, -and a slot replaced inside a window that is sub-microsecond wide unaided. Each -has an environment hook, read once and with no effect at all when unset: +Several failures these lanes have to pin cannot be provoked from a test: a +`malloc` coming back NULL part-way through a backing listing, a `readdir` that +fails part-way through a directory elfuse itself materialized, a slot replaced +inside a window that is sub-microsecond wide unaided, and, on the usbdevfs fd, +each of the three ways an open can fail before it returns. Each has an +environment hook, read once and with no effect at all when unset: | Variable | Effect | Driven by | |----------|--------|-----------| @@ -450,12 +451,18 @@ has an environment hook, read once and with no effect at all when unset: | `ELFUSE_DIR_PRIMARY_READ_FAULT=N` | `readdir` on the synthetic stream fails with `EIO` after N entries | `test-dir-primary-read-error` | | `ELFUSE_DIR_UNION_BACKING_DELAY_US=N` | widens the window between pinning a directory stream and looking its backing up | `test-dir-union-fd-reuse` | | `ELFUSE_FD_IDENTITY_WINDOW_US=N` | widens the window between reading a descriptor's stamp and pinning its host fd | `test-fstatfs-fd-identity` | +| `ELFUSE_USBDEV_OPEN_FAULT=info\|blob\|pipe` | fails one step of a usbdevfs open: the model lookup or the descriptor copy with `ENOMEM`, the readiness pipe with `ENFILE` | `test-usbdev-faults` | +| `ELFUSE_USBDEV_PUBLISH_DELAY_US=N` | widens the window between `fd_alloc` publishing a usbdevfs fd and the side table binding it, where a close finds no entry | `test-usbdev-faults` | +| `ELFUSE_USBDEV_RETIRE_DELAY_US=N` | widens the window between the side table binding a usbdevfs fd and the open's recheck, where a close can reap the entry and a sibling open can take its slot | `test-usbdev-faults` | `ELFUSE_USB_FIXTURE` is the same shape pointing at enumeration rather than failure: it stands a deterministic synthetic USB tree up in place of whatever IOKit reports, so the lanes below have devices to walk on any host. `ELFUSE_USB_FIXTURE=overflow` stands up 129 address-less devices on one bus for -the `devnum` cap. +the `devnum` cap. `ELFUSE_USB_FIXTURE=badifnum` adds one device whose only +interface declares `bInterfaceNumber` 200: every byte is well formed and the +range is the field's own, but nothing anyone can plug in emits it, so it is the +only way to reach the paths that index by that number. The lanes these drive: @@ -469,6 +476,7 @@ The lanes these drive: | `test-dir-union-alias` | every route to a second fd on one description shares one position and one union state | | `test-dir-fd-budget-union` | a union directory fd costs one host descriptor, like a plain one | | `test-fstatfs-fd-identity` | `fstatfs` answers for the descriptor it pinned, not for the fd number | +| `test-usbdev-faults` | an interface number wider than the table that indexes it, each open-time failure reported as itself, and a close inside the fd publish window leaking nothing | Two lanes carry rows that are recorded rather than asserted, and print as `XFAIL`. An `XFAIL` row is a measured Linux value the build knowingly does not diff --git a/mk/tests.mk b/mk/tests.mk index 12c2251d..63f84493 100644 --- a/mk/tests.mk +++ b/mk/tests.mk @@ -43,7 +43,8 @@ ELFUSE_HOST_NOFILE_MIN ?= $(shell bash "$(CURDIR)/tests/test-config.sh" --host-n test-nosysroot-literal-names test-sysroot-outside-names \ test-sysroot-root test-usb-sysfs test-usb-sysfs-sysroot \ test-usb-sysfs-matrix \ - test-usb-sysfs-overflow test-dir-fd-budget-union \ + test-usb-sysfs-overflow test-usbdev-ioctl test-usbdev-faults \ + test-dir-fd-budget-union \ test-dir-backing-drain-error test-dir-union-fd-reuse \ test-fstatfs-fd-identity \ test-dir-union-alias test-dir-primary-read-error \ @@ -266,6 +267,8 @@ $(call run-lane,test-usb-sysfs,synthetic USB tree contract) $(call run-lane,test-usb-sysfs-sysroot,synthetic USB /sys sharing a populated sysroot) $(call run-lane,test-usb-sysfs-matrix,every /sys and /dev/bus entry point against every path class) $(call run-lane,test-usb-sysfs-overflow,per-bus devnum cap under 127-device overflow) +$(call run-lane,test-usbdev-ioctl,the usbdevfs fd contract without hardware) +$(call run-lane,test-usbdev-faults,the usbdevfs fd's forced failures) $(call run-lane,test-dir-fd-budget-union,a union directory fd costs one host descriptor) $(call run-lane,test-dir-backing-drain-error,a lost union listing is reported not truncated) $(call run-lane,test-dir-union-fd-reuse,a union walk answers for the directory it pinned) @@ -1625,6 +1628,10 @@ test-casefold-walk-host: $(BUILD_DIR)/test-casefold-walk-host ## Assert the synthetic USB tree's contract and its two agreeing views # The device-dependent half only runs against whatever is attached, so the lane # prints the device count rather than letting an empty bus read as full cover. +# The fixture stays on: stage 2's FD_USBDEV constructor resolves its IOKit +# device on first use rather than at open, so a modeled device with no hardware +# behind it still opens, reads and stats like one -- which is what keeps this +# lane's device half running on a machine with no USB device attached. test-usb-sysfs: $(ELFUSE_BIN) $(TEST_DIR)/test-usb-sysfs ELFUSE_USB_FIXTURE=1 $(ELFUSE_BIN) $(TEST_DIR)/test-usb-sysfs @@ -1684,6 +1691,40 @@ test-usb-sysfs-matrix: $(ELFUSE_BIN) $(TEST_DIR)/test-usb-sysfs-matrix test-usb-sysfs-overflow: $(ELFUSE_BIN) $(TEST_DIR)/test-usb-sysfs-overflow ELFUSE_USB_FIXTURE=overflow $(ELFUSE_BIN) $(TEST_DIR)/test-usb-sysfs-overflow +## The usbdevfs fd's contract, decided before anything reaches the wire +# The fixture's devices are modeled with no IOKit service behind them, which is +# exactly the shape of a device the machine cannot reach: every answer below is +# the same on a host with no USB device attached and on one with a bus full of +# them, so the file contract, the ioctl gates and the whole argument-validation +# surface get a lane that does not depend on what is plugged in. +test-usbdev-ioctl: $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ELFUSE_USB_FIXTURE=1 $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + +## The usbdevfs fd's failures, one forced condition per run +# Six things this descriptor has to get right cannot be provoked from a guest on +# a healthy host: a device declaring an interface number wider than the table +# that indexes by it, the three ways an open can fail before it returns, and the +# two windows an open leaves around the moment the side table binds its guest +# fd -- before the bind, where a close finds no entry, and after it, where a +# close can reap the entry and a sibling open can take the slot back. Each run +# below forces exactly one and the binary asserts only that one, so a failure +# names the condition. The malformed-descriptor run is also where the +# out-of-bounds read lives: it is invisible in the answer -- both sides report +# EINVAL, which is what checkintf reports -- and shows up only under +# -fsanitize=array-bounds, which is why this lane is in the sanitizer set. +test-usbdev-faults: $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ELFUSE_USB_FIXTURE=badifnum $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ELFUSE_USB_FIXTURE=1 ELFUSE_USBDEV_OPEN_FAULT=info \ + $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ELFUSE_USB_FIXTURE=1 ELFUSE_USBDEV_OPEN_FAULT=blob \ + $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ELFUSE_USB_FIXTURE=1 ELFUSE_USBDEV_OPEN_FAULT=pipe \ + $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ELFUSE_USB_FIXTURE=1 ELFUSE_USBDEV_PUBLISH_DELAY_US=20000 \ + $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ELFUSE_USB_FIXTURE=1 ELFUSE_USBDEV_RETIRE_DELAY_US=20000 \ + $(ELFUSE_BIN) $(TEST_DIR)/test-usbdev-ioctl + ## fstatfs answers for the descriptor it pinned, not for the fd number # The identity is decided from the slot's stamp and from the descriptor itself, # and those used to be two lookups with a window between them. The window is far diff --git a/scripts/check-eintr-contract.py b/scripts/check-eintr-contract.py index ffbeaea9..fd48c4e7 100644 --- a/scripts/check-eintr-contract.py +++ b/scripts/check-eintr-contract.py @@ -112,6 +112,11 @@ "FUSE_INTERRUPT is on the wire and the request is detached, so a " "restart would re-issue the operation under a fresh unique.", ), + "syscall/usbdev.c::ioret_neg_errno": ( + "forbids", + "kIOReturnAborted means a sync transfer already handed to IOKit was " + "aborted mid-flight; a restart would send the request twice.", + ), # Waits that report EINTR before doing anything the guest can observe. "syscall/io.c::io_retry_backoff": ( "restartable", diff --git a/src/runtime/usb-desc.h b/src/runtime/usb-desc.h index c516a289..2979f227 100644 --- a/src/runtime/usb-desc.h +++ b/src/runtime/usb-desc.h @@ -25,6 +25,7 @@ #define USB_DT_DEVICE 1 #define USB_DT_CONFIG 2 #define USB_DT_INTERFACE 4 +#define USB_DT_ENDPOINT 5 /* Shortest descriptor that can carry a type: bLength + bDescriptorType. */ #define USB_DESC_MIN_LEN 2 @@ -33,6 +34,7 @@ #define USB_DEVICE_DESC_LEN 18 #define USB_CONFIG_DESC_LEN 9 #define USB_INTERFACE_DESC_LEN 9 +#define USB_ENDPOINT_DESC_LEN 7 typedef struct { const uint8_t *buf; diff --git a/src/runtime/usb-sysfs.c b/src/runtime/usb-sysfs.c index 494fd513..a811e3be 100644 --- a/src/runtime/usb-sysfs.c +++ b/src/runtime/usb-sysfs.c @@ -24,11 +24,11 @@ * canonicalizes to itself, which libusb (opens attrs relative to the entry) and * nusb (canonicalize() of the entry path) both tolerate. * - * Stage-1 open contract for /dev/bus/usb/BBB/DDD: O_RDONLY returns a synthetic - * host fd holding the descriptors blob (matching the usbfs read() view, - * devio.c:311-390); O_RDWR/O_WRONLY fails with EACCES. TEMPORARY: stage 2 - * replaces this constructor with a typed FD_USBDEV fd whose read() serves the - * same blob via usb_sysfs_descriptors_dup and whose ioctl set talks to IOKit. + * /dev/bus/usb/BBB/DDD opens: since stage 2, every non-O_PATH open is served by + * the typed FD_USBDEV constructor (syscall/usbdev.c) before this intercept + * runs; the node branch here only backs O_PATH opens with a synthetic blob fd + * (the FD_PATH + stat-stamp path). The blob it serves and the FD_USBDEV read() + * view are the same bytes (usb_sysfs_descriptors_dup). */ #include @@ -297,6 +297,16 @@ typedef struct { unsigned vid; unsigned pid; unsigned nifaces; + + /* bInterfaceNumber of the first interface; each further one counts up from + * it. Normally 0, so the numbers are 0, 1, ... and match the array + * positions. The knob exists because bInterfaceNumber is a device-supplied + * byte with the whole 0..255 range behind it while the consumers of it are + * sized for far fewer, and no device anyone can plug in declares a large + * one, so without a fixture that can emit one there is no way to assert + * what happens when a device does. + */ + unsigned ifnum_base; } usb_fixture_spec_t; /* The number of interface descriptors is the only thing that varies the blob @@ -306,7 +316,7 @@ typedef struct { static size_t usb_fixture_blob_len(unsigned nifaces) { return USB_DEVICE_DESC_LEN + USB_CONFIG_DESC_LEN + - (size_t) nifaces * USB_INTERFACE_DESC_LEN; + (size_t) nifaces * (USB_INTERFACE_DESC_LEN + USB_ENDPOINT_DESC_LEN); } /* Fill @d from @s, writing the device, configuration and interface descriptors @@ -337,7 +347,8 @@ static void usb_fixture_fill(usb_dev_t *d, const usb_fixture_spec_t *s) snprintf(d->name, sizeof(d->name), "%d-%d", s->busnum, s->port); size_t cfg_total = - USB_CONFIG_DESC_LEN + (size_t) s->nifaces * USB_INTERFACE_DESC_LEN; + USB_CONFIG_DESC_LEN + + (size_t) s->nifaces * (USB_INTERFACE_DESC_LEN + USB_ENDPOINT_DESC_LEN); build_device_descriptor(d, blob); uint8_t *c = blob + USB_DEVICE_DESC_LEN; c[0] = USB_CONFIG_DESC_LEN; @@ -351,16 +362,34 @@ static void usb_fixture_fill(usb_dev_t *d, const usb_fixture_spec_t *s) c[8] = 50; /* bMaxPower: 100 mA */ for (unsigned i = 0; i < s->nifaces; i++) { uint8_t *q = - c + USB_CONFIG_DESC_LEN + (size_t) i * USB_INTERFACE_DESC_LEN; + c + USB_CONFIG_DESC_LEN + + (size_t) i * (USB_INTERFACE_DESC_LEN + USB_ENDPOINT_DESC_LEN); q[0] = USB_INTERFACE_DESC_LEN; q[1] = USB_DT_INTERFACE; - q[2] = (uint8_t) i; /* bInterfaceNumber */ - q[3] = 0; /* bAlternateSetting */ - q[4] = 1; /* bNumEndpoints */ - q[5] = 0xff; /* bInterfaceClass: vendor-specific */ + unsigned ifnum = s->ifnum_base + i; + q[2] = (uint8_t) ifnum; /* bInterfaceNumber */ + q[3] = 0; /* bAlternateSetting */ + q[4] = 1; /* bNumEndpoints */ + q[5] = 0xff; /* bInterfaceClass: vendor-specific */ q[6] = 0x00; q[7] = 0x00; q[8] = 0; + + /* The endpoint the interface descriptor above says it has. Without it + * the blob was self-contradictory, and every endpoint-addressed + * usbdevfs path (BULK, CLEAR_HALT, RESETEP, the control endpoint + * recipient) had nothing to resolve against, so the fixture could not + * reach the code that decides between "no such endpoint" and "bad + * argument". Bulk IN, one per interface: 0x81, 0x82, ... + */ + uint8_t *e = q + USB_INTERFACE_DESC_LEN; + e[0] = USB_ENDPOINT_DESC_LEN; + e[1] = USB_DT_ENDPOINT; + e[2] = (uint8_t) (0x81 + i); /* bEndpointAddress: bulk IN */ + e[3] = 0x02; /* bmAttributes: bulk */ + e[4] = 0x40; /* wMaxPacketSize: 64 */ + e[5] = 0x00; + e[6] = 0; /* bInterval */ } } @@ -378,6 +407,14 @@ static void usb_fixture_fill(usb_dev_t *d, const usb_fixture_spec_t *s) * regression fixture for the devnum cap -- without the cap bus1's 129th device * takes devnum 129 and shares minor 128 with bus2's first, so the cap must drop * everything past devnum 127. + * + * ELFUSE_USB_FIXTURE=badifnum: the default set plus /dev/bus/usb/001/002, whose + * one interface declares bInterfaceNumber 200 and carries endpoint 0x81. It is + * a malformed descriptor only in the sense that no sane device emits one: every + * byte is well formed and the range is the field's own, which is why nothing + * short of a fixture reaches the paths that index by that number. Added as a + * separate mode rather than to the default set so the lanes that walk the tree + * keep the device list they were written against. */ static int usb_fixture_specs(usb_fixture_spec_t *specs, int cap) { @@ -385,15 +422,18 @@ static int usb_fixture_specs(usb_fixture_spec_t *specs, int cap) int n = 0; if (mode && !strcmp(mode, "overflow")) { for (int port = 1; port <= 129 && n < cap; port++) - specs[n++] = (usb_fixture_spec_t) {1, port, 0, 0x1d6b, 0x0002, 1}; + specs[n++] = + (usb_fixture_spec_t) {1, port, 0, 0x1d6b, 0x0002, 1, 0}; if (n < cap) - specs[n++] = (usb_fixture_spec_t) {2, 1, 0, 0x2109, 0x0100, 1}; + specs[n++] = (usb_fixture_spec_t) {2, 1, 0, 0x2109, 0x0100, 1, 0}; return n; } if (n < cap) - specs[n++] = (usb_fixture_spec_t) {1, 1, 1, 0x1d6b, 0x0002, 2}; + specs[n++] = (usb_fixture_spec_t) {1, 1, 1, 0x1d6b, 0x0002, 2, 0}; if (n < cap) - specs[n++] = (usb_fixture_spec_t) {2, 1, 1, 0x2109, 0x0100, 1}; + specs[n++] = (usb_fixture_spec_t) {2, 1, 1, 0x2109, 0x0100, 1, 0}; + if (mode && !strcmp(mode, "badifnum") && n < cap) + specs[n++] = (usb_fixture_spec_t) {1, 2, 2, 0x1d6b, 0x0002, 1, 200}; return n; } @@ -1327,10 +1367,15 @@ static void fill_synth_chardev(struct stat *st, { memset(st, 0, sizeof(*st)); - /* 0664 rather than udev's root:root 0660 policy: the single-user guest must - * be able to open its own device nodes. + /* 0666 rather than udev's root:root 0660 policy: the single-user guest must + * be able to open its own device nodes, which is what a desktop Linux + * spells as a uaccess ACL for the seat owner. The owner reported below is + * the host user, and the guest's own uid need not equal it, so 0664 left + * access(W_OK) answering EACCES for a node that open(O_RDWR) then served -- + * the two entry points onto one permission question disagreeing. Stage 1 + * had no writable open to disagree with; stage 2 does. */ - st->st_mode = S_IFCHR | 0664; + st->st_mode = S_IFCHR | 0666; st->st_nlink = 1; st->st_dev = USB_SYNTH_DEV; st->st_ino = usb_synth_ino(canon); @@ -1868,12 +1913,14 @@ int usb_sysfs_intercept_open(const char *path, int linux_flags, int mode) } int accmode = translate_open_flags(linux_flags) & O_ACCMODE; if (accmode != O_RDONLY) { - /* TEMPORARY (stage 1): writable opens are what stage 2's FD_USBDEV - * constructor will serve; until then they fail. + /* Unreachable through sys_openat_path: usbdev_open_path claims + * every non-O_PATH open of a node before proc_intercept_open runs + * (stage 2, syscall/usbdev.c). Kept as a guard for any other caller + * of the intercept. */ log_warn( - "usb-sysfs: O_RDWR open of %s not implemented yet " - "(stage 2)", + "usb-sysfs: writable open of %s bypassed the FD_USBDEV " + "constructor", path); err = EACCES; goto out; @@ -2227,3 +2274,47 @@ uint8_t *usb_sysfs_descriptors_dup(int busnum, int devnum, size_t *len_out) pthread_mutex_unlock(&usb_lock); return copy; } + +int usb_sysfs_device_info(int busnum, int devnum, usb_sysfs_devinfo_t *out) +{ + pthread_mutex_lock(&usb_lock); + int rc = -1; + if (ensure_usb_tree() == 0) { + usb_dev_t *d = find_dev(busnum, devnum); + if (d) { + out->location_id = d->location_id; + out->speed_code = d->speed_code; + out->cfg_value = d->cfg_value; + out->minor = usb_minor(d); + out->blob_len = d->blob_len; + out->vid = d->vid; + out->pid = d->pid; + str_copy_trunc(out->serial, d->serial, sizeof(out->serial)); + rc = 0; + } else { + errno = ENODEV; + } + } + pthread_mutex_unlock(&usb_lock); + return rc; +} + +int usb_sysfs_node_stat(int busnum, int devnum, struct stat *st) +{ + pthread_mutex_lock(&usb_lock); + int rc = -1; + if (ensure_usb_tree() == 0) { + usb_dev_t *d = find_dev(busnum, devnum); + if (d) { + char node[64]; + snprintf(node, sizeof(node), "/dev/bus/usb/%03d/%03d", busnum, + devnum); + fill_synth_chardev(st, node, d); + rc = 0; + } else { + errno = ENODEV; + } + } + pthread_mutex_unlock(&usb_lock); + return rc; +} diff --git a/src/runtime/usb-sysfs.h b/src/runtime/usb-sysfs.h index e230fbd0..55aae172 100644 --- a/src/runtime/usb-sysfs.h +++ b/src/runtime/usb-sysfs.h @@ -101,3 +101,34 @@ uint8_t *usb_sysfs_descriptors_dup(int busnum, int devnum, size_t *len_out); * tree (including when the tree does not exist). */ int usb_sysfs_guest_path_for_fd(int host_fd, char *out, size_t outsz); + +/* Identity snapshot of one enumerated device, for the stage-2 usbdevfs fd + * (syscall/usbdev.c): location_id keys the IOKit service lookup, speed_code is + * the raw registry 'Device Speed' code, cfg_value the active + * bConfigurationValue, minor the usbfs char-dev minor. vid/pid/serial carry the + * modeled identity so the fd constructor can verify the service it looked up by + * location is still the device this bus/dev number was modeled from (locationID + * names the port, not the device). + */ +typedef struct { + uint32_t location_id; + unsigned speed_code; + unsigned cfg_value; + int minor; + size_t blob_len; + unsigned vid, pid; + char serial[128]; /* "" when the device reports none */ +} usb_sysfs_devinfo_t; + +/* Fill *out for the device at busnum/devnum. + * + * Returns 0, or -1 with errno set (ENODEV when no such device). + */ +int usb_sysfs_device_info(int busnum, int devnum, usb_sysfs_devinfo_t *out); + +/* Synthesize the char-dev stat for /dev/bus/usb/BBB/DDD (same bytes the path + * stat intercept reports). + * + * Returns 0, or -1 with errno set. + */ +int usb_sysfs_node_stat(int busnum, int devnum, struct stat *st); diff --git a/src/syscall/fs-stat.c b/src/syscall/fs-stat.c index a1414ef5..910f6783 100644 --- a/src/syscall/fs-stat.c +++ b/src/syscall/fs-stat.c @@ -24,6 +24,7 @@ #include "syscall/fuse.h" #include "syscall/fs.h" #include "syscall/internal.h" +#include "syscall/usbdev.h" #include "syscall/path.h" #include "syscall/proc.h" #include "utils.h" @@ -241,6 +242,20 @@ static int64_t stat_empty_path_fd(int dirfd, struct stat *mac_st) } int64_t rc = 0; + + /* usbdevfs fds are char device 189:minor and the host fd behind them is a + * pipe, whose fstat must not leak through. Ahead of the stamp branch for + * the same reason the FUSE shim is ahead of both: the descriptor is + * answered by the emulation layer that owns it, not by re-resolving the + * name it was opened under. Behind it the branch was unreachable -- every + * FD_USBDEV fd carries a /dev/bus stamp, so fd_stat_answers_from_stamp + * claimed all of them and answered from the path. + */ + if (snap.type == FD_USBDEV) { + rc = usbdev_fstat(dirfd, mac_st); + goto done; + } + if (fd_stat_answers_from_stamp(&snap)) { /* The descriptor already names one object: an O_PATH open that followed * refers to the target, one made with O_NOFOLLOW to the link itself, so @@ -256,6 +271,7 @@ static int64_t stat_empty_path_fd(int dirfd, struct stat *mac_st) goto done; } } + if (fstat(ref.fd, mac_st) < 0) rc = linux_errno(); diff --git a/src/syscall/fs.c b/src/syscall/fs.c index 8a16e4a4..b9244bd4 100644 --- a/src/syscall/fs.c +++ b/src/syscall/fs.c @@ -51,6 +51,7 @@ _Static_assert(NAME_MAX == DIRENT64_NAME_MAX, #include "syscall/io.h" /* io_retry_backoff */ #include "syscall/net.h" /* absock_unregister_fd */ #include "syscall/path.h" +#include "syscall/usbdev.h" #include "syscall/poll.h" /* epoll_dup_fd */ #include "syscall/proc.h" @@ -884,6 +885,13 @@ int64_t sys_openat_path(guest_t *g, if (path_might_use_open_intercept(tx.intercept_path)) { if (!strcmp(tx.intercept_path, "/dev/fuse")) return fuse_proc_open(linux_flags); + + /* /dev/bus/usb/BBB/DDD: typed FD_USBDEV constructor (any access mode + * except O_PATH, which the stage-1 placeholder serves). + */ + int64_t usb_fd = usbdev_open_path(tx.intercept_path, linux_flags); + if (usb_fd != INT64_MIN) + return usb_fd; int64_t fuse_fd = fuse_open_path(g, tx.intercept_path, linux_flags, mode); if (fuse_fd != INT64_MIN) @@ -1302,6 +1310,18 @@ static int duplicate_guest_fd(int src_fd, linux_flags); } + /* TODO(stage 3): explicit usbdev alias sharing the side-table entry + * (fuse_dup_fd pattern). Refusing the dup beats handing out an alias whose + * ioctls would miss the side table keyed by the original fd. + */ + if (src_snap.type == FD_USBDEV) { + proc_pty_unlock_for_dup(); + if (new_host_fd >= 0) + close_keep_errno(new_host_fd); + errno = EBADF; + return -1; + } + /* eventfd dup must share the underlying counter and pipe state across the * source and destination fds (Linux contract). Pass src_snap's identity * through so eventfd_dup_fd can reject a close+reopen ABA between the @@ -2902,6 +2922,9 @@ int64_t sys_pipe2(guest_t *g, uint64_t fds_gva, int linux_flags) int64_t sys_lseek(int fd, int64_t offset, int whence) { int64_t frc = fuse_lseek_fd(fd, offset, whence); + if (frc != INT64_MIN) + return frc; + frc = usbdev_lseek_fd(fd, offset, whence); if (frc != INT64_MIN) return frc; diff --git a/src/syscall/internal.h b/src/syscall/internal.h index f21c6016..97e1796f 100644 --- a/src/syscall/internal.h +++ b/src/syscall/internal.h @@ -113,6 +113,21 @@ * which is what pins a session against * daemon exit while a request is in flight * inotify_lock (syscall/inotify.c): inotify watch table + * usbdev_table_lock (syscall/usbdev.c): FD_USBDEV side table. It is never + * held together with the per-entry usbdev + * lock: usbdev_acquire and + * usbdev_fd_cleanup both find their entry + * under the table lock, drop it, and only + * then take the entry lock, because a sync + * transfer can hold that lock for a whole + * timeout and no other fd may wait behind + * it. What keeps the slot from being torn + * down and reused in between is the refs + * and dead pair the lookup sets under the + * table lock, not a nesting. Never held + * together with any other file-scope lock + * in this list either, in either direction, + * so its position here is nominal * * Leaves. Each of these is the innermost lock on every path that takes it, so * it has no position in the order above and cannot be half of an inversion: @@ -608,7 +623,7 @@ static inline bool fd_type_is_synthetic(int type) { return type == FD_EVENTFD || type == FD_SIGNALFD || type == FD_TIMERFD || type == FD_INOTIFY || type == FD_NETLINK || type == FD_PIDFD || - type == FD_EPOLL; + type == FD_EPOLL || type == FD_USBDEV; } /* The status bits the host description is authoritative for. Everything outside diff --git a/src/syscall/io.c b/src/syscall/io.c index cea4d2c9..5c819089 100644 --- a/src/syscall/io.c +++ b/src/syscall/io.c @@ -52,6 +52,7 @@ #include "syscall/net.h" #include "syscall/net-identity.h" #include "syscall/net-sockopt.h" +#include "syscall/usbdev.h" #include "syscall/proc.h" #include "syscall/signal.h" #include "syscall/wakeup-pipe.h" @@ -1533,6 +1534,12 @@ int64_t sys_write(guest_t *g, int fd, uint64_t buf_gva, uint64_t count) if (type == FD_NETLINK) return netlink_send(fd, g, buf_gva, count); + /* usbdevfs has no write op. Falling through would scribble on the readiness + * pipe's read end. + */ + if (type == FD_USBDEV) + return usbdev_write_refused(fd); + host_fd_ref_t host_ref; fd_block_state_t write_st; int64_t err = host_fd_ref_open_checked(fd, &host_ref, &write_st); @@ -1612,6 +1619,8 @@ int64_t sys_read(guest_t *g, int fd, uint64_t buf_gva, uint64_t count) return netlink_read(fd, g, buf_gva, count); case FD_URANDOM: return urandom_read(g, fd, buf_gva, count); + case FD_USBDEV: + return usbdev_read(fd, g, buf_gva, count); } /* Pin the generation in the same fd_lock window as the host fd. The pty @@ -1690,6 +1699,12 @@ int64_t sys_pread64(guest_t *g, if (fuse_is_file_fd(fd)) return fuse_pread_fd(g, fd, buf_gva, count, offset); + /* usbdevfs serves its descriptors blob positionally too: every read path on + * the fd must agree, and the host fd behind it is a readiness pipe. + */ + if (fd_get_type(fd) == FD_USBDEV) + return usbdev_pread(fd, g, buf_gva, count, offset); + host_fd_ref_t host_ref; int64_t err = host_fd_ref_open_io(fd, &host_ref); if (err < 0) @@ -1726,6 +1741,14 @@ int64_t sys_pwrite64(guest_t *g, uint64_t count, int64_t offset) { + /* ksys_pwrite64 tests pos < 0 before it looks the descriptor up + * (read_write.c), so a malformed offset outranks the EBADF a usbdevfs fd + * opened O_RDONLY answers for having no write op at all. usbdev_pread + * already ordered the two this way; the write side did not, and the two + * halves of the same rule disagreed. + */ + if (fd_get_type(fd) == FD_USBDEV) + return offset < 0 ? -LINUX_EINVAL : usbdev_write_refused(fd); host_fd_ref_t host_ref; int64_t err = host_fd_ref_open_checked(fd, &host_ref, NULL); if (err < 0) @@ -1913,6 +1936,18 @@ static int64_t vec_zero_iovcnt(int fd, bool op_is_write, bool positional) if (!fd_snapshot(fd, &snap) || snap.type == FD_PATH) return -LINUX_EBADF; + /* usbdevfs is the one type whose host fd says nothing useful and whose + * guest-visible mode is nonetheless known: the fd sits on a readiness pipe, + * and the open mode lives in the slot's linux_flags. It is seekable, so + * only the direction test applies, and it is answered by the same rule the + * non-empty calls use rather than by a second opinion. + */ + if (snap.type == FD_USBDEV) { + if (op_is_write) + return usbdev_write_refused(fd); + return usbdev_read_refused(fd); + } + bool host_mode_mirrors_guest = snap.type == FD_REGULAR || snap.type == FD_DIR || snap.type == FD_PIPE || snap.type == FD_SOCKET || @@ -1938,6 +1973,61 @@ static int64_t vec_zero_iovcnt(int fd, bool op_is_write, bool positional) return ret; } + +/* readv()/preadv() on an FD_USBDEV fd. Linux's usbdev file has only a plain + * read op, so vectored reads take do_loop_readv_writev: one read per iovec + * entry, stopping at the first short transfer. positional keeps the fd position + * untouched and reads at offset plus the bytes already copied; otherwise each + * usbdev_read advances the fd position like read(2). + */ +static int64_t usbdev_vec_read(guest_t *g, + int fd, + uint64_t iov_gva, + int iovcnt, + int64_t offset, + bool positional) +{ + if (positional && offset < 0) + return -LINUX_EINVAL; /* do_preadv: before the fd lookup */ + if (!iov_count_ok(iovcnt)) + return -LINUX_EINVAL; + + linux_iovec_t stack_giov[SYSCALL_IOV_STACK_MAX]; + linux_iovec_t *giov = stack_giov; + linux_iovec_t *heap_giov = NULL; + if (iovcnt > SYSCALL_IOV_STACK_MAX) { + heap_giov = malloc((size_t) iovcnt * sizeof(*giov)); + if (!heap_giov) + return -LINUX_ENOMEM; + giov = heap_giov; + } + int64_t ret = validate_iov_total(g, iov_gva, iovcnt, giov); + if (ret == 0) { + for (int i = 0; i < iovcnt; i++) { + if (giov[i].iov_len == 0) + continue; + int64_t got = + positional + ? usbdev_pread(fd, g, giov[i].iov_base, giov[i].iov_len, + offset + ret) + : usbdev_read(fd, g, giov[i].iov_base, giov[i].iov_len); + if (got < 0) { + ret = ret > 0 ? ret : got; + break; + } + ret += got; + + /* A short entry ends the transfer; POSIX forbids packing the tail + * of entry i into entry i+1. + */ + if ((uint64_t) got < giov[i].iov_len) + break; + } + } + free(heap_giov); + return ret; +} + int64_t sys_readv(guest_t *g, int fd, uint64_t iov_gva, int iovcnt) { if (iovcnt == 0) @@ -2035,6 +2125,8 @@ int64_t sys_readv(guest_t *g, int fd, uint64_t iov_gva, int iovcnt) return -LINUX_EFAULT; return sys_read(g, fd, giov.iov_base, giov.iov_len); } + if (type == FD_USBDEV) + return usbdev_vec_read(g, fd, iov_gva, iovcnt, 0, false); host_fd_ref_t host_ref; uint64_t readv_gen; @@ -2098,6 +2190,11 @@ int64_t sys_readv(guest_t *g, int fd, uint64_t iov_gva, int iovcnt) int64_t sys_writev(guest_t *g, int fd, uint64_t iov_gva, int iovcnt) { + /* Ahead of every shortcut below, including the empty-vector one: + * do_iter_write refuses on the file's mode before it looks at the vector. + */ + if (fd_get_type(fd) == FD_USBDEV) + return usbdev_write_refused(fd); if (iovcnt == 0) return vec_zero_iovcnt(fd, true, false); @@ -2196,6 +2293,8 @@ int64_t sys_preadv(guest_t *g, return err; return sys_pread64(g, fd, giov.iov_base, giov.iov_len, offset); } + if (fd_get_type(fd) == FD_USBDEV) + return usbdev_vec_read(g, fd, iov_gva, iovcnt, offset, true); host_fd_ref_t host_ref; int64_t err = host_fd_ref_open_io(fd, &host_ref); @@ -2230,6 +2329,12 @@ int64_t sys_pwritev(guest_t *g, int iovcnt, int64_t offset) { + /* do_pwritev tests pos < 0 before the fd lookup (read_write.c), so the + * negative-offset answer outranks the descriptor's own, empty vector or + * not. This was already the order for iovcnt == 0 below. + */ + if (fd_get_type(fd) == FD_USBDEV) + return offset < 0 ? -LINUX_EINVAL : usbdev_write_refused(fd); if (iovcnt == 0) { /* Same ordering as sys_preadv: negative offset EINVAL first. */ if (offset < 0) @@ -2365,6 +2470,15 @@ int64_t sys_pwritev2(guest_t *g, return -LINUX_EINVAL; return vec_zero_iovcnt(fd, true, offset != -1); } + + /* do_iter_write refuses a descriptor with no write op before + * kiocb_set_rw_flags ever sees the RWF bits (read_write.c), so usbdevfs + * answers ahead of the flags. Without this the RWF_APPEND arm below was the + * one write path with no usbdevfs dispatch: it reached the readiness pipe + * and answered ESPIPE for a descriptor that owes EBADF or EINVAL. + */ + if (fd_get_type(fd) == FD_USBDEV) + return offset < -1 ? -LINUX_EINVAL : usbdev_write_refused(fd); if (flags & ~RWF_SUPPORTED) return -LINUX_EOPNOTSUPP; int64_t r; @@ -2593,6 +2707,59 @@ int64_t sys_ioctl(guest_t *g, int fd, uint64_t request, uint64_t arg) return 0; } + /* usbdevfs fds answer their own ioctl set; the host fd behind them is a + * readiness pipe, so nothing below applies. usbdev_ioctl re-snapshots the + * fd and pins the side-table entry by generation itself. + * + * FIONBIO and FIOASYNC are not part of that set. do_vfs_ioctl answers both + * for every file before it ever calls f_op->unlocked_ioctl + * (fs/ioctl.c:818-822), so on Linux they never reach usbdevfs at all -- + * which means they also never meet its FMODE_WRITE gate. Sent into + * usbdev_ioctl they came back EPERM on an O_RDONLY fd and ENOTTY on a + * writable one, while fcntl(F_SETFL, O_NONBLOCK) on the same descriptor + * succeeded: two entry points onto one flag, disagreeing. Answered here, + * where FIOCLEX and FIONCLEX are already answered, they sit exactly where + * the kernel puts them relative to the file's own ioctl handler, and no + * other fd type's path changes. + */ + if (fd_get_type(fd) == FD_USBDEV) { + if (request == LINUX_FIONBIO || request == LINUX_FIOASYNC) { + /* get_user runs first in both kernel helpers, so a bad argument + * pointer outranks everything else, the access mode included. + */ + int32_t on = 0; + if (guest_read_small(g, arg, &on, sizeof(on)) < 0) + return -LINUX_EFAULT; + if (request == LINUX_FIONBIO) { + /* ioctl_fionbio only edits f_flags (fs/ioctl.c:342-363): it + * asks the file nothing and always reports success. The guest + * flag word is where F_GETFL and the readiness paths read this + * fd's O_NONBLOCK from, and the readiness pipe behind it must + * stay nonblocking whatever the guest asks, so the request goes + * to the shadow -- the same place F_SETFL sends it. + */ + if (!fd_apply_guest_nonblock(fd, on != 0)) + return -LINUX_EBADF; + return 0; + } + + /* ioctl_fioasync (fs/ioctl.c:365-385) consults f_op->fasync only + * when the request would change the FASYNC state, and + * usbdev_file_operations declares no .fasync (devio.c:2846-2856). + * So a request for the state the fd already has is 0, and a request + * to change it is ENOTTY -- and it must not reach the O_ASYNC arm + * below, which would arm a SIGIO watcher on a file the kernel + * refuses to arm at all. + */ + fd_entry_t snap; + if (!fd_snapshot(fd, &snap)) + return -LINUX_EBADF; + bool armed = (snap.linux_flags & LINUX_O_ASYNC) != 0; + return armed == (on != 0) ? 0 : -LINUX_ENOTTY; + } + return usbdev_ioctl(g, fd, request, arg); + } + if (request == LINUX_SIOCGIFHWADDR) { fd_entry_t snap; if (!fd_snapshot(fd, &snap)) diff --git a/src/syscall/linux-wire.h b/src/syscall/linux-wire.h index aa952d58..8c767ebb 100644 --- a/src/syscall/linux-wire.h +++ b/src/syscall/linux-wire.h @@ -112,6 +112,8 @@ typedef struct { #define LINUX_EILSEQ 84 /* Illegal byte sequence */ #define LINUX_EHOSTDOWN 112 /* Host is down */ #define LINUX_ENODATA 61 /* No data available (xattr missing, stream) */ +#define LINUX_EPIPE 32 /* Broken pipe; also usbfs endpoint stall */ +#define LINUX_ETIME 62 /* Timer expired (USB device not responding) */ /* Linux FD flags. */ #define LINUX_FD_CLOEXEC 1 @@ -508,6 +510,7 @@ typedef struct { #define FD_FUSE_FILE 15 #define FD_FUSE_DIR 16 #define FD_URANDOM 17 +#define FD_USBDEV 18 #define FD_VIRTUAL_PATH_MAX 64 /* File sealing flags (F_SEAL_*) for memfd_create. Tracked per-FD. */ diff --git a/src/syscall/syscall.c b/src/syscall/syscall.c index e9d32dd3..12da9191 100644 --- a/src/syscall/syscall.c +++ b/src/syscall/syscall.c @@ -66,6 +66,7 @@ #include "syscall/proc-pidfd.h" #include "syscall/signal.h" #include "syscall/sys.h" +#include "syscall/usbdev.h" #include "syscall/sysvipc.h" #include "proved/timespec.h" @@ -115,6 +116,7 @@ void syscall_init(void) inotify_init(); netlink_init(); fuse_init(); + usbdev_init(); pidfd_init(); io_init(); fd_register_cleanup(FD_URANDOM, urandom_fd_cleanup); diff --git a/src/syscall/usbdev.c b/src/syscall/usbdev.c new file mode 100644 index 00000000..7529c9a0 --- /dev/null +++ b/src/syscall/usbdev.c @@ -0,0 +1,2232 @@ +/* + * usbdevfs (/dev/bus/usb/BBB/DDD) fd emulation over IOKit + * + * Copyright 2026 elfuse contributors + * SPDX-License-Identifier: Apache-2.0 + * + * Stage 2: a typed FD_USBDEV fd whose synchronous usbdevfs ioctls are mapped + * onto IOUSBDeviceInterface650 / IOUSBInterfaceInterface800 plugin calls + * (research doc D's op table). Semantics mirror drivers/usb/core/devio.c (doc A + * sections A1 and A2): + * + * - open of any access mode succeeds; read() serves the descriptors blob + * (byte-identical to the sysfs `descriptors` attribute) at a per-open + * file position; SEEK_END is -EINVAL (no_seek_end_llseek). + * - every ioctl requires a writable fd: O_RDONLY fd -> -EPERM + * (devio.c:2605-2606 FMODE_WRITE gate). + * - CLAIMINTERFACE returns -EBUSY when a macOS kernel driver is bound; the + * "kernel driver" test is an IORegistry child of the IOUSBHostInterface + * service in the service plane (libusb darwin_usb.c:2746-2770), and the + * claim itself is USBInterfaceOpen (kIOReturnExclusiveAccess -> -EBUSY). + * - CONTROL/BULK are Linux's sync paths (do_proc_control/do_proc_bulk): + * bounce buffers around DeviceRequestTO / Read|WritePipeTO, timeout in ms + * (0 = unlimited on both sides), -ETIMEDOUT from + * kIOUSBTransactionTimeout, stall -> -EPIPE, and Linux's implicit + * claim of the recipient interface (check_ctrlrecip/checkintf). + * + * Documented stage-2 deviations from Linux: + * - USBDEVFS_RESET does not re-enumerate: USBDeviceReEnumerate(0) would + * tear down every open plugin handle (doc D "reset" row), so RESET clears + * the stall state of all claimed pipes and returns 0. TODO(stage 3+): + * full re-enumeration with pending_device adoption. + * - Sync BULK on an interrupt endpoint is -EINVAL (Linux converts it to an + * interrupt URB; IOKit's ReadPipeTO/WritePipeTO reject interrupt pipes, + * IOUSBLib.h "BadArgument if TO on interrupt pipe"). TODO(later): route + * through the async path with a watchdog. + * - SUBMITURB/DISCARDURB/REAPURB* are -ENOTTY until stage 3. + * - dup()/fork() of an FD_USBDEV fd are refused (-EBADF): IOKit plugin + * handles are process-local and the side table is keyed by the guest fd. + * TODO(later): explicit dup alias (fuse_dup_fd pattern). + * - DISCONNECT/CONNECT/DISCONNECT_CLAIM cannot unbind Apple drivers without + * root or the com.apple.vm.device-access entitlement, so a bound kernel + * driver yields -EACCES (matching Linux's privileges-dropped answer). + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "core/guest.h" +#include "debug/log.h" +#include "runtime/usb-sysfs.h" +#include "syscall/internal.h" +#include "syscall/linux-wire.h" +#include "syscall/proc.h" +#include "syscall/usbdev.h" +#include "utils.h" + +/* usbdevfs wire ABI (LP64; x86_64 == aarch64) */ + +#define USBDEVFS_CONTROL 0xc0185500u +#define USBDEVFS_BULK 0xc0185502u +#define USBDEVFS_RESETEP 0x80045503u +#define USBDEVFS_SETINTERFACE 0x80085504u +#define USBDEVFS_SETCONFIGURATION 0x80045505u +#define USBDEVFS_GETDRIVER 0x41045508u +#define USBDEVFS_SUBMITURB 0x8038550au +#define USBDEVFS_DISCARDURB 0x0000550bu +#define USBDEVFS_REAPURB 0x4008550cu +#define USBDEVFS_REAPURBNDELAY 0x4008550du +#define USBDEVFS_DISCSIGNAL 0x8010550eu +#define USBDEVFS_CLAIMINTERFACE 0x8004550fu +#define USBDEVFS_RELEASEINTERFACE 0x80045510u +#define USBDEVFS_CONNECTINFO 0x40085511u +#define USBDEVFS_IOCTL 0xc0105512u +#define USBDEVFS_RESET 0x00005514u +#define USBDEVFS_CLEAR_HALT 0x80045515u +#define USBDEVFS_GET_CAPABILITIES 0x8004551au +#define USBDEVFS_DISCONNECT_CLAIM 0x8108551bu +#define USBDEVFS_GET_SPEED 0x0000551fu + +/* Sub-codes of USBDEVFS_IOCTL (_IO('U', 22) / _IO('U', 23)). */ +#define USBDEVFS_IOCTL_DISCONNECT 0x00005516 +#define USBDEVFS_IOCTL_CONNECT 0x00005517 + +/* Capability bits (uapi/linux/usbdevice_fs.h:152-161). Every one of them + * describes the SUBMITURB/REAPURB machinery: ZERO_PACKET and BULK_CONTINUATION + * are URB flags, NO_PACKET_SIZE_LIM and BULK_SCATTER_GATHER are properties of + * how a URB is split, REAP_AFTER_DISCONNECT is about reaping. Stage 2 answers + * -ENOTTY to all of those ioctls, so it advertises none of it and reports 0; + * the async stage raises the word as it lands each one. MMAP, DROP_PRIVILEGES, + * CONNINFO_EX and SUSPEND stay clear for the same reason (doc A section A7.7). + */ +#define USBDEVFS_CAP_ZERO_PACKET 0x01u +#define USBDEVFS_CAP_BULK_CONTINUATION 0x02u +#define USBDEVFS_CAP_NO_PACKET_SIZE_LIM 0x04u +#define USBDEVFS_CAP_BULK_SCATTER_GATHER 0x08u +#define USBDEVFS_CAP_REAP_AFTER_DISCONNECT 0x10u +#define USBDEV_CAPS 0u + +#define USBDEVFS_DISCONNECT_CLAIM_IF_DRIVER 0x01u +#define USBDEVFS_DISCONNECT_CLAIM_EXCEPT_DRIVER 0x02u + +typedef struct { + uint8_t bRequestType; + uint8_t bRequest; + uint16_t wValue; + uint16_t wIndex; + uint16_t wLength; + uint32_t timeout; /* ms; 0 = unlimited */ + uint32_t pad; /* natural LP64 hole before the pointer */ + uint64_t data; +} linux_usbdevfs_ctrltransfer_t; /* sizeof == 24, data at offset 16 */ + +typedef struct { + uint32_t ep; + uint32_t len; + uint32_t timeout; /* ms; 0 = unlimited */ + uint32_t pad; + uint64_t data; +} linux_usbdevfs_bulktransfer_t; /* sizeof == 24, data at offset 16 */ + +typedef struct { + uint32_t interface; + uint32_t altsetting; +} linux_usbdevfs_setinterface_t; + +typedef struct { + uint32_t interface; + char driver[256]; +} linux_usbdevfs_getdriver_t; /* sizeof == 260 */ + +typedef struct { + uint32_t devnum; + uint8_t slow; + uint8_t pad[3]; +} linux_usbdevfs_connectinfo_t; /* sizeof == 8 */ + +typedef struct { + int32_t ifno; + int32_t ioctl_code; + uint64_t data; +} linux_usbdevfs_ioctl_t; /* sizeof == 16 */ + +typedef struct { + uint32_t interface; + uint32_t flags; + char driver[256]; +} linux_usbdevfs_disconnect_claim_t; /* sizeof == 264 */ + +/* do_proc_control caps wLength at PAGE_SIZE (devio.c:1182-1183). */ +#define USBDEV_CTRL_MAX 4096 + +/* usbfs_memory_mb: 16 MB by default (devio.c:134), and not a per-call size cap. + * It is one module-global allowance for every usbfs transfer in flight, charged + * by usbfs_increase_memory_usage and given back by usbfs_decrease_memory_usage + * when the transfer settles (devio.c:145-178). Reading it as a per-call ceiling + * got both halves wrong: a single request of exactly the allowance was + * accepted, because only the buffer was measured and the URB was not, and + * nothing accumulated across calls or across fds, so a guest holding every + * side-table slot could keep USBDEV_MAX_FDS host buffers of that size alive at + * once. The per-request half is what the lane asserts: len == the allowance is + * ENOMEM, len just under it goes through twice, and it survives a faulting + * transfer, so both refunds land. The across-fd half is what sharing one + * counter adds, and it is not asserted anywhere -- the loopback model retires a + * transfer before a second can overlap it, so concurrent charges never meet + * there. Refusing 32 concurrent 16 MB requests does not show it either: the + * per-request boundary already refuses every one of them on its own. + */ +#define USBDEV_MEMORY_MAX (16ull * 1024 * 1024) + +/* Linux charges len + sizeof(struct urb), so a transfer of exactly the + * allowance never fits (devio.c:1308). sizeof(struct urb) is kernel-internal + * and config-dependent, and nothing on this side can observe it; what the model + * has to reproduce is that the per-transfer charge strictly exceeds the length, + * which is what decides the boundary case. The constant is named for that job + * rather than claimed to be the kernel's number. + */ +#define USBDEV_URB_OVERHEAD 192ull + +/* Bytes charged against USBDEV_MEMORY_MAX, summed across every fd. */ +static _Atomic uint64_t usbdev_memory_usage; + +/* usbfs_increase_memory_usage (devio.c:146-165): take the whole amount or none + * of it. The compare-exchange stands in for the kernel's spinlock, and the sum + * cannot overflow -- the ceiling bounds the accumulator and the caller has + * already refused a length at INT32_MAX. + */ +static bool usbdev_memory_charge(uint64_t amount) +{ + uint64_t cur = + atomic_load_explicit(&usbdev_memory_usage, memory_order_relaxed); + do { + if (cur + amount > USBDEV_MEMORY_MAX) + return false; + } while (!atomic_compare_exchange_weak_explicit( + &usbdev_memory_usage, &cur, cur + amount, memory_order_acq_rel, + memory_order_relaxed)); + return true; +} + +/* usbfs_decrease_memory_usage (devio.c:168-178). Every exit from a charged + * transfer owes this call, the error arms included: an allowance that is not + * given back is one the next transfer never sees again. + */ +static void usbdev_memory_refund(uint64_t amount) +{ + atomic_fetch_sub_explicit(&usbdev_memory_usage, amount, + memory_order_release); +} + +/* side table */ + +/* No usbfs limit corresponds to this: Linux allocates a usb_dev_state per open. + * The fixed table is a stage-2 simplification, so exhaustion is spelled -ENOMEM + * -- a kernel-side resource shortfall -- rather than -EMFILE, which would tell + * the guest its own descriptor limit is exhausted when it is not. + */ +#define USBDEV_MAX_FDS 32 + +/* claimintf refuses ifnum >= 8 * sizeof(ps->ifclaimed) and ifclaimed is an + * unsigned long (devio.c:75, :785), so the bound is 64 on every LP64 ABI elfuse + * emulates, not 32. Between the two an interface number is merely absent, which + * is -ENOENT from usb_ifnum_to_if, not -EINVAL. + */ +#define USBDEV_MAX_IFACES 64 +#define USBDEV_MAX_PIPES 30 + +typedef struct { + bool claimed; + IOUSBInterfaceInterface800 **intf; + int npipes; + uint8_t pipe_ep[USBDEV_MAX_PIPES]; /* pipeRef-1 -> bEndpointAddress */ + uint8_t pipe_type[USBDEV_MAX_PIPES]; /* kUSBControl..kUSBInterrupt */ +} usbdev_iface_t; + +typedef struct { + bool used; /* slot allocated (table lock) */ + bool dead; /* torn down, awaiting slot release (table lock) */ + int refs; /* live usbdev_acquire pins (table lock) */ + int guest_fd; + uint64_t generation; /* fd-table generation captured at open (ABA) */ + int busnum, devnum; + uint32_t location_id; + unsigned vid, pid; /* modeled identity, re-checked at every lookup */ + char serial[128]; + unsigned speed_code; /* raw registry 'Device Speed' */ + unsigned cfg_value; /* active bConfigurationValue */ + uint8_t *blob; /* usbfs descriptors blob (read() source) */ + size_t blob_len; + off_t pos; /* read()/lseek() file position */ + int pipe_wr; /* write end of the readiness pipe (stage-3 completions) */ + io_service_t service; /* retained IOUSBDevice service */ + IOUSBDeviceInterface650 **dev; /* lazily created device plugin */ + bool dev_open; /* USBDeviceOpen succeeded */ + bool dev_open_tried; + usbdev_iface_t ifaces[USBDEV_MAX_IFACES]; + + /* Lock-free mirrors for cross-fd reads (SETCONFIGURATION's device-wide + * claim check): claimed_mask mirrors ifaces[].claimed bit-per-interface, + * devkey names the bound device (nonzero while bound). A handler runs under + * its own entry lock, and no path takes a second entry lock while holding + * one -- an entry lock can be held across a whole transfer timeout, so + * waiting on a peer's would stall this fd for as long as that peer's + * transfer, and two fds doing it in opposite order would deadlock. Another + * slot's entry lock is therefore never taken here; the atomics are read + * instead. + */ + _Atomic uint64_t claimed_mask; + _Atomic uint64_t devkey; + + /* Guards every field above except used/dead/refs/guest_fd/generation, which + * the table lock guards, and the atomic mirrors. + * + * Held across the blocking IOKit transfer calls, so two ioctls on one fd + * serialize. Linux does NOT: do_proc_control and do_proc_bulk drop the + * device lock around usbfs_start_wait_urb and retake it after + * (devio.c:1219/1245 and :1337/1357), so a trivial ioctl on the same fd + * answers while a transfer is in flight. Measured here, a GET_SPEED issued + * during a 6 s BULK on the same fd waits 5970 ms. Dropping the lock needs + * the interface handle to be refcounted so a concurrent RELEASEINTERFACE + * cannot close it under the transfer, which is the async stage's machinery; + * until then this is a recorded deviation, not the kernel behavior it used + * to claim to be. What it is not any more is cross-fd: the lock is no + * longer taken underneath the table lock. + */ + pthread_mutex_t lock; +} usbdev_t; + +_Static_assert(USBDEV_MAX_IFACES <= 64, + "claimed_mask carries one bit per interface"); + +/* The one definition of how the lock-free cross-fd mirrors are reached. + * + * claimed_mask and devkey are read without the owning entry lock by + * usbdev_claimed_elsewhere, which walks the other slots on SETCONFIGURATION + * while holding its own entry lock, where no second entry lock may be taken. + * devkey is the publish flag: a binder resets claimed_mask to 0 and then stores + * the new key, and the reader tests devkey == key before it reads claimed_mask, + * so the key store releases and the key load acquires. Ordering the + * claimed_mask reset before the key release is what stops a slot recycled to + * the same bus/dev from exposing the new key beside a stale nonzero mask left + * by its previous life, which would be a spurious -EBUSY. The mask's own + * set/clear ride the same release so a cross-fd reader that already matched an + * unchanged key still sees a fresh claim; every mutator otherwise runs under + * the entry lock, which is the whole ordering requirement. + */ +static inline void claimed_mask_set(usbdev_t *u, unsigned bit) +{ + atomic_fetch_or_explicit(&u->claimed_mask, 1ull << bit, + memory_order_release); +} + +static inline void claimed_mask_clear(usbdev_t *u, unsigned bit) +{ + atomic_fetch_and_explicit(&u->claimed_mask, ~(1ull << bit), + memory_order_release); +} + +static inline void claimed_mask_reset(usbdev_t *u) +{ + atomic_store_explicit(&u->claimed_mask, 0, memory_order_release); +} + +static inline uint64_t claimed_mask_load(const usbdev_t *u) +{ + return atomic_load_explicit(&u->claimed_mask, memory_order_acquire); +} + +static inline void devkey_publish(usbdev_t *u, uint64_t key) +{ + atomic_store_explicit(&u->devkey, key, memory_order_release); +} + +static inline void devkey_retire(usbdev_t *u) +{ + atomic_store_explicit(&u->devkey, 0, memory_order_release); +} + +static inline uint64_t devkey_load(const usbdev_t *u) +{ + return atomic_load_explicit(&u->devkey, memory_order_acquire); +} + +/* usbdev_table_lock is never held together with a per-entry lock: a lookup + * finds its entry under the table lock, drops it, and only then takes the entry + * lock, because a sync transfer holds an entry lock for a whole timeout and no + * other fd may wait behind it. What pins the slot across the gap is the refs + * and dead pair the lookup sets under the table lock, not a nesting. The table + * lock is also a leaf with respect to fd_lock (never held while taking it, and + * vice versa). See internal.h's lock order block. + */ +static pthread_mutex_t usbdev_table_lock = PTHREAD_MUTEX_INITIALIZER; +static usbdev_t usbdev_fds[USBDEV_MAX_FDS]; +static bool usbdev_ready; + +/* IOReturn -> -LINUX_E* (doc D table (b)) */ + +#ifndef kUSBHostReturnPipeStalled +#define kUSBHostReturnPipeStalled 0xe0005000u +#endif + +static int64_t ioret_neg_errno(IOReturn r) +{ + switch ((uint32_t) r) { + case kIOReturnSuccess: + case kIOReturnUnderrun: /* short transfer == success for usbfs */ + return 0; + case kIOUSBPipeStalled: + case kUSBHostReturnPipeStalled: + return -LINUX_EPIPE; + case kIOUSBTransactionTimeout: + return -LINUX_ETIMEDOUT; + case kIOReturnNoDevice: + case kIOReturnNotOpen: + case kIOReturnNotAttached: + return -LINUX_ENODEV; + case kIOReturnOverrun: + return -LINUX_EOVERFLOW; + case kIOReturnAborted: + /* A sync transfer that comes back Aborted was already on the wire + * (another thread's teardown aborted the pipe), so the dispatcher must + * not re-execute the ioctl and send it again. + */ + syscall_restart_forbid(); + return -LINUX_EINTR; + case kIOReturnExclusiveAccess: + case kIOReturnBusy: + return -LINUX_EBUSY; + case kIOReturnNotPermitted: + case kIOReturnNotPrivileged: + return -LINUX_EACCES; + case kIOReturnBadArgument: + return -LINUX_EINVAL; + case kIOReturnNoMemory: + case kIOReturnNoResources: + case kIOReturnCannotWire: + return -LINUX_ENOMEM; + case kIOReturnUnsupported: + return -LINUX_ENOTTY; + case kIOUSBUnknownPipeErr: + case kIOUSBEndpointNotFound: + case kIOUSBInterfaceNotFound: + return -LINUX_ENOENT; + case kIOReturnNotResponding: + return -LINUX_ETIME; + default: + return -LINUX_EPROTO; + } +} + +/* IOKit helpers */ + +static long usbdev_ioreg_num(io_service_t s, const char *key) +{ + CFStringRef k = CFStringCreateWithCString(kCFAllocatorDefault, key, + kCFStringEncodingUTF8); + if (!k) + return -1; + CFTypeRef v = IORegistryEntryCreateCFProperty(s, k, kCFAllocatorDefault, 0); + CFRelease(k); + long n = -1; + if (v && CFGetTypeID(v) == CFNumberGetTypeID()) + CFNumberGetValue((CFNumberRef) v, kCFNumberLongType, &n); + if (v) + CFRelease(v); + return n; +} + +static bool usbdev_ioreg_str(io_service_t s, + const char *key, + char *out, + size_t n) +{ + CFStringRef k = CFStringCreateWithCString(kCFAllocatorDefault, key, + kCFStringEncodingUTF8); + if (!k) + return false; + CFTypeRef v = IORegistryEntryCreateCFProperty(s, k, kCFAllocatorDefault, 0); + CFRelease(k); + out[0] = '\0'; + bool ok = false; + if (v && CFGetTypeID(v) == CFStringGetTypeID()) + ok = CFStringGetCString((CFStringRef) v, out, (CFIndex) n, + kCFStringEncodingUTF8) && + out[0] != '\0'; + if (v) + CFRelease(v); + return ok; +} + +/* The modeled bus/dev numbers name a device observed at model-build time; the + * locationID they map back to names a PORT. If the device was swapped since + * (unplug + different device into the same port, model not rebuilt), the + * location lookup happily returns the newcomer. Compare the live registry + * identity against the modeled one so an open cannot hand the guest a device + * other than the one its descriptors blob describes. + */ +static bool usbdev_identity_matches(io_service_t svc, + unsigned want_vid, + unsigned want_pid, + const char *want_serial) +{ + long vid = usbdev_ioreg_num(svc, "idVendor"); + long pid = usbdev_ioreg_num(svc, "idProduct"); + if (vid != (long) want_vid || pid != (long) want_pid) + return false; + + /* "USB Serial Number" is the property name. kUSBSerialNumberString is the + * SDK macro that spells it (USBSpec.h), and passing the macro's own name as + * the key made the first lookup unmatchable, so only the fallback ever did + * anything. One lookup, one spelling. + */ + char serial[128] = ""; + (void) usbdev_ioreg_str(svc, "USB Serial Number", serial, sizeof(serial)); + return strcmp(serial, want_serial) == 0; +} + +/* Retained IOUSBDevice service whose locationID matches, or IO_OBJECT_NULL. */ +static io_service_t usbdev_service_for_location(uint32_t location_id) +{ + CFMutableDictionaryRef match = IOServiceMatching("IOUSBDevice"); + if (!match) + return IO_OBJECT_NULL; + io_iterator_t it = IO_OBJECT_NULL; + if (IOServiceGetMatchingServices(kIOMainPortDefault, match, &it) != + kIOReturnSuccess) + return IO_OBJECT_NULL; + io_service_t found = IO_OBJECT_NULL; + io_service_t svc; + while ((svc = IOIteratorNext(it))) { + if (found == IO_OBJECT_NULL && + usbdev_ioreg_num(svc, "locationID") == (long) location_id) { + found = svc; /* keep the iterator's reference */ + continue; + } + IOObjectRelease(svc); + } + IOObjectRelease(it); + return found; +} + +/* Resolve u->service on first use, or 0 when this port carries no device with + * the modeled identity. + * + * The lookup is deferred rather than done in the constructor because open(2) + * must not disagree with the names beside it: stat, access and an O_PATH open + * of the same node are all answered from the model, so requiring live hardware + * at open time made a plain O_RDONLY the one entry point that reported ENODEV + * for a node the other four described. It also took the ELFUSE_USB_FIXTURE + * model, whose devices have no IOKit service at all, out of reach of every + * assertion about this fd. Deferring moves the missing-device answer onto the + * operations that actually need the wire, where -ENODEV is what Linux reports + * for a device that is gone. + */ +static int64_t usbdev_ensure_service(usbdev_t *u) +{ + if (u->service != IO_OBJECT_NULL) + return 0; + io_service_t svc = usbdev_service_for_location(u->location_id); + if (svc == IO_OBJECT_NULL) + return -LINUX_ENODEV; + if (!usbdev_identity_matches(svc, u->vid, u->pid, u->serial)) { + /* The port holds some device, but not the modeled one. */ + IOObjectRelease(svc); + return -LINUX_ENODEV; + } + u->service = svc; + return 0; +} + +/* Create u->dev on first use. GetConfigurationDescriptorPtr-class calls and + * CreateInterfaceIterator need only the plugin, not USBDeviceOpen. + */ +static int64_t usbdev_ensure_dev_plugin(usbdev_t *u) +{ + if (u->dev) + return 0; + int64_t srv = usbdev_ensure_service(u); + if (srv < 0) + return srv; + IOCFPlugInInterface **plug = NULL; + SInt32 score = 0; + IOReturn r = IOCreatePlugInInterfaceForService( + u->service, kIOUSBDeviceUserClientTypeID, kIOCFPlugInInterfaceID, &plug, + &score); + if (r != kIOReturnSuccess || !plug) { + log_warn("usbdev: device plugin for %d-%d failed 0x%x", u->busnum, + u->devnum, r); + return r == kIOReturnSuccess ? -LINUX_ENOMEM : ioret_neg_errno(r); + } + IOUSBDeviceInterface650 **dev = NULL; + HRESULT hr = (*plug)->QueryInterface( + plug, CFUUIDGetUUIDBytes(kIOUSBDeviceInterfaceID650), (LPVOID *) &dev); + (*plug)->Release(plug); + if (hr != S_OK || !dev) + return -LINUX_ENOMEM; + u->dev = dev; + return 0; +} + +/* Lazy exclusive device open; kIOReturnExclusiveAccess is tolerated the way + * libusb tolerates it (device stays usable for ep0 requests and interface + * claims; only SetConfiguration demands a real open). + */ +static void usbdev_lazy_device_open(usbdev_t *u) +{ + if (u->dev_open || u->dev_open_tried || !u->dev) + return; + u->dev_open_tried = true; + IOReturn r = (*u->dev)->USBDeviceOpen(u->dev); + if (r == kIOReturnSuccess) + u->dev_open = true; + else + log_debug("usbdev: USBDeviceOpen %d-%d -> 0x%x (tolerated)", u->busnum, + u->devnum, r); +} + +/* Retained IOUSBHostInterface service for bInterfaceNumber ifnum in the active + * configuration, or IO_OBJECT_NULL. Uses CreateInterfaceIterator so "exists" + * means exactly what claimintf's usb_ifnum_to_if means. + */ +static io_service_t usbdev_iface_service(usbdev_t *u, unsigned ifnum) +{ + if (usbdev_ensure_dev_plugin(u) < 0) + return IO_OBJECT_NULL; + IOUSBFindInterfaceRequest fr = { + .bInterfaceClass = kIOUSBFindInterfaceDontCare, + .bInterfaceSubClass = kIOUSBFindInterfaceDontCare, + .bInterfaceProtocol = kIOUSBFindInterfaceDontCare, + .bAlternateSetting = kIOUSBFindInterfaceDontCare, + }; + io_iterator_t it = IO_OBJECT_NULL; + if ((*u->dev)->CreateInterfaceIterator(u->dev, &fr, &it) != + kIOReturnSuccess) + return IO_OBJECT_NULL; + io_service_t found = IO_OBJECT_NULL; + io_service_t svc; + while ((svc = IOIteratorNext(it))) { + if (found == IO_OBJECT_NULL && + usbdev_ioreg_num(svc, "bInterfaceNumber") == (long) ifnum) { + found = svc; + continue; + } + IOObjectRelease(svc); + } + IOObjectRelease(it); + return found; +} + +/* "Kernel driver bound" == the interface service has a driver child in the + * service plane (libusb darwin_usb.c:2746-2770). Fills name (class name, + * truncated) when one exists. + * + * A user client is not a driver. IOKit publishes an + * AppleUSBHostInterfaceUserClient child for every USBInterfaceOpen, this + * layer's own included, so taking the first child of any class reported a peer + * usbfs consumer -- another elfuse process, or another fd of this one -- as a + * bound host driver, and that answer drove GETDRIVER, DISCONNECT_CLAIM and + * SETCONFIGURATION. Walk the children and answer for the first one that is not + * a user client. + */ +static bool usbdev_iface_driver(io_service_t ifs, char *name, size_t n) +{ + io_iterator_t it = IO_OBJECT_NULL; + if (IORegistryEntryGetChildIterator(ifs, kIOServicePlane, &it) != + kIOReturnSuccess || + it == IO_OBJECT_NULL) + return false; + bool bound = false; + io_registry_entry_t child; + while ((child = IOIteratorNext(it))) { + if (!bound && !IOObjectConformsTo(child, "IOUserClient")) { + io_name_t cls; + if (IOObjectGetClass(child, cls) == kIOReturnSuccess) + str_copy_trunc(name, cls, n); + else + str_copy_trunc(name, "unknown", n); + bound = true; + } + IOObjectRelease(child); + } + IOObjectRelease(it); + return bound; +} + +/* Returns 0, or the negative Linux errno IOKit's answer maps to: a device + * pulled out mid-claim answers kIOReturnNoDevice, which is the -ENODEV Linux + * reports for it, and flattening every failure here into -EIO renamed that as + * an I/O error. -EIO stays only for a code the map has no entry for. + */ +static int64_t usbdev_build_pipe_map(usbdev_iface_t *fi) +{ + /* Clear the whole map, not just the count: a GetPipeProperties failure at + * one pipeRef of a SETINTERFACE rebuild used to leave the previous + * altsetting's address at that index while npipes still covered it, so a + * later lookup could match a stale address and return a pipeRef that now + * means a different endpoint. + */ + fi->npipes = 0; + memset(fi->pipe_ep, 0, sizeof(fi->pipe_ep)); + memset(fi->pipe_type, 0, sizeof(fi->pipe_type)); + UInt8 ne = 0; + IOReturn r = (*fi->intf)->GetNumEndpoints(fi->intf, &ne); + if (r != kIOReturnSuccess) { + int64_t err = ioret_neg_errno(r); + return err < 0 ? err : -LINUX_EIO; + } + if (ne > USBDEV_MAX_PIPES) + ne = USBDEV_MAX_PIPES; + for (UInt8 p = 1; p <= ne; p++) { + UInt8 dir = 0, num = 0, type = 0, interval = 0; + UInt16 mps = 0; + if ((*fi->intf)->GetPipeProperties(fi->intf, p, &dir, &num, &type, &mps, + &interval) != kIOReturnSuccess) + continue; + fi->pipe_ep[p - 1] = (uint8_t) (num | (dir == kUSBIn ? 0x80 : 0)); + fi->pipe_type[p - 1] = type; + fi->npipes = p; + } + return 0; +} + +/* claim / release (entry lock held) */ + +static int64_t usbdev_claim_locked(usbdev_t *u, unsigned ifnum) +{ + if (ifnum >= USBDEV_MAX_IFACES) + return -LINUX_EINVAL; /* claimintf devio.c:786 */ + usbdev_iface_t *fi = &u->ifaces[ifnum]; + if (fi->claimed) + return 0; /* already ours */ + + /* Ahead of the interface lookup so a device that is not there answers + * -ENODEV rather than "no such interface". + */ + int64_t drc = usbdev_ensure_dev_plugin(u); + if (drc < 0) + return drc; + + io_service_t ifs = usbdev_iface_service(u, ifnum); + if (ifs == IO_OBJECT_NULL) + return -LINUX_ENOENT; + + /* Linux: a bound kernel driver makes CLAIMINTERFACE -EBUSY + * (usb_driver_claim_interface, driver.c:558). macOS arbitration is + * per-IOKit-object and dynamic rather than per-device and static, so the + * bound driver is not the question: USBInterfaceOpen answers + * kIOReturnExclusiveAccess exactly while that driver holds that interface, + * and succeeds while it is idle. Attempt the open and map what IOKit + * answers. Pre-refusing on the mere presence of a driver child refused work + * macOS grants -- the ESP32-S3's CDC data interface opens whenever nothing + * holds /dev/cu.usbmodem1101 -- and it did so from a registry snapshot that + * no live host state corresponds to. + */ + IOCFPlugInInterface **plug = NULL; + SInt32 score = 0; + IOReturn r = IOCreatePlugInInterfaceForService( + ifs, kIOUSBInterfaceUserClientTypeID, kIOCFPlugInInterfaceID, &plug, + &score); + IOObjectRelease(ifs); + if (r != kIOReturnSuccess || !plug) + return r == kIOReturnSuccess ? -LINUX_ENOMEM : ioret_neg_errno(r); + IOUSBInterfaceInterface800 **intf = NULL; + HRESULT hr = (*plug)->QueryInterface( + plug, CFUUIDGetUUIDBytes(kIOUSBInterfaceInterfaceID800), + (LPVOID *) &intf); + (*plug)->Release(plug); + if (hr != S_OK || !intf) + return -LINUX_ENOMEM; + + r = (*intf)->USBInterfaceOpen(intf); + if (r != kIOReturnSuccess) { + (*intf)->Release(intf); + int64_t e = ioret_neg_errno(r); + return e == 0 ? -LINUX_EBUSY : e; + } + fi->intf = intf; + int64_t maprc = usbdev_build_pipe_map(fi); + if (maprc < 0) { + (*intf)->USBInterfaceClose(intf); + (*intf)->Release(intf); + fi->intf = NULL; + return maprc; + } + fi->claimed = true; + claimed_mask_set(u, ifnum); + return 0; +} + +static int64_t usbdev_release_locked(usbdev_t *u, unsigned ifnum) +{ + if (ifnum >= USBDEV_MAX_IFACES) + return -LINUX_EINVAL; + usbdev_iface_t *fi = &u->ifaces[ifnum]; + if (!fi->claimed) { + /* releaseintf checks usb_ifnum_to_if first: a nonexistent interface is + * -ENOENT, an existing unclaimed one -EINVAL (devio.c:815-833). + */ + int64_t drc = usbdev_ensure_dev_plugin(u); + if (drc < 0) + return drc; + io_service_t ifs = usbdev_iface_service(u, ifnum); + if (ifs == IO_OBJECT_NULL) + return -LINUX_ENOENT; + IOObjectRelease(ifs); + return -LINUX_EINVAL; + } + (*fi->intf)->USBInterfaceClose(fi->intf); + (*fi->intf)->Release(fi->intf); + fi->intf = NULL; + fi->claimed = false; + claimed_mask_clear(u, ifnum); + fi->npipes = 0; + return 0; +} + +/* usbdev_ep_owner_iface's two failures, kept apart because Linux answers them + * differently: an endpoint no altsetting carries is -ENOENT (findintfep's own + * return), while one whose owning interface number is past the claim bitmap is + * -EINVAL (checkintf, devio.c:842). + */ +#define USBDEV_EP_OWNER_NONE (-1) +#define USBDEV_EP_OWNER_OUT_OF_RANGE (-2) + +/* findintfep (devio.c:853-876): which interface of the active config carries + * bEndpointAddress ep, searching every altsetting. Parsed from the descriptors + * blob. + * + * Returns USBDEV_EP_OWNER_NONE when not found, or USBDEV_EP_OWNER_OUT_OF_RANGE + * when the endpoint is carried by an interface number this layer cannot + * represent. Never returns a number ifaces[] does not hold. + */ +static int usbdev_ep_owner_iface(const usbdev_t *u, uint8_t ep) +{ + const uint8_t *b = u->blob; + size_t len = u->blob_len; + size_t off = 18; + while (off + 9 <= len && b[off + 1] == 0x02 /* CONFIG */) { + size_t total = (size_t) b[off + 2] | ((size_t) b[off + 3] << 8); + if (total < 9 || off + total > len) + break; + bool active = b[off + 5] == (uint8_t) u->cfg_value; + if (active) { + size_t p = off + 9; + int cur_if = USBDEV_EP_OWNER_NONE; + while (p + 2 <= off + total && b[p] >= 2) { + uint8_t dlen = b[p], dtype = b[p + 1]; + if (p + dlen > off + total) + break; + if (dtype == 0x04 && dlen >= 9) { /* INTERFACE */ + /* bInterfaceNumber is a device-supplied byte with the whole + * 0..255 range behind it, and ifaces[] is 64 entries, the + * width of the unsigned long checkintf bounds against. The + * range test belongs here, where the number enters from the + * descriptor, not at the array index below it: a device + * declaring bInterfaceNumber 200 with a matching endpoint + * read up to 191 entries past ifaces[] before the bound + * inside usbdev_claim_locked ever ran, and nothing attached + * to a developer's machine declares such a number, so no + * fixture and no sanitizer run on real hardware could reach + * it. Out of range is carried rather than dropped so the + * lookup still answers what checkintf answers. + */ + cur_if = b[p + 2] < USBDEV_MAX_IFACES + ? (int) b[p + 2] + : USBDEV_EP_OWNER_OUT_OF_RANGE; + } else if (dtype == 0x05 && dlen >= 7 && /* ENDPOINT */ + b[p + 2] == ep && cur_if != USBDEV_EP_OWNER_NONE) { + return cur_if; + } + p += dlen; + } + } + off += total; + } + return USBDEV_EP_OWNER_NONE; +} + +/* Resolve ep -> (claimed iface, pipeRef), implicitly claiming the owner + * interface the way checkintf does for the sync paths. -ENOENT when no + * altsetting of the active config carries the endpoint. + */ +static int64_t usbdev_pipe_for_ep(usbdev_t *u, + unsigned int ep, + usbdev_iface_t **fi_out, + uint8_t *pipe_out) +{ + /* findintfep rejects everything outside USB_DIR_IN|0xf before it looks at + * anything (devio.c:860). The check belongs here rather than in each + * caller: BULK and the control endpoint recipient had it and CLEAR_HALT and + * RESETEP did not, so a malformed address fell out of the lookup as -ENOENT + * on two of the four entry points onto the same question. + * + * The argument is the caller's whole unsigned int, as Linux tests it. The + * ioctls carry a 32-bit endpoint word and narrowing it to a byte first + * meant the test only ever saw eight bits, so ep 0x183 and ep 0x01000083 + * both passed as 0x83 and the transfer went to an endpoint the caller did + * not name. + */ + if (ep & ~0x8fu) + return -LINUX_EINVAL; + uint8_t ep8 = (uint8_t) ep; + for (int pass = 0; pass < 2; pass++) { + for (int i = 0; i < USBDEV_MAX_IFACES; i++) { + usbdev_iface_t *fi = &u->ifaces[i]; + if (!fi->claimed) + continue; + for (int p = 0; p < fi->npipes; p++) { + if (fi->pipe_ep[p] == ep8) { + *fi_out = fi; + *pipe_out = (uint8_t) (p + 1); + return 0; + } + } + } + if (pass == 1) + break; + int owner = usbdev_ep_owner_iface(u, ep8); + if (owner == USBDEV_EP_OWNER_OUT_OF_RANGE) + return -LINUX_EINVAL; /* checkintf devio.c:842 */ + if (owner == USBDEV_EP_OWNER_NONE) + return -LINUX_ENOENT; + if (u->ifaces[owner].claimed) + return -LINUX_ENOENT; /* claimed but ep not in current alt */ + int64_t rc = usbdev_claim_locked(u, (unsigned) owner); + if (rc < 0) + return rc; + } + return -LINUX_ENOENT; +} + +/* check_ctrlrecip's endpoint-recipient arm (devio.c:905-933) for the control + * paths: wIndex is masked to its low byte first, exactly like Linux. + */ +static int64_t usbdev_check_ep_recip(usbdev_t *u, uint16_t wIndex) +{ + /* The mask stays: check_ctrlrecip narrows wIndex to its low byte itself + * (devio.c:904) before calling findintfep, so the reserved-bit test the + * lookup runs on its whole argument is meant to see eight bits here and + * thirty-two on the ioctls that carry an endpoint word. + */ + uint8_t index = (uint8_t) (wIndex & 0xff); + + /* The default control endpoint belongs to no interface: allowed with no + * claim and no lookup (devio.c:909-911) -- lsusb -v sends GET_STATUS to + * endpoint 0 this way. + */ + if ((index & 0x7f) == 0) + return 0; + + usbdev_iface_t *fi = NULL; + uint8_t pipe = 0; + int64_t rc = usbdev_pipe_for_ep(u, index, &fi, &pipe); + if (rc == -LINUX_ENOENT) { + /* Some Win apps pass the endpoint number where the address (with its + * direction bit) belongs; Linux flips the direction, warns, and lets + * the request through (devio.c:913-928). + */ + rc = usbdev_pipe_for_ep(u, index ^ 0x80, &fi, &pipe); + if (rc == 0) + log_warn( + "usbdev: control recipient requests ep %02x but needs " + "%02x", + index, index ^ 0x80); + } + return rc; +} + +/* side-table lookup */ + +/* Drop one pin, releasing the slot when the last pin leaves a dead entry. */ +static void usbdev_unref(usbdev_t *u) +{ + pthread_mutex_lock(&usbdev_table_lock); + if (--u->refs == 0 && u->dead) { + u->used = false; + u->dead = false; + u->guest_fd = -1; + u->generation = 0; + } + pthread_mutex_unlock(&usbdev_table_lock); +} + +/* Find the entry for guest fd and return it with its lock held and one pin + * taken; NULL when the fd is not a live FD_USBDEV fd (or was closed+reused: + * generation mismatch). + * + * The pin, rather than taking the entry lock under the table lock, is what + * keeps the slot from being reallocated between the two. Nesting them meant a + * thread whose sync transfer held the entry lock also held the table lock on + * every other thread's behalf, so one BULK with the "unlimited" timeout=0 that + * usbdevfs documents wedged every usbdevfs fd in the process -- lookups on + * unrelated fds, opens of other devices, and close(), which needs the same + * table lock. Measured: an 18.6 s close() of an unrelated fd behind one 20 s + * transfer. The in-code claim that this "briefly" stalled other fds' lookups + * was neither brief nor bounded. + */ +static usbdev_t *usbdev_acquire(int fd) +{ + fd_entry_t snap; + if (!fd_snapshot(fd, &snap) || snap.type != FD_USBDEV) + return NULL; + usbdev_t *u = NULL; + pthread_mutex_lock(&usbdev_table_lock); + for (int i = 0; i < USBDEV_MAX_FDS; i++) { + if (usbdev_fds[i].used && !usbdev_fds[i].dead && + usbdev_fds[i].guest_fd == fd && + usbdev_fds[i].generation == snap.generation) { + u = &usbdev_fds[i]; + u->refs++; + break; + } + } + pthread_mutex_unlock(&usbdev_table_lock); + if (!u) + return NULL; + pthread_mutex_lock(&u->lock); + if (u->dead) { + pthread_mutex_unlock(&u->lock); + usbdev_unref(u); + return NULL; + } + return u; +} + +/* Unlock and unpin an entry usbdev_acquire returned. */ +static void usbdev_release(usbdev_t *u) +{ + pthread_mutex_unlock(&u->lock); + usbdev_unref(u); +} + +static void usbdev_teardown_locked(usbdev_t *u) +{ + for (int i = 0; i < USBDEV_MAX_IFACES; i++) + if (u->ifaces[i].claimed) + (void) usbdev_release_locked(u, (unsigned) i); + if (u->dev) { + if (u->dev_open) + (*u->dev)->USBDeviceClose(u->dev); + (*u->dev)->Release(u->dev); + u->dev = NULL; + } + u->dev_open = false; + u->dev_open_tried = false; + if (u->service != IO_OBJECT_NULL) { + IOObjectRelease(u->service); + u->service = IO_OBJECT_NULL; + } + if (u->pipe_wr >= 0) { + close(u->pipe_wr); + u->pipe_wr = -1; + } + free(u->blob); + u->blob = NULL; + + /* Retire the lock-free mirrors last so no cross-fd reader can match a slot + * that is mid-teardown. + */ + claimed_mask_reset(u); + devkey_retire(u); +} + +static void usbdev_fd_cleanup(int guest_fd) +{ + /* The fd-table slot is already closed and free when this runs + * (fd_cleanup_entry is called outside fd_lock), so a sibling thread's + * open() can have won the same fd number and bound a second entry here + * before this call arrives, and both entries then answer to it. Matching on + * the number alone tore down whichever sat at the lower index -- about half + * the time the NEW one, whose guest fd was still open and which then + * reported EBADF on every use. The lane below drives 8000 open/read/close + * rounds across four threads: with the tiebreak removed it loses fds on + * every run (281, 313 and 371 over three), and none with it. The count is a + * race and varies; that it is never zero without the tiebreak is the point. + * + * fd_alloc stamps a globally monotonic generation, so among entries that + * answer to one fd number the closing one is always the one with the + * smaller generation. The cleanup vtable is void(*)(int) and hands over no + * snapshot, so that ordering is what identifies the entry. + */ + pthread_mutex_lock(&usbdev_table_lock); + usbdev_t *u = NULL; + for (int i = 0; i < USBDEV_MAX_FDS; i++) { + usbdev_t *o = &usbdev_fds[i]; + if (!o->used || o->dead || o->guest_fd != guest_fd) + continue; + if (!u || o->generation < u->generation) + u = o; + } + if (!u) { + pthread_mutex_unlock(&usbdev_table_lock); + return; + } + u->dead = true; + u->refs++; + pthread_mutex_unlock(&usbdev_table_lock); + + /* Outside the table lock: a sync transfer in flight on this fd holds the + * entry lock, and waiting for it here must not block every other fd. + */ + pthread_mutex_lock(&u->lock); + usbdev_teardown_locked(u); + pthread_mutex_unlock(&u->lock); + usbdev_unref(u); +} + +void usbdev_init(void) +{ + pthread_mutex_lock(&usbdev_table_lock); + if (!usbdev_ready) { + for (int i = 0; i < USBDEV_MAX_FDS; i++) { + memset(&usbdev_fds[i], 0, sizeof(usbdev_fds[i])); + usbdev_fds[i].guest_fd = -1; + usbdev_fds[i].pipe_wr = -1; + pthread_mutex_init(&usbdev_fds[i].lock, NULL); + } + usbdev_ready = true; + } + pthread_mutex_unlock(&usbdev_table_lock); + fd_register_cleanup(FD_USBDEV, usbdev_fd_cleanup); +} + +/* constructor */ + +/* Test seam: ELFUSE_USBDEV_OPEN_FAULT names one step of the open path and makes + * it fail the way the host can, because none of the three failures the + * constructor has to tell apart can be provoked from a guest. + * + * info the model lookup fails with ENOMEM rather than ENODEV + * blob the descriptor copy fails with ENOMEM + * pipe the readiness pipe cannot be created (ENFILE) + * + * Resolved once per process into an enum and cached, the shape + * fd_identity_window_delay uses in syscall/fs-stat.c, so an open on the + * failure-free path costs one relaxed load rather than a walk of the + * environment for every stage it passes. tests/test-usbdev-ioctl.c drives all + * three. + */ +typedef enum { + USBDEV_FAULT_UNREAD = -1, + USBDEV_FAULT_NONE = 0, + USBDEV_FAULT_INFO, + USBDEV_FAULT_BLOB, + USBDEV_FAULT_PIPE, +} usbdev_open_fault_t; + +static usbdev_open_fault_t usbdev_open_fault(void) +{ + static _Atomic int cached = USBDEV_FAULT_UNREAD; + int v = atomic_load_explicit(&cached, memory_order_relaxed); + if (v == USBDEV_FAULT_UNREAD) { + const char *env = getenv("ELFUSE_USBDEV_OPEN_FAULT"); + v = USBDEV_FAULT_NONE; + if (env && !strcmp(env, "info")) + v = USBDEV_FAULT_INFO; + else if (env && !strcmp(env, "blob")) + v = USBDEV_FAULT_BLOB; + else if (env && !strcmp(env, "pipe")) + v = USBDEV_FAULT_PIPE; + atomic_store_explicit(&cached, v, memory_order_relaxed); + } + return (usbdev_open_fault_t) v; +} + +/* Widen one of the two windows usbdev_open_path leaves around the publish, off + * unless the named variable holds a positive microsecond count. Same shape and + * the same reasoning as fd_identity_window_delay in syscall/fs-stat.c: both + * windows are real and a few instructions wide, and what the entry has to + * survive is a guest that closes -- or closes and reopens -- a fd number it + * predicted inside one of them. tests/test-usbdev-ioctl.c drives both. + */ +static void usbdev_window_delay(const char *name, _Atomic long *cached) +{ + long v = atomic_load_explicit(cached, memory_order_relaxed); + if (v < 0) { /* -1 = unread */ + const char *env = getenv(name); + long long n = env ? strtoll(env, NULL, 10) : 0; + v = (n > 0 && n < 1000000) ? (long) n : 0; + atomic_store_explicit(cached, v, memory_order_relaxed); + } + if (v > 0) + usleep((useconds_t) v); +} + +/* Between fd_alloc handing back the number and the side table binding it: the + * entry is not yet findable by fd number, so a close lands on nothing. + */ +static void usbdev_publish_window_delay(void) +{ + static _Atomic long cached = -1; + usbdev_window_delay("ELFUSE_USBDEV_PUBLISH_DELAY_US", &cached); +} + +/* Between the bind and the recheck that follows it: the entry is findable and + * therefore also freeable, so the close can reap it and a sibling open can take + * the slot back before the recheck runs. + */ +static void usbdev_retire_window_delay(void) +{ + static _Atomic long cached = -1; + usbdev_window_delay("ELFUSE_USBDEV_RETIRE_DELAY_US", &cached); +} + +/* Retire an entry whose guest fd was closed before the entry could be found by + * fd number. Claims it the way usbdev_fd_cleanup does -- dead under the table + * lock, one reference held across the teardown -- so a cleanup arriving from + * the other side can only claim it once. + * + * u is a raw pointer into static slot storage, and by the time this runs the + * allocation it named can already be gone: the close reaps the entry, + * usbdev_unref frees the slot, and a sibling usbdev_open_path binds its own + * live fd there. Reading the slot is therefore always defined but never proof + * of identity, so the claim is the whole tuple the caller allocated rather than + * "not dead": used and alive, this fd number, this generation. Testing !dead + * alone marked the sibling's entry dead, freed its blob and closed its pipe, + * and the sibling's still-open fd answered EBADF on every read and ioctl -- the + * failure usbdev_fd_cleanup's generation tiebreak exists to avoid, reintroduced + * on the other side of the same window. + */ +static void usbdev_retire_unpublished(usbdev_t *u, int guest_fd, uint64_t gen) +{ + pthread_mutex_lock(&usbdev_table_lock); + bool mine = + u->used && !u->dead && u->guest_fd == guest_fd && u->generation == gen; + if (mine) { + u->dead = true; + u->refs++; + } + pthread_mutex_unlock(&usbdev_table_lock); + if (!mine) + return; + pthread_mutex_lock(&u->lock); + usbdev_teardown_locked(u); + pthread_mutex_unlock(&u->lock); + usbdev_unref(u); +} + +/* The rule every failure in usbdev_open_path answers to: carry the errno the + * step that failed set, and translate exactly one of them. ENODEV is the model + * saying nothing answers to this address, which is the ENOENT open(2) owes for + * a name with nothing behind it. Every other errno means something else -- + * ENOMEM from an allocation, whatever mkdtemp or mkdir reported while the + * scratch tree was being built -- and inventing ENOENT for those told the guest + * a node was missing whenever the host was merely out of memory. Anything added + * here later carries its errno the same way. + */ +static int64_t usbdev_open_errno(void) +{ + return errno == ENODEV ? -LINUX_ENOENT : linux_errno(); +} + +static bool usbdev_parse_node(const char *path, int *bus, int *dev) +{ + unsigned b, d; + char tail; + if (sscanf(path, "/dev/bus/usb/%3u/%3u%c", &b, &d, &tail) != 2) + return false; + + /* Reject non-canonical spellings the tree never lists ("/dev/bus/usb/1/1" + * still parses above; the scratch tree only carries %03d names, and the + * stage-1 stat intercept agrees, so keep both views consistent). + */ + char canon[64]; + snprintf(canon, sizeof(canon), "/dev/bus/usb/%03u/%03u", b, d); + if (strcmp(canon, path) != 0) + return false; + *bus = (int) b; + *dev = (int) d; + return true; +} + +int64_t usbdev_open_path(const char *path, int linux_flags) +{ + int bus, dev; + if (!usbdev_parse_node(path, &bus, &dev)) + return INT64_MIN; + + /* O_PATH fds carry no I/O capability; the stage-1 placeholder (blob fd + * typed FD_PATH, stat-stamped) serves them without burning a slot here. + */ + if (linux_flags & LINUX_O_PATH) + return INT64_MIN; + + /* Existence is decided before the flags are. A name that spells a node but + * addresses no device is ENOENT whatever the caller asked for -- the kernel + * refuses O_DIRECTORY only once the lookup has produced an inode -- and + * answering ENOTDIR from the spelling alone let a sysroot file planted at + * /dev/bus/usb//001 turn every O_DIRECTORY open into the host's + * answer for it, where open(2) without the flag reported ENOENT. + */ + usb_sysfs_devinfo_t info; + bool have_info; + if (usbdev_open_fault() == USBDEV_FAULT_INFO) { + errno = ENOMEM; + have_info = false; + } else { + have_info = usb_sysfs_device_info(bus, dev, &info) == 0; + } + if (!have_info) + return usbdev_open_errno(); + if (linux_flags & LINUX_O_DIRECTORY) + return -LINUX_ENOTDIR; + size_t blob_len = 0; + uint8_t *blob; + if (usbdev_open_fault() == USBDEV_FAULT_BLOB) { + errno = ENOMEM; + blob = NULL; + } else { + blob = usb_sysfs_descriptors_dup(bus, dev, &blob_len); + } + if (!blob) + return usbdev_open_errno(); + + /* The IOKit service is resolved on first use, not here: see + * usbdev_ensure_service for why open(2) must not be the one entry point + * onto this node that demands live hardware. + */ + + usbdev_t *u = NULL; + pthread_mutex_lock(&usbdev_table_lock); + for (int i = 0; i < USBDEV_MAX_FDS; i++) { + if (!usbdev_fds[i].used) { + u = &usbdev_fds[i]; + u->used = true; + u->dead = false; + u->refs = 0; + u->guest_fd = -1; + u->generation = 0; + break; + } + } + pthread_mutex_unlock(&usbdev_table_lock); + if (!u) { + free(blob); + return -LINUX_ENOMEM; + } + + /* Stays {-1, -1} when pipe() itself fails: the error arm below must not + * close two indeterminate descriptors (netlink_socket's split). + */ + int pipefd[2] = {-1, -1}; + bool pipe_ok; + if (usbdev_open_fault() == USBDEV_FAULT_PIPE) { + errno = ENFILE; + pipe_ok = false; + } else { + pipe_ok = pipe(pipefd) == 0 && fd_set_nonblock(pipefd[0]) == 0 && + fd_set_nonblock(pipefd[1]) == 0; + } + if (!pipe_ok) { + /* Read the errno before the unwind: close() and pthread_mutex_lock() + * may both set it. Reporting EMFILE unconditionally named the guest's + * own descriptor limit for a failure that is the host's -- ENFILE, or + * whatever fcntl reported -- and the guest cannot act on a limit it has + * not reached. + */ + int64_t err = linux_errno(); + if (pipefd[0] >= 0) { + close(pipefd[0]); + close(pipefd[1]); + } + pthread_mutex_lock(&usbdev_table_lock); + u->used = false; + pthread_mutex_unlock(&usbdev_table_lock); + free(blob); + return err; + } + fcntl(pipefd[0], F_SETFD, FD_CLOEXEC); + fcntl(pipefd[1], F_SETFD, FD_CLOEXEC); + + pthread_mutex_lock(&u->lock); + u->busnum = bus; + u->devnum = dev; + u->location_id = info.location_id; + u->vid = info.vid; + u->pid = info.pid; + str_copy_trunc(u->serial, info.serial, sizeof(u->serial)); + u->speed_code = info.speed_code; + u->cfg_value = info.cfg_value; + u->blob = blob; + u->blob_len = blob_len; + u->pos = 0; + u->pipe_wr = pipefd[1]; + u->service = IO_OBJECT_NULL; + u->dev = NULL; + u->dev_open = false; + u->dev_open_tried = false; + memset(u->ifaces, 0, sizeof(u->ifaces)); + claimed_mask_reset(u); + /* Nonzero while bound; equal for every fd open on the same device node. */ + devkey_publish( + u, (1ull << 63) | ((uint64_t) (uint32_t) bus << 32) | (uint32_t) dev); + pthread_mutex_unlock(&u->lock); + + /* fd_alloc_from's out_gen, not a later read of the slot: the generation has + * to be this allocation's own stamp, captured inside the fd_lock section + * that stamped it. Re-deriving it after the slot was publishable read + * whatever generation the number carried by then, which in the interleaving + * below is a sibling allocation's -- see the publish. + */ + uint64_t gen = 0; + int guest_fd = + fd_alloc_from(0, FD_USBDEV, pipefd[0], usbdev_fd_cleanup, &gen); + if (guest_fd < 0) { + close(pipefd[0]); + pthread_mutex_lock(&u->lock); + usbdev_teardown_locked(u); + pthread_mutex_unlock(&u->lock); + pthread_mutex_lock(&usbdev_table_lock); + u->used = false; + pthread_mutex_unlock(&usbdev_table_lock); + return -LINUX_EMFILE; + } + + /* Stamp the node path so /proc/self/fd/N readlink reports the guest + * spelling (stage-1 mechanism), and publish the fd's flags. + */ + pthread_mutex_lock(&fd_lock); + if (fd_table[guest_fd].type == FD_USBDEV && + fd_table[guest_fd].host_fd == pipefd[0]) + str_copy_trunc(fd_table[guest_fd].proc_path, path, + sizeof(fd_table[guest_fd].proc_path)); + pthread_mutex_unlock(&fd_lock); + fd_publish_linux_flags(guest_fd, linux_flags); + + usbdev_publish_window_delay(); + pthread_mutex_lock(&usbdev_table_lock); + u->guest_fd = guest_fd; + u->generation = gen; + pthread_mutex_unlock(&usbdev_table_lock); + usbdev_retire_window_delay(); + + /* Two windows, one identity. + * + * usbdev_fd_cleanup matches on guest_fd, which was -1 for everything above + * the bind, so a close arriving before it found no entry to tear down and + * left this one holding its table slot, its descriptor blob and the write + * end of the pipe for the life of the process. The entry is findable now, + * so ask the fd table whether the number is still the one that was + * allocated: a generation that has moved, or a slot that is no longer this + * type, means the close already came and went and this entry has to retire + * itself. Both reads are taken outside usbdev_table_lock, which never nests + * fd_lock. The fd number is still what open(2) returns -- the guest closed + * it, which is its own race to lose, and Linux hands back a number a + * sibling thread can have closed just as readily. + * + * What makes the retire land on this allocation and no other is the tuple, + * and three facts about it rather than the shape of the code. Slot storage + * is static and never freed, so reading u after the slot has been recycled + * is defined. used, dead, guest_fd and generation are all written and read + * under usbdev_table_lock, so the four are read as one value. And + * fd_next_generation (fdtable.c:331) is a globally monotonic counter + * stamped inside the allocating fd_lock section, so gen here is this + * allocation's own number and can never be handed out again -- which is why + * no two live entries can present the same {guest_fd, generation}, why + * usbdev_acquire's match on that pair names exactly one entry, and why the + * claim below can only reach the entry this call created. Both windows + * follow from it: a close-and-reopen before the bind cannot make gen equal + * the reopener's stamp, and a reap-and-reuse after the bind cannot make the + * recycled slot answer to it. + */ + if (fd_current_generation(guest_fd) != gen || + fd_get_type(guest_fd) != FD_USBDEV) + usbdev_retire_unpublished(u, guest_fd, gen); + return guest_fd; +} + +/* read / lseek / fstat */ + +/* The two capability bits, derived the way the kernel derives them, once. + * + * OPEN_FMODE (fs.h:3631) is (flags + 1) & O_ACCMODE, not a comparison against + * O_RDONLY: access modes 0, 1 and 2 give FMODE_READ, FMODE_WRITE and both, and + * access mode 3 gives neither. Mode 3 is reachable -- open(2) takes it, and + * ACC_MODE(3) asks the 0666 node for read plus write, which it grants -- so + * Linux hands back a descriptor that can do nothing: vfs_read and vfs_write + * answer EBADF and every usbdevfs ioctl answers EPERM. + * + * Each gate here used to test the access mode against O_RDONLY or O_WRONLY on + * its own, so all four agreed on modes 0, 1 and 2 and all four were wrong on + * mode 3: the fd read the descriptors blob and was handed the whole ioctl set. + * One derivation with four callers is what keeps the fourth case from having to + * be remembered four times. + */ +#define USBDEV_FMODE_READ 1u +#define USBDEV_FMODE_WRITE 2u + +static unsigned usbdev_fmode(int linux_flags) +{ + return (unsigned) (((linux_flags & LINUX_O_ACCMODE) + 1) & 3); +} + +int64_t usbdev_read(int fd, guest_t *g, uint64_t buf_gva, uint64_t count) +{ + fd_entry_t snap; + if (!fd_snapshot(fd, &snap) || snap.type != FD_USBDEV) + return -LINUX_EBADF; + if (!(usbdev_fmode(snap.linux_flags) & USBDEV_FMODE_READ)) + return -LINUX_EBADF; /* vfs: read needs FMODE_READ */ + usbdev_t *u = usbdev_acquire(fd); + if (!u) + return -LINUX_EBADF; + int64_t ret; + if ((uint64_t) u->pos >= u->blob_len || count == 0) { + ret = 0; + } else { + size_t avail = u->blob_len - (size_t) u->pos; + size_t n = count < avail ? (size_t) count : avail; + if (guest_write(g, buf_gva, u->blob + u->pos, n) < 0) { + ret = -LINUX_EFAULT; + } else { + u->pos += (off_t) n; + ret = (int64_t) n; + } + } + usbdev_release(u); + return ret; +} + +/* pread(2)/preadv(2) arm: the same descriptors blob, served at the caller's + * offset. A positional read never moves the fd position (vfs pread), a negative + * offset is -EINVAL (ksys_pread64 refuses it before the fd lookup, so it + * outranks even -EBADF), and count 0 reads nothing. + */ +int64_t usbdev_pread(int fd, + guest_t *g, + uint64_t buf_gva, + uint64_t count, + int64_t offset) +{ + if (offset < 0) + return -LINUX_EINVAL; + fd_entry_t snap; + if (!fd_snapshot(fd, &snap) || snap.type != FD_USBDEV) + return -LINUX_EBADF; + if (!(usbdev_fmode(snap.linux_flags) & USBDEV_FMODE_READ)) + return -LINUX_EBADF; /* vfs: read needs FMODE_READ */ + usbdev_t *u = usbdev_acquire(fd); + if (!u) + return -LINUX_EBADF; + int64_t ret; + if ((uint64_t) offset >= u->blob_len || count == 0) { + ret = 0; + } else { + size_t avail = u->blob_len - (size_t) offset; + size_t n = count < avail ? (size_t) count : avail; + if (guest_write(g, buf_gva, u->blob + offset, n) < 0) + ret = -LINUX_EFAULT; + else + ret = (int64_t) n; + } + usbdev_release(u); + return ret; +} + +int64_t usbdev_lseek_fd(int fd, int64_t offset, int whence) +{ + if (fd_get_type(fd) != FD_USBDEV) + return INT64_MIN; + usbdev_t *u = usbdev_acquire(fd); + if (!u) + return -LINUX_EBADF; + int64_t ret; + int64_t base; + switch (whence) { + case 0: /* SEEK_SET */ + base = 0; + break; + case 1: /* SEEK_CUR */ + base = u->pos; + break; + default: /* SEEK_END and friends: no_seek_end_llseek -> -EINVAL */ + usbdev_release(u); + return -LINUX_EINVAL; + } + + /* generic_file_llseek_size rejects a result that does not fit off_t + * (-EINVAL). Computing it first is signed overflow, and it is reachable: + * lseek(fd, INT64_MAX, SEEK_SET) followed by lseek(fd, 1, SEEK_CUR) wraps + * negative. + */ + int64_t npos; + if (__builtin_add_overflow(base, offset, &npos) || npos < 0) { + ret = -LINUX_EINVAL; + } else { + u->pos = (off_t) npos; + ret = npos; + } + usbdev_release(u); + return ret; +} + +/* usbdevfs has no write op, so vfs_write answers -EBADF for a descriptor with + * no FMODE_WRITE and -EINVAL for every other one (FMODE_CAN_WRITE), in that + * order. write(2) had this and writev, pwrite and pwritev did not, so those + * three fell through to the host descriptor behind the fd -- the read end of + * the readiness pipe -- and answered -EBADF for a writable fd. + * + * Returns the Linux errno; the caller has already established the fd type. + */ +int64_t usbdev_write_refused(int fd) +{ + fd_entry_t snap; + if (!fd_snapshot(fd, &snap) || snap.type != FD_USBDEV) + return -LINUX_EBADF; + if (!(usbdev_fmode(snap.linux_flags) & USBDEV_FMODE_WRITE)) + return -LINUX_EBADF; + return -LINUX_EINVAL; +} + +/* The read half of the same question, for the empty-vector arm in io.c that has + * to answer the direction test without going through usbdev_read. 0 when the + * descriptor can read, -EBADF when it cannot. + */ +int64_t usbdev_read_refused(int fd) +{ + fd_entry_t snap; + if (!fd_snapshot(fd, &snap) || snap.type != FD_USBDEV) + return -LINUX_EBADF; + return (usbdev_fmode(snap.linux_flags) & USBDEV_FMODE_READ) ? 0 + : -LINUX_EBADF; +} + +int64_t usbdev_fstat(int fd, struct stat *st) +{ + usbdev_t *u = usbdev_acquire(fd); + if (!u) + return -LINUX_EBADF; + int bus = u->busnum, dev = u->devnum; + usbdev_release(u); + if (usb_sysfs_node_stat(bus, dev, st) < 0) + return -LINUX_ENODEV; + return 0; +} + +/* ioctl handlers (entry lock held unless noted) */ + +static int64_t usbdev_do_control(usbdev_t *u, guest_t *g, uint64_t arg) +{ + linux_usbdevfs_ctrltransfer_t ct; + if (guest_read_small(g, arg, &ct, sizeof(ct)) < 0) + return -LINUX_EFAULT; + + /* check_ctrlrecip (devio.c:878-935) runs before the wLength cap + * (devio.c:1177-1183): a request naming an interface or endpoint the device + * does not have is -ENOENT however long it is. Capping first answered + * -EINVAL for requests Linux rejects by recipient. + * + * Vendor-type requests bypass the recipient check; interface/endpoint + * recipients implicitly claim the owning interface first. + */ + if ((ct.bRequestType & 0x60) != 0x40) { + unsigned recip = ct.bRequestType & 0x1f; + if (recip == 1) { /* interface */ + int64_t rc = usbdev_claim_locked(u, ct.wIndex & 0xff); + if (rc < 0) + return rc; + } else if (recip == 2) { /* endpoint */ + int64_t rc = usbdev_check_ep_recip(u, ct.wIndex); + if (rc < 0) + return rc; + } + } + if (ct.wLength > USBDEV_CTRL_MAX) + return -LINUX_EINVAL; + + int64_t rc = usbdev_ensure_dev_plugin(u); + if (rc < 0) + return rc; + usbdev_lazy_device_open(u); + + uint8_t *buf = NULL; + if (ct.wLength > 0) { + buf = malloc(ct.wLength); + if (!buf) + return -LINUX_ENOMEM; + } + bool in = (ct.bRequestType & 0x80) != 0; + if (!in && ct.wLength > 0 && guest_read(g, ct.data, buf, ct.wLength) < 0) { + free(buf); + return -LINUX_EFAULT; + } + + IOUSBDevRequestTO req = { + .bmRequestType = ct.bRequestType, + .bRequest = ct.bRequest, + .wValue = ct.wValue, + .wIndex = ct.wIndex, + .wLength = ct.wLength, + .pData = buf, + .noDataTimeout = ct.timeout, + .completionTimeout = ct.timeout, + }; + IOReturn r = (*u->dev)->DeviceRequestTO(u->dev, &req); + if ((uint32_t) r == (uint32_t) kIOReturnNotOpen && !u->dev_open) { + /* Some requests demand an open device; retry once after opening. */ + IOReturn ro = (*u->dev)->USBDeviceOpen(u->dev); + if (ro == kIOReturnSuccess) { + u->dev_open = true; + r = (*u->dev)->DeviceRequestTO(u->dev, &req); + } + } + int64_t err = ioret_neg_errno(r); + if (err < 0) { + /* On -ETIMEDOUT/-EINTR partial IN data is NOT copied out + * (devio.c:1227). + */ + free(buf); + return err; + } + int64_t actlen = req.wLenDone; + if (in && actlen > 0 && guest_write(g, ct.data, buf, (size_t) actlen) < 0) { + free(buf); + return -LINUX_EFAULT; + } + free(buf); + return actlen; +} + +static int64_t usbdev_do_bulk(usbdev_t *u, guest_t *g, uint64_t arg) +{ + linux_usbdevfs_bulktransfer_t bt; + if (guest_read_small(g, arg, &bt, sizeof(bt)) < 0) + return -LINUX_EFAULT; + + /* do_proc_bulk resolves and claims the endpoint's interface before it looks + * at the length (devio.c:1289-1298), so an absent endpoint is -ENOENT + * whatever the length says. Checking the length first answered -ENOMEM and + * -EINVAL for requests Linux rejects by endpoint. + */ + usbdev_iface_t *fi; + uint8_t pipe; + int64_t rc = usbdev_pipe_for_ep(u, bt.ep, &fi, &pipe); + if (rc < 0) + return rc; + + /* proc_bulk: only a near-INT_MAX length is malformed (-EINVAL, + * devio.c:1298); a merely-too-large one fails the usbfs_memory_mb allowance + * with -ENOMEM (devio.c:1308-1316). + */ + if (bt.len >= (uint32_t) INT32_MAX) + return -LINUX_EINVAL; + + uint8_t type = fi->pipe_type[pipe - 1]; + if (type == kUSBInterrupt) { + /* Linux converts BULK-on-interrupt-ep to an interrupt URB + * (devio.c:1327); ReadPipeTO/WritePipeTO reject interrupt pipes. + * TODO(later): async submit + timed wait. + */ + log_warn("usbdev: sync BULK on interrupt ep 0x%02x unsupported", bt.ep); + return -LINUX_EINVAL; + } + if (type != kUSBBulk) + return -LINUX_EINVAL; /* control/iso ep: proc_bulk EINVAL */ + + /* The allowance, taken where Linux takes it: immediately before the buffer + * this transfer needs (devio.c:1308-1316), and given back on every exit + * from here down. The arms above answer EINVAL and run before the charge, + * so their order is unchanged; everything below is charged, which is why + * the OUT path's guest_read failure now joins the single exit instead of + * returning from the middle. + */ + uint64_t charge = (uint64_t) bt.len + USBDEV_URB_OVERHEAD; + if (!usbdev_memory_charge(charge)) + return -LINUX_ENOMEM; + + uint8_t *buf = NULL; + if (bt.len > 0) { + buf = malloc(bt.len); + if (!buf) { + usbdev_memory_refund(charge); + return -LINUX_ENOMEM; + } + } + int64_t ret; + if (bt.ep & 0x80) { + UInt32 size = bt.len; + IOReturn r = (*fi->intf)->ReadPipeTO(fi->intf, pipe, buf, &size, + bt.timeout, bt.timeout); + int64_t err = ioret_neg_errno(r); + if (err < 0) { + ret = err; /* partial data not copied on error, as Linux */ + } else if (size > 0 && guest_write(g, bt.data, buf, size) < 0) { + ret = -LINUX_EFAULT; + } else { + ret = size; + } + } else if (bt.len > 0 && guest_read(g, bt.data, buf, bt.len) < 0) { + ret = -LINUX_EFAULT; + } else { + IOReturn r = (*fi->intf)->WritePipeTO(fi->intf, pipe, buf, bt.len, + bt.timeout, bt.timeout); + if ((uint32_t) r == (uint32_t) kIOReturnUnderrun) { + /* WritePipeTO has no out-length, so the count actually sent is not + * recoverable here. ioret_neg_errno folds Underrun into success for + * the IN path, where IOKit does report the length; folding it here + * would report a short write as a complete one. + */ + log_warn("usbdev: short bulk OUT on ep 0x%02x, length unknown", + bt.ep); + ret = -LINUX_EIO; + } else { + int64_t err = ioret_neg_errno(r); + ret = err < 0 ? err : bt.len; + } + } + free(buf); + usbdev_memory_refund(charge); + return ret; +} + +/* Whether another usbfs fd open on this device holds ifnum. Reads the lock-free + * mirrors for the reason usbdev_claimed_elsewhere does: the caller holds its + * own entry lock, and no path takes a second one. + */ +static bool usbdev_iface_claimed_elsewhere(const usbdev_t *u, unsigned ifnum) +{ + if (ifnum >= USBDEV_MAX_IFACES) + return false; + uint64_t key = devkey_load(u); + for (int i = 0; i < USBDEV_MAX_FDS; i++) { + const usbdev_t *o = &usbdev_fds[i]; + if (o == u) + continue; + if (devkey_load(o) == key && + (claimed_mask_load(o) & (1ull << ifnum)) != 0) + return true; + } + return false; +} + +static int64_t usbdev_do_getdriver(usbdev_t *u, guest_t *g, uint64_t arg) +{ + linux_usbdevfs_getdriver_t gd; + if (guest_read_small(g, arg, &gd, sizeof(gd)) < 0) + return -LINUX_EFAULT; + + /* Ahead of everything below: an interface question about a device that is + * not reachable is -ENODEV, not "no driver" (devio.c's connected() gate). + */ + int64_t drc = usbdev_ensure_dev_plugin(u); + if (drc < 0) + return drc; + + /* proc_getdriver: no such interface and no driver are the same answer, + * -ENODATA (devio.c:1445-1446); usb_ifnum_to_if has no range error. + */ + if (gd.interface >= USBDEV_MAX_IFACES) + return -LINUX_ENODATA; + memset(gd.driver, 0, sizeof(gd.driver)); + + /* usbfs is one driver device-wide, so an interface another usbfs fd holds + * reports usbfs here too, exactly as intf->dev.driver would. + */ + if (u->ifaces[gd.interface].claimed || + usbdev_iface_claimed_elsewhere(u, gd.interface)) { + str_copy_trunc(gd.driver, "usbfs", sizeof(gd.driver)); + } else { + io_service_t ifs = usbdev_iface_service(u, gd.interface); + if (ifs == IO_OBJECT_NULL) + return -LINUX_ENODATA; /* usb_ifnum_to_if NULL */ + bool bound = usbdev_iface_driver(ifs, gd.driver, sizeof(gd.driver)); + IOObjectRelease(ifs); + if (!bound) + return -LINUX_ENODATA; + } + if (guest_write_small(g, arg, &gd, sizeof(gd)) < 0) + return -LINUX_EFAULT; + return 0; +} + +static int64_t usbdev_do_setinterface(usbdev_t *u, guest_t *g, uint64_t arg) +{ + linux_usbdevfs_setinterface_t si; + if (guest_read_small(g, arg, &si, sizeof(si)) < 0) + return -LINUX_EFAULT; + int64_t rc = usbdev_claim_locked(u, si.interface); /* implicit claim */ + if (rc < 0) + return rc; + + /* usb_altnum_to_altsetting compares the interface's __u8 bAlternateSetting + * against the caller's unsigned argument (usb.c:391), so a value above 255 + * matches no altsetting and usb_set_interface answers -EINVAL + * (message.c:1548). SetAlternateInterface takes a UInt8, and narrowing into + * it made altsetting 256 select setting 0: an argument Linux refuses + * changed the interface instead. Checked after the claim, because + * proc_setintf runs checkintf before usb_set_interface (devio.c:1533-1539) + * and a claim failure is the answer the guest gets first. + */ + if (si.altsetting > 0xff) + return -LINUX_EINVAL; + usbdev_iface_t *fi = &u->ifaces[si.interface]; + IOReturn r = + (*fi->intf)->SetAlternateInterface(fi->intf, (UInt8) si.altsetting); + if (r != kIOReturnSuccess) { + int64_t err = ioret_neg_errno(r); + log_debug("usbdev: SetAlternateInterface(%u, %u) -> 0x%x", si.interface, + si.altsetting, r); + + /* usb_set_interface answers -EINVAL for an altsetting the interface + * does not have (usb_find_alt_setting NULL, message.c). IOKit's code + * for that is not in the errno table, so it arrived as the map's + * default -EPROTO, an errno no usbfs caller expects from an argument + * mistake. Rewrite the two answers that can mean "bad altsetting" and + * pass every other one through, so a device-loss or aborted-transfer + * answer keeps its own meaning. + */ + if (err == -LINUX_ENOENT || err == -LINUX_EPROTO) + err = -LINUX_EINVAL; + return err; + } + return usbdev_build_pipe_map(fi); +} + +/* proc_setconfig's claim check is device-wide (usb_interface_claimed, + * devio.c:1561-1576): a claim through ANY usbfs open of this device blocks + * SetConfiguration, not only one through the calling fd. The caller holds its + * own entry lock and no path takes a second one, so the other slots' entry + * locks cannot be taken here; their lock-free mirrors are read instead. + */ +static bool usbdev_claimed_elsewhere(usbdev_t *u) +{ + uint64_t key = devkey_load(u); + for (int i = 0; i < USBDEV_MAX_FDS; i++) { + usbdev_t *o = &usbdev_fds[i]; + if (o == u) + continue; + if (devkey_load(o) == key && claimed_mask_load(o) != 0) + return true; + } + return false; +} + +static int64_t usbdev_do_setconfiguration(usbdev_t *u, guest_t *g, uint64_t arg) +{ + uint32_t cfg; + if (guest_read_small(g, arg, &cfg, sizeof(cfg)) < 0) + return -LINUX_EFAULT; + /* -1/0 -> unconfigure (message.c:2064); SetConfiguration(0) does that. */ + if (cfg == 0xffffffffu) + cfg = 0; + if (cfg > 255) + return -LINUX_EINVAL; + + /* proc_setconfig: -EBUSY when ANY interface of the device is claimed -- by + * this fd, by another usbfs fd, or by a bound host driver + * (devio.c:1561-1578). + */ + for (int i = 0; i < USBDEV_MAX_IFACES; i++) + if (u->ifaces[i].claimed) + return -LINUX_EBUSY; + if (usbdev_claimed_elsewhere(u)) + return -LINUX_EBUSY; + int64_t rc = usbdev_ensure_dev_plugin(u); + if (rc < 0) + return rc; + + /* A bound host (Apple) driver claims its interface exactly like a Linux + * driver would (usb_interface_claimed covers every driver, not just usbfs): + * one iterator pass over the device's interfaces. + */ + IOUSBFindInterfaceRequest fr = { + .bInterfaceClass = kIOUSBFindInterfaceDontCare, + .bInterfaceSubClass = kIOUSBFindInterfaceDontCare, + .bInterfaceProtocol = kIOUSBFindInterfaceDontCare, + .bAlternateSetting = kIOUSBFindInterfaceDontCare, + }; + io_iterator_t it = IO_OBJECT_NULL; + if ((*u->dev)->CreateInterfaceIterator(u->dev, &fr, &it) == + kIOReturnSuccess) { + bool bound = false; + io_service_t svc; + while ((svc = IOIteratorNext(it))) { + char drv[64]; + if (!bound && usbdev_iface_driver(svc, drv, sizeof(drv))) + bound = true; + IOObjectRelease(svc); + } + IOObjectRelease(it); + if (bound) + return -LINUX_EBUSY; + } + usbdev_lazy_device_open(u); + if (!u->dev_open) + return -LINUX_EBUSY; /* exclusive holder elsewhere */ + IOReturn r = (*u->dev)->SetConfiguration(u->dev, (UInt8) cfg); + if (r != kIOReturnSuccess) { + int64_t err = ioret_neg_errno(r); + return err == -LINUX_ENOENT ? -LINUX_EINVAL : err; + } + u->cfg_value = cfg; + return 0; +} + +static int64_t usbdev_do_clear_halt(usbdev_t *u, guest_t *g, uint64_t arg) +{ + uint32_t ep; + if (guest_read_small(g, arg, &ep, sizeof(ep)) < 0) + return -LINUX_EFAULT; + usbdev_iface_t *fi; + uint8_t pipe; + int64_t rc = usbdev_pipe_for_ep(u, ep, &fi, &pipe); + if (rc < 0) + return rc; + + /* ClearPipeStallBothEnds == CLEAR_FEATURE(ENDPOINT_HALT) + host-side toggle + * reset (IOUSBLib.h:2928-2941), exactly usb_clear_halt. + */ + return ioret_neg_errno((*fi->intf)->ClearPipeStallBothEnds(fi->intf, pipe)); +} + +static int64_t usbdev_do_resetep(usbdev_t *u, guest_t *g, uint64_t arg) +{ + /* proc_resetep is a host-side toggle/seq reset only (message.c:1377). + * ClearPipeStallBothEnds is the closest IOKit equivalent (it also sends the + * wire CLEAR_FEATURE, a benign superset). + */ + return usbdev_do_clear_halt(u, g, arg); +} + +static int64_t usbdev_do_disconnect_claim(usbdev_t *u, guest_t *g, uint64_t arg) +{ + linux_usbdevfs_disconnect_claim_t dc; + if (guest_read_small(g, arg, &dc, sizeof(dc)) < 0) + return -LINUX_EFAULT; + + /* proc_disconnect_claim has no range check of its own: usb_ifnum_to_if + * answers for the number and a NULL result is -EINVAL (devio.c:2467-2469), + * which is the opposite of claimintf's -ENOENT for the same shape. + */ + if (dc.interface >= USBDEV_MAX_IFACES) + return -LINUX_EINVAL; + dc.driver[sizeof(dc.driver) - 1] = '\0'; + int64_t drc = usbdev_ensure_dev_plugin(u); + if (drc < 0) + return drc; + + char drv[256] = ""; + bool bound = false; + if (u->ifaces[dc.interface].claimed || + usbdev_iface_claimed_elsewhere(u, dc.interface)) { + str_copy_trunc(drv, "usbfs", sizeof(drv)); + bound = true; + } else { + io_service_t ifs = usbdev_iface_service(u, dc.interface); + if (ifs == IO_OBJECT_NULL) + return -LINUX_EINVAL; + bound = usbdev_iface_driver(ifs, drv, sizeof(drv)); + IOObjectRelease(ifs); + } + if (bound) { + if ((dc.flags & USBDEVFS_DISCONNECT_CLAIM_IF_DRIVER) && + strcmp(dc.driver, drv) != 0) + return -LINUX_EBUSY; + if ((dc.flags & USBDEVFS_DISCONNECT_CLAIM_EXCEPT_DRIVER) && + strcmp(dc.driver, drv) == 0) + return -LINUX_EBUSY; + if (strcmp(drv, "usbfs") != 0) { + /* A real (Apple) driver would need whole-device capture, which + * requires root or com.apple.vm.device-access; mirror the + * privileges-dropped Linux answer (devio.c:2475-2476). + */ + return -LINUX_EACCES; + } + } + return usbdev_claim_locked(u, dc.interface); +} + +static int64_t usbdev_do_driver_ioctl(usbdev_t *u, guest_t *g, uint64_t arg) +{ + linux_usbdevfs_ioctl_t ic; + if (guest_read_small(g, arg, &ic, sizeof(ic)) < 0) + return -LINUX_EFAULT; + if (ic.ifno < 0 || ic.ifno >= USBDEV_MAX_IFACES) + return -LINUX_EINVAL; + unsigned ifnum = (unsigned) ic.ifno; + int64_t drc = usbdev_ensure_dev_plugin(u); + if (drc < 0) + return drc; + switch ((uint32_t) ic.ioctl_code) { + case USBDEVFS_IOCTL_DISCONNECT: { + if (u->ifaces[ifnum].claimed) + return usbdev_release_locked(u, ifnum); /* unbind "usbfs" */ + if (usbdev_iface_claimed_elsewhere(u, ifnum)) { + /* Linux releases the usbfs claim whichever open made it and answers + * 0. The claim here is another fd's IOKit handle, which this one + * cannot close (documented gap). + */ + return -LINUX_EBUSY; + } + io_service_t ifs = usbdev_iface_service(u, ifnum); + if (ifs == IO_OBJECT_NULL) + return -LINUX_EINVAL; + char drv[64]; + bool bound = usbdev_iface_driver(ifs, drv, sizeof(drv)); + IOObjectRelease(ifs); + if (!bound) + return -LINUX_ENODATA; + return -LINUX_EACCES; /* cannot unbind Apple drivers non-root */ + } + case USBDEVFS_IOCTL_CONNECT: { + /* proc_ioctl: an interface that already has a driver is -EBUSY, and + * only a free one reaches device_attach (devio.c:2362-2368). Re-attach + * itself stays out of reach -- IOKit rematches on its own schedule -- + * but answering -EACCES for the bound case reported the wrong reason + * for a call Linux never gets that far with. + */ + if (u->ifaces[ifnum].claimed || + usbdev_iface_claimed_elsewhere(u, ifnum)) + return -LINUX_EBUSY; + io_service_t ifs = usbdev_iface_service(u, ifnum); + if (ifs == IO_OBJECT_NULL) + return -LINUX_EINVAL; + char drv[64]; + bool bound = usbdev_iface_driver(ifs, drv, sizeof(drv)); + IOObjectRelease(ifs); + if (bound) + return -LINUX_EBUSY; + return -LINUX_EACCES; /* device_attach needs the host's consent */ + } + default: + return -LINUX_ENOTTY; + } +} + +/* Registry 'Device Speed' -> USB_SPEED_* enum (ch9.h:1217-1222): the ioctl's + * return value, not an out parameter. + */ +static int64_t usbdev_speed_enum(unsigned code) +{ + switch (code) { + case 0: + return 1; /* LOW */ + case 1: + return 2; /* FULL */ + case 2: + return 3; /* HIGH */ + case 3: + return 5; /* SUPER */ + case 4: + case 5: + return 6; /* SUPER_PLUS */ + default: + return 2; + } +} + +int64_t usbdev_ioctl(guest_t *g, int fd, uint64_t request, uint64_t arg) +{ + fd_entry_t snap; + if (!fd_snapshot(fd, &snap) || snap.type != FD_USBDEV) + return -LINUX_EBADF; + /* Every usbdev ioctl needs FMODE_WRITE (devio.c:2605-2606). */ + if (!(usbdev_fmode(snap.linux_flags) & USBDEV_FMODE_WRITE)) + return -LINUX_EPERM; + + usbdev_t *u = usbdev_acquire(fd); + if (!u) + return -LINUX_EBADF; + + int64_t ret; + switch ((uint32_t) request) { + case USBDEVFS_CLAIMINTERFACE: { + uint32_t ifnum; + ret = guest_read_small(g, arg, &ifnum, sizeof(ifnum)) < 0 + ? -LINUX_EFAULT + : usbdev_claim_locked(u, ifnum); + break; + } + case USBDEVFS_RELEASEINTERFACE: { + uint32_t ifnum; + ret = guest_read_small(g, arg, &ifnum, sizeof(ifnum)) < 0 + ? -LINUX_EFAULT + : usbdev_release_locked(u, ifnum); + break; + } + case USBDEVFS_SETINTERFACE: + ret = usbdev_do_setinterface(u, g, arg); + break; + case USBDEVFS_SETCONFIGURATION: + ret = usbdev_do_setconfiguration(u, g, arg); + break; + case USBDEVFS_CLEAR_HALT: + ret = usbdev_do_clear_halt(u, g, arg); + break; + case USBDEVFS_RESETEP: + ret = usbdev_do_resetep(u, g, arg); + break; + case USBDEVFS_GETDRIVER: + ret = usbdev_do_getdriver(u, g, arg); + break; + case USBDEVFS_GET_CAPABILITIES: { + uint32_t caps = USBDEV_CAPS; + ret = guest_write_small(g, arg, &caps, sizeof(caps)) < 0 ? -LINUX_EFAULT + : 0; + break; + } + case USBDEVFS_GET_SPEED: + ret = usbdev_speed_enum(u->speed_code); + break; + case USBDEVFS_CONNECTINFO: { + linux_usbdevfs_connectinfo_t ci = { + .devnum = (uint32_t) u->devnum, + .slow = u->speed_code == 0, + }; + ret = + guest_write_small(g, arg, &ci, sizeof(ci)) < 0 ? -LINUX_EFAULT : 0; + break; + } + case USBDEVFS_CONTROL: + ret = usbdev_do_control(u, g, arg); + break; + case USBDEVFS_BULK: + ret = usbdev_do_bulk(u, g, arg); + break; + case USBDEVFS_RESET: { + /* Stage-2 deviation (see file header): clear stalls on every claimed + * pipe instead of re-enumerating, and report success. + */ + for (int i = 0; i < USBDEV_MAX_IFACES; i++) { + usbdev_iface_t *fi = &u->ifaces[i]; + if (!fi->claimed) + continue; + for (int p = 1; p <= fi->npipes; p++) + (void) (*fi->intf)->ClearPipeStallBothEnds(fi->intf, (UInt8) p); + } + log_debug( + "usbdev: RESET emulated as pipe-stall clear (no " + "re-enumeration)"); + ret = 0; + break; + } + case USBDEVFS_DISCONNECT_CLAIM: + ret = usbdev_do_disconnect_claim(u, g, arg); + break; + case USBDEVFS_IOCTL: + ret = usbdev_do_driver_ioctl(u, g, arg); + break; + case USBDEVFS_SUBMITURB: + log_warn("usbdev: SUBMITURB not implemented (stage 3)"); + ret = -LINUX_ENOTTY; + break; + case USBDEVFS_DISCARDURB: + case USBDEVFS_REAPURB: + case USBDEVFS_REAPURBNDELAY: + case USBDEVFS_DISCSIGNAL: + log_debug("usbdev: async URB ioctl 0x%llx not implemented (stage 3)", + (unsigned long long) request); + ret = -LINUX_ENOTTY; + break; + default: + ret = -LINUX_ENOTTY; + break; + } + usbdev_release(u); + return ret; +} diff --git a/src/syscall/usbdev.h b/src/syscall/usbdev.h new file mode 100644 index 00000000..0429d403 --- /dev/null +++ b/src/syscall/usbdev.h @@ -0,0 +1,74 @@ +/* + * usbdevfs (/dev/bus/usb/BBB/DDD) fd emulation over IOKit + * + * Copyright 2026 elfuse contributors + * SPDX-License-Identifier: Apache-2.0 + * + * Stage 2 of the usbdevfs emulation: a real FD_USBDEV fd type whose synchronous + * ioctls (CLAIMINTERFACE, CONTROL, BULK, ...) are served by + * IOUSBDeviceInterface/IOUSBInterfaceInterface plugins. Async URBs + * (SUBMITURB/REAPURB) are stage 3. + */ + +#pragma once + +#include +#include + +#include "core/guest.h" + +/* Register the FD_USBDEV cleanup hook. Called once from syscall_init(). */ +void usbdev_init(void); + +/* Constructor: open("/dev/bus/usb/BBB/DDD") for any access mode except O_PATH. + * + * Returns the guest fd, -LINUX_* on error, or INT64_MIN when the path is not a + * usbdev node (caller falls through to the generic intercepts; the O_PATH case + * falls through on purpose so the stage-1 placeholder keeps serving path-only + * fds). + */ +int64_t usbdev_open_path(const char *path, int linux_flags); + +/* read() on an FD_USBDEV fd: the usbfs descriptors blob at the fd's file + * position (device descriptor then raw config descriptors, devio.c:311-390). + */ +int64_t usbdev_read(int fd, guest_t *g, uint64_t buf_gva, uint64_t count); + +/* pread()/preadv() on an FD_USBDEV fd: the descriptors blob at the caller's + * offset, without moving the fd's file position. Negative offset -> -EINVAL. + */ +int64_t usbdev_pread(int fd, + guest_t *g, + uint64_t buf_gva, + uint64_t count, + int64_t offset); + +/* lseek() arm: SEEK_SET/SEEK_CUR against the blob position, SEEK_END is -EINVAL + * (no_seek_end_llseek). + * + * Returns INT64_MIN when fd is not FD_USBDEV. + */ +int64_t usbdev_lseek_fd(int fd, int64_t offset, int whence); + +/* The answer every write-family entry point owes an FD_USBDEV fd: -EBADF when + * the fd has no FMODE_WRITE, -EINVAL otherwise (usbdevfs has no write op, so + * FMODE_CAN_WRITE is clear). write, writev, pwrite and pwritev must all route + * here rather than reaching the readiness pipe behind the fd. + */ +int64_t usbdev_write_refused(int fd); + +/* The read half: 0 when the fd has FMODE_READ, -EBADF when it does not. For the + * empty-vector arms, which owe the direction test without a transfer. Both + * halves derive the capability bits from OPEN_FMODE, so an access mode with + * neither bit is refused in both directions. + */ +int64_t usbdev_read_refused(int fd); + +/* fstat(): char device 189:minor, matching the path-stat intercept. */ +int64_t usbdev_fstat(int fd, struct stat *st); + +/* The USBDEVFS_* ioctl set. + * + * Returns -LINUX_* or the (possibly positive) ioctl result. + */ +int64_t usbdev_ioctl(guest_t *g, int fd, uint64_t request, uint64_t arg); diff --git a/tests/test-usb-sysfs.c b/tests/test-usb-sysfs.c index 135d62db..d5317f80 100644 --- a/tests/test-usb-sysfs.c +++ b/tests/test-usb-sysfs.c @@ -1104,9 +1104,40 @@ static int check_devices(void) EXPECT_TRUE(open(node, O_RDONLY | O_DIRECTORY) < 0 && errno == ENOTDIR, "O_DIRECTORY on a char device"); - TEST("a writable open of the node is EACCES"); - EXPECT_TRUE(open(node, O_RDWR) < 0 && errno == EACCES, - "O_RDWR on the node"); + /* The usbdevfs fd now serves writable opens, and it has to keep the + * one-blob invariant: reading it must still return exactly what the + * descriptors attribute does, or the two views have drifted apart the + * moment a second generator appeared. + */ + TEST("a writable open of the node serves the same blob"); + int wfd = na > 0 ? open(node, O_RDWR) : -1; + if (wfd < 0) { + FAIL("O_RDWR on the node"); + } else { + /* One byte over the attribute's length, so a node serving more than + * the attribute does is a failure rather than a prefix that + * silently compares equal. + */ + size_t cap = (size_t) na + 1; + unsigned char *w = malloc(cap); + ssize_t nw = 0; + if (!w) { + FAIL("malloc"); + } else { + while ((size_t) nw < cap) { + ssize_t r = read(wfd, w + nw, cap - (size_t) nw); + if (r < 0 && errno == EINTR) + continue; + if (r <= 0) + break; + nw += r; + } + EXPECT_TRUE(nw == na && memcmp(w, a, (size_t) na) == 0, + "usbdevfs fd read matches descriptors"); + free(w); + } + close(wfd); + } free(a); free(b); diff --git a/tests/test-usbdev-ioctl.c b/tests/test-usbdev-ioctl.c new file mode 100644 index 00000000..24d8000a --- /dev/null +++ b/tests/test-usbdev-ioctl.c @@ -0,0 +1,1219 @@ +/* + * The usbdevfs descriptor's contract, without hardware + * + * Copyright 2026 elfuse contributors + * SPDX-License-Identifier: Apache-2.0 + * + * Code under test: src/syscall/usbdev.c, plus the write-family and lseek arms + * in src/syscall/io.c and src/syscall/fs.c. + * + * This lane runs against ELFUSE_USB_FIXTURE, whose devices are modeled but have + * no IOKit service behind them. That is the whole point: it is exactly the + * shape of a device the model knows about and the machine cannot reach, so + * every answer below is deterministic on any host, plugged in or not. What it + * covers is the half of usbdevfs that is decided before the wire -- the file + * contract (read, pread, lseek, the write family, fstat, dup), the ioctl gates + * (FMODE_WRITE, unknown codes), and every argument the kernel validates from + * the descriptors it already has (interface numbers, endpoint addresses, + * transfer lengths, and the order those checks run in). The half that needs a + * device -- a real claim, a real transfer, arbitration against a bound host + * driver -- cannot be asserted here and is not pretended to be. + * + * Where elfuse knowingly answers something other than Linux, the case is + * printed as an XFAIL carrying both values rather than dropped, so the gap + * stays visible in the lane's own output. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "test-harness.h" + +int passes = 0, fails = 0; + +#define NODE "/dev/bus/usb/001/001" + +/* Only ELFUSE_USB_FIXTURE=badifnum stands this one up: one interface declaring + * bInterfaceNumber 200, carrying endpoint 0x81. + */ +#define BAD_IFNUM_NODE "/dev/bus/usb/001/002" + +/* The default fixture's second device, on the other bus. A race that leaves the + * wrong entry answering one fd number is invisible while both entries describe + * the same device, so the reopen half of the publish-window races opens this + * one and reads its idVendor back. + */ +#define OTHER_NODE "/dev/bus/usb/002/001" +#define OTHER_VID 0x2109 +#define NODE_VID 0x1d6b +#define USB_MAJOR 189 + +#ifndef RWF_APPEND +#define RWF_APPEND 0x00000010 +#endif + +/* pwritev2 by hand rather than through the C library: the guest's libc reports + * EOPNOTSUPP for any failing pwritev2 carrying a nonzero flag, whatever the + * kernel said, and the assertions below are about what the kernel said. The + * sixth argument is the high half of the offset, zero on a 64-bit ABI. + */ +static long pwritev2_raw(int fd, + const struct iovec *iov, + int iovcnt, + long offset, + int flags) +{ + return syscall(SYS_pwritev2, (long) fd, (long) iov, (long) iovcnt, offset, + 0L, (long) flags); +} + +/* uapi/linux/usbdevice_fs.h, spelled out rather than included: the guest + * sysroot need not carry the header, and the numbers are part of the contract + * under test. + */ +#define USBDEVFS_CONTROL 0xc0185500u +#define USBDEVFS_BULK 0xc0185502u +#define USBDEVFS_RESETEP 0x80045503u +#define USBDEVFS_SETINTERFACE 0x80085504u +#define USBDEVFS_SETCONFIGURATION 0x80045505u +#define USBDEVFS_GETDRIVER 0x41045508u +#define USBDEVFS_DISCARDURB 0x0000550bu +#define USBDEVFS_RESET 0x00005514u +#define USBDEVFS_CLEAR_HALT 0x80045515u +#define USBDEVFS_DISCONNECT 0x00005516u +#define USBDEVFS_CONNECT 0x00005517u +#define USBDEVFS_CLAIMINTERFACE 0x8004550fu +#define USBDEVFS_RELEASEINTERFACE 0x80045510u +#define USBDEVFS_CONNECTINFO 0x40085511u +#define USBDEVFS_IOCTL 0xc0105512u +#define USBDEVFS_SUBMITURB 0x8038550au +#define USBDEVFS_GET_CAPABILITIES 0x8004551au +#define USBDEVFS_DISCONNECT_CLAIM 0x8108551bu +#define USBDEVFS_GET_SPEED 0x0000551fu + +struct ctrltransfer { + uint8_t bRequestType, bRequest; + uint16_t wValue, wIndex, wLength; + uint32_t timeout; + void *data; +}; + +struct bulktransfer { + unsigned int ep, len, timeout; + void *data; +}; + +struct setinterface { + unsigned int interface, altsetting; +}; + +struct getdriver { + unsigned int interface; + char driver[256]; +}; + +struct usbdevfs_ioctl { + int ifno, ioctl_code; + void *data; +}; + +struct disconnect_claim { + unsigned int interface, flags; + char driver[256]; +}; + +/* ioctl(2) collapses every failure onto -1; the assertions below are about + * which errno, so report it as a negative value the way the kernel does. + */ +static long io(int fd, unsigned long req, void *arg) +{ + int r = ioctl(fd, req, arg); + return r < 0 ? -errno : r; +} + +static void check_open_agrees(void) +{ + printf("test-usbdev-ioctl: the node's three permission answers\n"); + + /* One node, three entry points onto "may this guest write it". Stage 1 had + * no writable open to disagree with; stage 2 does, and a 0664 node owned by + * a uid the guest does not have made access(W_OK) refuse what open(O_RDWR) + * then served. + */ + TEST("access(R_OK) on the node"); + EXPECT_EQ(access(NODE, R_OK), 0, "access R_OK"); + TEST("access(W_OK) on the node"); + EXPECT_EQ(access(NODE, W_OK), 0, "access W_OK"); + + int rw = open(NODE, O_RDWR); + TEST("open(O_RDWR) on the node"); + EXPECT_TRUE(rw >= 0, "open O_RDWR"); + + /* The constructor resolves its IOKit device on first use, not at open, so a + * modeled device with nothing behind it still opens and still reads. + */ + int ro = open(NODE, O_RDONLY); + TEST("open(O_RDONLY) on the node"); + EXPECT_TRUE(ro >= 0, "open O_RDONLY"); + + unsigned char buf[64]; + ssize_t n = ro >= 0 ? read(ro, buf, sizeof(buf)) : -1; + TEST("read serves the descriptors blob"); + EXPECT_TRUE(n >= 18 && buf[0] == 18 && buf[1] == 1, "blob device desc"); + + struct stat st; + TEST("fstat reports char 189:minor"); + EXPECT_TRUE(ro >= 0 && fstat(ro, &st) == 0 && S_ISCHR(st.st_mode) && + major(st.st_rdev) == USB_MAJOR && minor(st.st_rdev) == 0, + "fstat rdev"); + + TEST("dup of a usbdevfs fd is EBADF"); + EXPECT_ERRNO(dup(ro), EBADF, "dup"); + + if (rw >= 0) + close(rw); + if (ro >= 0) + close(ro); +} + +/* Linux: vfs_write checks FMODE_WRITE before FMODE_CAN_WRITE, so an O_RDONLY fd + * is EBADF and a writable one EINVAL -- from write, writev, pwrite and pwritev + * alike. Only write() answered that here; the other three reached the readiness + * pipe behind the fd and reported its EBADF for a writable fd. + */ +static void check_write_family(void) +{ + printf("\ntest-usbdev-ioctl: the write family\n"); + char b[8] = {0}; + struct iovec iov[2] = {{b, 4}, {b + 4, 4}}; + + int rw = open(NODE, O_RDWR); + if (rw < 0) { + TEST("writable open for the write family"); + FAIL("open O_RDWR"); + return; + } + TEST("write on a writable fd is EINVAL"); + EXPECT_ERRNO(write(rw, b, 4), EINVAL, "write"); + TEST("writev on a writable fd is EINVAL"); + EXPECT_ERRNO(writev(rw, iov, 2), EINVAL, "writev"); + TEST("pwrite on a writable fd is EINVAL"); + EXPECT_ERRNO(pwrite(rw, b, 4, 0), EINVAL, "pwrite"); + TEST("pwritev on a writable fd is EINVAL"); + EXPECT_ERRNO(pwritev(rw, iov, 2, 0), EINVAL, "pwritev"); + close(rw); + + int ro = open(NODE, O_RDONLY); + if (ro < 0) { + TEST("read-only open for the write family"); + FAIL("open O_RDONLY"); + return; + } + TEST("write on a read-only fd is EBADF"); + EXPECT_ERRNO(write(ro, b, 4), EBADF, "write ro"); + TEST("writev on a read-only fd is EBADF"); + EXPECT_ERRNO(writev(ro, iov, 2), EBADF, "writev ro"); + TEST("pwrite on a read-only fd is EBADF"); + EXPECT_ERRNO(pwrite(ro, b, 4, 0), EBADF, "pwrite ro"); + TEST("pwritev on a read-only fd is EBADF"); + EXPECT_ERRNO(pwritev(ro, iov, 2, 0), EBADF, "pwritev ro"); + + /* Every ioctl needs FMODE_WRITE (devio.c:2605), and that gate is ahead of + * everything else, including the device lookup. + */ + uint32_t caps = 0; + TEST("an ioctl on a read-only fd is EPERM"); + EXPECT_EQ(io(ro, USBDEVFS_GET_CAPABILITIES, &caps), -EPERM, "ro ioctl"); + close(ro); +} + +static void check_seek(void) +{ + printf("\ntest-usbdev-ioctl: the file position\n"); + int fd = open(NODE, O_RDWR); + if (fd < 0) { + TEST("open for the seek contract"); + FAIL("open"); + return; + } + TEST("SEEK_END is EINVAL"); + EXPECT_ERRNO(lseek(fd, 0, SEEK_END), EINVAL, "SEEK_END"); + TEST("SEEK_SET to INT64_MAX is allowed"); + EXPECT_EQ(lseek(fd, INT64_MAX, SEEK_SET), INT64_MAX, "SEEK_SET max"); + + /* One past it is not: the sum used to be computed before the range test, + * which is signed overflow, and it wrapped to a negative position. + */ + TEST("one byte past INT64_MAX is EINVAL"); + EXPECT_ERRNO(lseek(fd, 1, SEEK_CUR), EINVAL, "SEEK_CUR overflow"); + TEST("the failed seek left the position alone"); + EXPECT_EQ(lseek(fd, 0, SEEK_CUR), INT64_MAX, "position after EINVAL"); + + unsigned char a[8], b[8]; + TEST("pread does not move the position"); + EXPECT_TRUE(pread(fd, a, sizeof(a), 0) == (ssize_t) sizeof(a) && + lseek(fd, 0, SEEK_CUR) == INT64_MAX, + "pread position"); + TEST("read past the blob is EOF, not an error"); + EXPECT_EQ(read(fd, b, sizeof(b)), 0, "read at INT64_MAX"); + close(fd); +} + +/* claimintf refuses ifnum >= 8 * sizeof(unsigned long) -- 64, not 32. Between + * the two, an interface number is merely absent, which is a question about the + * device rather than about the argument. + */ +static void check_interface_bound(void) +{ + printf("\ntest-usbdev-ioctl: the interface-number bound\n"); + int fd = open(NODE, O_RDWR); + if (fd < 0) { + TEST("open for the interface bound"); + FAIL("open"); + return; + } + uint32_t n63 = 63, n64 = 64; + TEST("CLAIMINTERFACE 64 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_CLAIMINTERFACE, &n64), -EINVAL, "claim 64"); + TEST("CLAIMINTERFACE 63 is not an argument error"); + EXPECT_EQ(io(fd, USBDEVFS_CLAIMINTERFACE, &n63), -ENODEV, "claim 63"); + TEST("RELEASEINTERFACE 64 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_RELEASEINTERFACE, &n64), -EINVAL, "release 64"); + TEST("RELEASEINTERFACE 63 is not an argument error"); + EXPECT_EQ(io(fd, USBDEVFS_RELEASEINTERFACE, &n63), -ENODEV, "release 63"); + + struct setinterface si = {.interface = 63, .altsetting = 0}; + TEST("SETINTERFACE 63 is not an argument error"); + EXPECT_EQ(io(fd, USBDEVFS_SETINTERFACE, &si), -ENODEV, "setinterface 63"); + si.interface = 64; + TEST("SETINTERFACE 64 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_SETINTERFACE, &si), -EINVAL, "setinterface 64"); + + /* bAlternateSetting is a byte, so 256 matches no altsetting of any device + * and usb_set_interface answers EINVAL (usb.c:391, message.c:1548). What + * the fixture can assert is the order: proc_setintf claims the interface + * before it validates the altsetting (devio.c:1533-1539), so the claim + * answers first and a check hoisted above the claim would show up here. The + * effect of the argument itself needs a device that can be claimed and is + * asserted against the board, where altsetting 256 used to be narrowed to + * setting 0 and took effect. + */ + si.interface = 0; + si.altsetting = 256; + TEST("SETINTERFACE reports the claim before the altsetting"); + EXPECT_EQ(io(fd, USBDEVFS_SETINTERFACE, &si), -ENODEV, + "setinterface alt 256"); + si.interface = 64; + TEST("an interface past the bound outranks the altsetting"); + EXPECT_EQ(io(fd, USBDEVFS_SETINTERFACE, &si), -EINVAL, + "setinterface 64/256"); + + struct disconnect_claim dc; + memset(&dc, 0, sizeof(dc)); + dc.interface = 64; + TEST("DISCONNECT_CLAIM 64 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_DISCONNECT_CLAIM, &dc), -EINVAL, "dc 64"); + + struct usbdevfs_ioctl ic = { + .ifno = 64, .ioctl_code = (int) USBDEVFS_DISCONNECT, .data = NULL}; + TEST("USBDEVFS_IOCTL ifno 64 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_IOCTL, &ic), -EINVAL, "usbdevfs_ioctl 64"); + close(fd); +} + +/* findintfep rejects a malformed endpoint address before it looks at anything, + * and do_proc_bulk resolves the endpoint before it looks at the length. Both + * orderings are observable, and both were the other way round: a length error + * outranked an absent endpoint, and two of the four entry points onto the + * endpoint lookup had no reserved-bit test of their own. + */ +static void check_endpoint_arguments(void) +{ + printf("\ntest-usbdev-ioctl: endpoint addresses and check order\n"); + int fd = open(NODE, O_RDWR); + if (fd < 0) { + TEST("open for the endpoint arguments"); + FAIL("open"); + return; + } + char scratch[64]; + struct bulktransfer bt = { + .ep = 0x05, .len = 32u * 1024 * 1024, .timeout = 10, .data = scratch}; + TEST("BULK on an absent ep outranks a too-large length"); + EXPECT_EQ(io(fd, USBDEVFS_BULK, &bt), -ENOENT, "bulk absent ep 32M"); + bt.len = 0x7fffffffu; + TEST("BULK on an absent ep outranks a malformed length"); + EXPECT_EQ(io(fd, USBDEVFS_BULK, &bt), -ENOENT, "bulk absent ep INT_MAX"); + bt.len = 8; + TEST("BULK on an absent ep is ENOENT"); + EXPECT_EQ(io(fd, USBDEVFS_BULK, &bt), -ENOENT, "bulk absent ep"); + bt.ep = 0x30; + TEST("BULK on a reserved-bit ep is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_BULK, &bt), -EINVAL, "bulk reserved ep"); + + uint32_t ep = 0x30; + TEST("CLEAR_HALT on a reserved-bit ep is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_CLEAR_HALT, &ep), -EINVAL, "clear_halt 0x30"); + TEST("RESETEP on a reserved-bit ep is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_RESETEP, &ep), -EINVAL, "resetep 0x30"); + ep = 0x05; + TEST("CLEAR_HALT on an absent ep is ENOENT"); + EXPECT_EQ(io(fd, USBDEVFS_CLEAR_HALT, &ep), -ENOENT, "clear_halt 0x05"); + + /* findintfep tests the caller's whole unsigned int, not its low byte + * (devio.c:860), and CLEAR_HALT, RESETEP and BULK all carry a 32-bit + * endpoint word. Narrowed to a byte before the test, 0x183 became 0x83 and + * 0x181 became the fixture device's own 0x81, so the lookup answered about + * an endpoint the caller had not named -- and on hardware the stall clear + * reached it. + */ + ep = 0x183; + TEST("CLEAR_HALT above the address byte is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_CLEAR_HALT, &ep), -EINVAL, "clear_halt 0x183"); + ep = 0x01000083; + TEST("RESETEP with high bits set is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_RESETEP, &ep), -EINVAL, "resetep 0x01000083"); + bt.ep = 0x181; + bt.len = 8; + TEST("BULK above the address byte is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_BULK, &bt), -EINVAL, "bulk 0x181"); + + struct ctrltransfer ct = {.bRequestType = 0x82, /* IN, standard, endpoint */ + .bRequest = 0, + .wValue = 0, + .wIndex = 0x05, + .wLength = 8192, + .timeout = 10, + .data = scratch}; + TEST("CONTROL recipient check outranks the wLength cap"); + EXPECT_EQ(io(fd, USBDEVFS_CONTROL, &ct), -ENOENT, "control absent ep"); + ct.wIndex = 0x30; + ct.wLength = 8; + TEST("CONTROL on a reserved-bit ep is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_CONTROL, &ct), -EINVAL, "control reserved ep"); + + /* The wider test must not spread to the control path: check_ctrlrecip masks + * wIndex to its low byte before findintfep ever sees it (devio.c:904), so + * 0x0181 still names endpoint 0x81 and still resolves to its interface, + * which without a device behind it is -ENODEV rather than -EINVAL. + */ + ct.bRequestType = 0x82; + ct.wIndex = 0x0181; + ct.wLength = 8; + TEST("CONTROL folds wIndex to its low byte"); + EXPECT_EQ(io(fd, USBDEVFS_CONTROL, &ct), -ENODEV, "control wIndex 0x181"); + + /* The cap itself still applies once the recipient is not the question. */ + ct.bRequestType = 0x80; /* IN, standard, device */ + ct.wIndex = 0; + ct.wLength = 8192; + TEST("CONTROL over 4096 bytes is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_CONTROL, &ct), -EINVAL, "control wLength"); + close(fd); +} + +/* The two orderings the positional and vector write paths owe Linux, both of + * them ahead of anything this descriptor has to say about itself: + * + * ksys_pwrite64 and do_pwritev test pos < 0 before they look the descriptor + * up (read_write.c), so a negative offset is EINVAL even on a descriptor that + * owes EBADF for having no write op. usbdev_pread already ordered the two + * this way and the write side did not. + * + * vfs_readv and vfs_writev test FMODE_READ/FMODE_WRITE, then FMODE_CAN_*, + * before an empty vector returns 0 (read_write.c), so direction still + * decides. An empty vector used to return 0 on this fd for every direction, + * and pwritev2(RWF_APPEND) had no dispatch at all: it reached the readiness + * pipe and answered for that instead. + */ +static void check_offset_and_vector_edges(void) +{ + printf("\ntest-usbdev-ioctl: negative offsets and empty vectors\n"); + int ro = open(NODE, O_RDONLY); + int rw = open(NODE, O_RDWR); + int wo = open(NODE, O_WRONLY); + if (ro < 0 || rw < 0 || wo < 0) { + TEST("three opens for the offset and vector edges"); + FAIL("open"); + goto out; + } + + char b[8] = {0}; + struct iovec iov = {b, sizeof(b)}; + + TEST("pwrite at a negative offset is EINVAL"); + EXPECT_ERRNO(pwrite(ro, b, 4, -1), EINVAL, "pwrite -1"); + TEST("pwritev at a negative offset is EINVAL"); + EXPECT_ERRNO(pwritev(ro, &iov, 1, -1), EINVAL, "pwritev -1"); + + TEST("readv of nothing on a write-only fd is EBADF"); + EXPECT_ERRNO(readv(wo, &iov, 0), EBADF, "readv 0"); + TEST("preadv of nothing on a write-only fd is EBADF"); + EXPECT_ERRNO(preadv(wo, &iov, 0, 0), EBADF, "preadv 0"); + TEST("readv of nothing on a readable fd is 0"); + EXPECT_EQ(readv(ro, &iov, 0), 0, "readv 0 ro"); + + TEST("pwritev2 of nothing on a read-only fd is EBADF"); + EXPECT_ERRNO(pwritev2_raw(ro, &iov, 0, 0, 0), EBADF, "pwritev2 0 ro"); + TEST("pwritev2 of nothing on a writable fd is EINVAL"); + EXPECT_ERRNO(pwritev2_raw(rw, &iov, 0, 0, 0), EINVAL, "pwritev2 0 rw"); + TEST("pwritev2(RWF_APPEND) on a writable fd is EINVAL"); + EXPECT_ERRNO(pwritev2_raw(rw, &iov, 1, -1, RWF_APPEND), EINVAL, + "pwritev2 append rw"); + TEST("pwritev2(RWF_APPEND) on a read-only fd is EBADF"); + EXPECT_ERRNO(pwritev2_raw(ro, &iov, 1, -1, RWF_APPEND), EBADF, + "pwritev2 append ro"); + +out: + if (ro >= 0) + close(ro); + if (rw >= 0) + close(rw); + if (wo >= 0) + close(wo); +} + +/* An endpoint whose owning interface number is one this layer cannot hold. + * + * bInterfaceNumber is a device-supplied byte with the whole 0..255 range behind + * it, and the endpoint lookup uses it to index a 64-entry array, so a device + * declaring 200 read 191 entries past the end -- device-controlled input, and + * unreachable from any attached device, which is why it needs a fixture that + * emits one. checkintf refuses exactly this number for exactly this reason + * (devio.c:842), so EINVAL is the answer both before and after the bound: what + * changes is that nothing reads out of bounds first, which is visible under + * -fsanitize=array-bounds and nowhere else. + */ +static void check_malformed_interface_number(void) +{ + printf("\ntest-usbdev-ioctl: an interface number the table cannot hold\n"); + int fd = open(BAD_IFNUM_NODE, O_RDWR); + if (fd < 0) { + TEST("open of the malformed-descriptor node"); + FAIL("open"); + return; + } + + /* Assert the fixture carries what this is about, so the lane cannot go + * quietly vacuous if the mode ever stops emitting it. Device descriptor 18 + * bytes, configuration header 9, then the interface descriptor. + */ + unsigned char blob[64]; + ssize_t n = read(fd, blob, sizeof(blob)); + TEST("the fixture declares bInterfaceNumber 200"); + EXPECT_TRUE(n >= 27 + 9 && blob[27 + 1] == 0x04 && blob[27 + 2] == 200, + "fixture interface number"); + TEST("the fixture carries that interface's endpoint"); + EXPECT_TRUE(n >= 36 + 7 && blob[36 + 1] == 0x05 && blob[36 + 2] == 0x81, + "fixture endpoint"); + + uint32_t ep = 0x81; + TEST("CLEAR_HALT on an ep owned by interface 200 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_CLEAR_HALT, &ep), -EINVAL, "clear_halt owner"); + TEST("RESETEP on an ep owned by interface 200 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_RESETEP, &ep), -EINVAL, "resetep owner"); + + char scratch[64]; + struct bulktransfer bt = { + .ep = 0x81, .len = 8, .timeout = 10, .data = scratch}; + TEST("BULK on an ep owned by interface 200 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_BULK, &bt), -EINVAL, "bulk owner"); + + struct ctrltransfer ct = {.bRequestType = 0x82, /* IN, standard, endpoint */ + .bRequest = 0, + .wValue = 0, + .wIndex = 0x81, + .wLength = 8, + .timeout = 10, + .data = scratch}; + TEST("CONTROL to an ep owned by interface 200 is EINVAL"); + EXPECT_EQ(io(fd, USBDEVFS_CONTROL, &ct), -EINVAL, "control owner"); + + /* An endpoint no interface carries is still merely absent. */ + ep = 0x82; + TEST("an ep no interface carries is still ENOENT"); + EXPECT_EQ(io(fd, USBDEVFS_CLEAR_HALT, &ep), -ENOENT, "clear_halt absent"); + close(fd); +} + +/* Each open-time failure keeps its own errno. ENODEV -- nothing answers to this + * address -- is the one that becomes the ENOENT open(2) owes for a name with + * nothing behind it; a resource failure reported as ENOENT told the guest a + * node was missing when the host had merely run out of something. + */ +static void check_open_failure_errno(const char *stage) +{ + printf("\ntest-usbdev-ioctl: a failed open reports what failed (%s)\n", + stage); + int want = strcmp(stage, "pipe") == 0 ? ENFILE : ENOMEM; + TEST("the failure keeps its own errno"); + EXPECT_ERRNO(open(NODE, O_RDWR), want, stage); +} + +/* How many fds the side table can hold at once, measured by filling it. */ +static int count_table_slots(void) +{ + enum { CAP = 64 }; + int fds[CAP]; + int n = 0; + for (; n < CAP; n++) { + fds[n] = open(NODE, O_RDWR); + if (fds[n] < 0) + break; + } + for (int i = 0; i < n; i++) + close(fds[i]); + return n; +} + +static int race_fd = -1; +static int race_sibling = -1; +static const char *race_reopen_node; + +static void *race_closer(void *unused) +{ + (void) unused; + usleep(4000); + close(race_fd); + if (race_reopen_node) + race_sibling = open(race_reopen_node, O_RDWR); + return NULL; +} + +/* idVendor from the descriptors blob the fd serves (device descriptor bytes + * 8..9, little endian). Which device an fd is bound to is the only thing that + * tells two live entries apart, so the races read it rather than counting. + */ +static int fd_vid(int fd) +{ + unsigned char d[18]; + if (fd < 0 || pread(fd, d, sizeof(d), 0) != (ssize_t) sizeof(d)) + return -1; + return d[8] | (d[9] << 8); +} + +/* A close that lands between fd_alloc publishing the guest fd and the side + * table binding it. The teardown hook matches on the fd number, which is still + * unbound inside that window, so the close found no entry, tore nothing down, + * and the slot, its descriptor blob and the write end of its pipe stayed held + * for the life of the process. Counted rather than observed directly: the table + * is fixed, so a leaked slot is one fewer simultaneous open. + * + * The window is a few instructions wide unaided, so this runs under + * ELFUSE_USBDEV_PUBLISH_DELAY_US. The fd number the closer aims at is the one + * the next open must take: the lowest free number, which is the one a probe + * open just gave back. + */ +static void check_publish_race(void) +{ + printf("\ntest-usbdev-ioctl: a close inside the publish window\n"); + int before = count_table_slots(); + int rounds = 0; + for (int i = 0; i < 4; i++) { + int probe = open(NODE, O_RDWR); + if (probe < 0) + break; + race_fd = probe; + close(probe); + pthread_t t; + if (pthread_create(&t, NULL, race_closer, NULL) != 0) + break; + int fd = open(NODE, O_RDWR); + pthread_join(t, NULL); + if (fd >= 0) + close(fd); + rounds++; + } + int after = count_table_slots(); + TEST("the table held slots to race for"); + EXPECT_TRUE(before > 1 && rounds == 4, "race preconditions"); + TEST("a close inside the publish window leaks no slot"); + EXPECT_EQ(after, before, "slots after the race"); + printf(" slots: %d before, %d after %d rounds\n", before, after, rounds); +} + +/* The same window, with the close followed by a reopen of the number it freed. + * + * The generation this open publishes has to be the one fd_alloc stamped for it. + * Re-derived from the fd table after the slot was already publishable, it was + * whatever the number carried by then -- and inside this window that is the + * reopening thread's stamp. Two entries then held the identical {guest_fd, + * generation}, so the guard that retires an unbound entry compared equal and + * retired nothing, and usbdev_acquire, which matches on that same pair, + * answered from whichever sat at the lower index. Both consequences are + * asserted: the fd reads the device it was actually opened on, and the entry + * that lost the number gives its slot back. + * + * The reopen deliberately names the other bus, because two entries describing + * one device are indistinguishable by anything the guest can read. + */ +static void check_publish_window_reopen(void) +{ + printf( + "\ntest-usbdev-ioctl: a close and reopen inside the publish window\n"); + int before = count_table_slots(); + int probe = open(NODE, O_RDWR); + if (probe < 0) { + TEST("probe open for the reopen race"); + FAIL("open"); + return; + } + race_fd = probe; + race_sibling = -1; + race_reopen_node = OTHER_NODE; + close(probe); + + pthread_t t; + if (pthread_create(&t, NULL, race_closer, NULL) != 0) { + TEST("closer thread for the reopen race"); + FAIL("pthread_create"); + return; + } + int fd = open(NODE, O_RDWR); + pthread_join(t, NULL); + race_reopen_node = NULL; + + int sib = race_sibling; + TEST("the reopen took the number the close freed"); + EXPECT_TRUE(fd >= 0 && sib == fd, "same fd number"); + + /* The reopener owns the number now, so the entry that answers to it must be + * the one it opened -- the other bus's device, not this one's. + */ + TEST("the fd answers for the device it was reopened on"); + EXPECT_EQ(fd_vid(sib), OTHER_VID, "idVendor"); + + if (sib >= 0) + close(sib); + if (fd >= 0 && fd != sib) + close(fd); + int after = count_table_slots(); + TEST("the entry that lost the number gives its slot back"); + EXPECT_EQ(after, before, "slots after the race"); + printf(" slots: %d before, %d after\n", before, after); +} + +/* The window on the other side of the bind: the entry is findable, so the close + * reaps it and usbdev_unref frees the slot, and a sibling open can take that + * same slot and bind its own live fd before this open's recheck runs. + * + * The recheck then sees a generation that has moved and retires -- and claiming + * the entry on "not dead" alone claimed the sibling's, marked it dead, freed + * its descriptor blob and closed its pipe. The sibling's fd was never closed + * and answered EBADF on every read and ioctl from then on. Nothing about the + * slot's contents can distinguish the two allocations; only the tuple the + * caller allocated can, which is what the retire path now claims on. + */ +static void check_retire_window_race(void) +{ + printf("\ntest-usbdev-ioctl: a close and reopen after the bind\n"); + int before = count_table_slots(); + int probe = open(NODE, O_RDWR); + if (probe < 0) { + TEST("probe open for the retire race"); + FAIL("open"); + return; + } + race_fd = probe; + race_sibling = -1; + race_reopen_node = NODE; + close(probe); + + pthread_t t; + if (pthread_create(&t, NULL, race_closer, NULL) != 0) { + TEST("closer thread for the retire race"); + FAIL("pthread_create"); + return; + } + int fd = open(NODE, O_RDWR); + pthread_join(t, NULL); + race_reopen_node = NULL; + + int sib = race_sibling; + TEST("the reopen took the number the close freed"); + EXPECT_TRUE(fd >= 0 && sib == fd, "same fd number"); + + /* Both assertions are about the sibling's fd, which was never closed. */ + unsigned char d[18]; + TEST("the reopened fd still reads its descriptors"); + EXPECT_TRUE(sib >= 0 && pread(sib, d, sizeof(d), 0) == (ssize_t) sizeof(d), + "pread on the live sibling"); + TEST("the reopened fd still answers its ioctls"); + EXPECT_EQ(io(sib, USBDEVFS_GET_SPEED, NULL), 2, "get_speed"); /* FULL */ + + if (sib >= 0) + close(sib); + if (fd >= 0 && fd != sib) + close(fd); + int after = count_table_slots(); + TEST("neither entry leaks its slot"); + EXPECT_EQ(after, before, "slots after the race"); + printf(" slots: %d before, %d after\n", before, after); +} + +static void check_answers_without_a_device(void) +{ + printf("\ntest-usbdev-ioctl: what is answered from the model\n"); + int fd = open(NODE, O_RDWR); + if (fd < 0) { + TEST("open for the model-served ioctls"); + FAIL("open"); + return; + } + + /* Every capability bit names part of the URB machinery this stage answers + * ENOTTY for, so the word is 0 until that machinery lands. + */ + uint32_t caps = 0xffffffffu; + TEST("GET_CAPABILITIES reports no URB capabilities"); + EXPECT_TRUE(io(fd, USBDEVFS_GET_CAPABILITIES, &caps) == 0 && caps == 0, + "caps"); + + TEST("GET_SPEED returns the enum as its value"); + EXPECT_EQ(io(fd, USBDEVFS_GET_SPEED, NULL), 2, "get_speed"); /* FULL */ + + struct { + uint32_t devnum; + uint32_t slow; + } ci = {0, 0}; + TEST("CONNECTINFO reports the devnum"); + EXPECT_TRUE(io(fd, USBDEVFS_CONNECTINFO, &ci) == 0 && ci.devnum == 1 && + ci.slow == 0, + "connectinfo"); + + TEST("an unknown ioctl is ENOTTY"); + EXPECT_EQ(io(fd, 0x00005563u /* _IO('U', 99) */, NULL), -ENOTTY, + "unknown ioctl"); + TEST("SUBMITURB is ENOTTY at this stage"); + EXPECT_EQ(io(fd, USBDEVFS_SUBMITURB, NULL), -ENOTTY, "submiturb"); + TEST("DISCARDURB is ENOTTY at this stage"); + EXPECT_EQ(io(fd, USBDEVFS_DISCARDURB, NULL), -ENOTTY, "discardurb"); + + /* Everything that has to reach the wire says so, with the errno Linux uses + * for a device that is not there. + */ + struct getdriver gd; + memset(&gd, 0, sizeof(gd)); + TEST("GETDRIVER without a device is ENODEV"); + EXPECT_EQ(io(fd, USBDEVFS_GETDRIVER, &gd), -ENODEV, "getdriver"); + + uint32_t cfg = 1; + TEST("SETCONFIGURATION without a device is ENODEV"); + EXPECT_EQ(io(fd, USBDEVFS_SETCONFIGURATION, &cfg), -ENODEV, "setconfig"); + + struct disconnect_claim dc; + memset(&dc, 0, sizeof(dc)); + TEST("DISCONNECT_CLAIM without a device is ENODEV"); + EXPECT_EQ(io(fd, USBDEVFS_DISCONNECT_CLAIM, &dc), -ENODEV, "dc"); + + struct usbdevfs_ioctl ic = { + .ifno = 0, .ioctl_code = (int) USBDEVFS_CONNECT, .data = NULL}; + TEST("USBDEVFS_IOCTL CONNECT without a device is ENODEV"); + EXPECT_EQ(io(fd, USBDEVFS_IOCTL, &ic), -ENODEV, "connect"); + close(fd); +} + +/* FIONBIO and FIOASYNC never reach a file's own ioctl handler on Linux: + * do_vfs_ioctl answers both for every file before it calls f_op->unlocked_ioctl + * (fs/ioctl.c:818-822), so they also never meet usbdevfs's FMODE_WRITE gate. + * Sent into it here they came back EPERM on a read-only fd and ENOTTY on a + * writable one, while fcntl(F_SETFL) on the same descriptor set O_NONBLOCK and + * F_GETFL reported it: two entry points onto one flag, disagreeing about it. + * + * Measured on Linux (gcc:14, a char device and a plain file, access modes 0, 1, + * 2 and 3): FIONBIO(1) is 0 and sets O_NONBLOCK in every one of them; + * FIOASYNC(1) is ENOTTY and FIOASYNC(0) is 0, because ioctl_fioasync only + * consults f_op->fasync when the request would change the FASYNC state and + * usbdev_file_operations declares none (devio.c:2846-2856). + */ +static void check_vfs_ioctls(void) +{ + printf("\ntest-usbdev-ioctl: the two ioctls the vfs answers first\n"); + const int modes[2] = {O_RDONLY, O_RDWR}; + const char *names[2] = {"read-only", "writable"}; + for (int i = 0; i < 2; i++) { + int fd = open(NODE, modes[i]); + if (fd < 0) { + TEST("open for the vfs ioctls"); + FAIL("open"); + return; + } + int one = 1, zero = 0; + char t[64]; + + /* The access mode must not be consulted: the kernel has already + * answered by the time the file's own gate would run. + */ + snprintf(t, sizeof(t), "FIONBIO on a %s fd is 0", names[i]); + TEST(t); + EXPECT_EQ(io(fd, FIONBIO, &one), 0, "fionbio"); + snprintf(t, sizeof(t), "and O_NONBLOCK is set on the %s fd", names[i]); + TEST(t); + EXPECT_TRUE((fcntl(fd, F_GETFL) & O_NONBLOCK) != 0, "F_GETFL"); + + /* The other entry point onto the same flag has to agree with it. */ + snprintf(t, sizeof(t), "FIONBIO(0) clears it on the %s fd", names[i]); + TEST(t); + EXPECT_TRUE(io(fd, FIONBIO, &zero) == 0 && + (fcntl(fd, F_GETFL) & O_NONBLOCK) == 0, + "fionbio 0"); + snprintf(t, sizeof(t), "F_SETFL then agrees on the %s fd", names[i]); + TEST(t); + EXPECT_TRUE(fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK) == 0 && + (fcntl(fd, F_GETFL) & O_NONBLOCK) != 0, + "F_SETFL"); + + /* No .fasync, so arming is refused and asking for the state it already + * has is granted. It must not reach the O_ASYNC arm in sys_ioctl, which + * would arm a SIGIO watcher the kernel refuses to arm. + */ + snprintf(t, sizeof(t), "FIOASYNC(1) on a %s fd is ENOTTY", names[i]); + TEST(t); + EXPECT_EQ(io(fd, FIOASYNC, &one), -ENOTTY, "fioasync 1"); + snprintf(t, sizeof(t), "FIOASYNC(0) on a %s fd is 0", names[i]); + TEST(t); + EXPECT_EQ(io(fd, FIOASYNC, &zero), 0, "fioasync 0"); + close(fd); + } + + /* get_user runs first in both kernel helpers, so a bad argument pointer + * outranks every other answer, the access mode included. + */ + int fd = open(NODE, O_RDONLY); + if (fd >= 0) { + TEST("FIONBIO with a bad argument is EFAULT"); + EXPECT_EQ(io(fd, FIONBIO, (void *) 8), -EFAULT, "fionbio efault"); + TEST("FIOASYNC with a bad argument is EFAULT"); + EXPECT_EQ(io(fd, FIOASYNC, (void *) 8), -EFAULT, "fioasync efault"); + close(fd); + } +} + +/* Every gate on this descriptor derives its capability bits the way OPEN_FMODE + * does, or none of them do. + * + * OPEN_FMODE (fs.h:3631) is (flags + 1) & O_ACCMODE: access modes 0, 1 and 2 + * give FMODE_READ, FMODE_WRITE and both, and access mode 3 gives neither. Mode + * 3 is reachable -- open(2) takes it and ACC_MODE(3) asks this 0666 node for + * read plus write, which it grants -- so Linux hands back a descriptor that can + * do nothing. Measured on Linux for a char device and a plain file: the open + * succeeds and read, pread and write are all EBADF. + * + * Each gate here tested the access mode against O_RDONLY or O_WRONLY on its + * own, so all four agreed on modes 0, 1 and 2 and all four were wrong on mode + * 3: the fd read the descriptors blob and was handed the whole ioctl set. + */ +static void check_access_mode_three(void) +{ + printf("\ntest-usbdev-ioctl: an access mode with neither capability\n"); + int fd = open(NODE, 3); + TEST("open with access mode 3 succeeds"); + EXPECT_TRUE(fd >= 0, "open"); + if (fd < 0) + return; + + char b[8] = {0}; + struct iovec empty = {b, 0}; + TEST("read on it is EBADF"); + EXPECT_ERRNO(read(fd, b, 4), EBADF, "read"); + TEST("pread on it is EBADF"); + EXPECT_ERRNO(pread(fd, b, 4, 0), EBADF, "pread"); + TEST("an empty readv on it is EBADF"); + EXPECT_ERRNO(readv(fd, &empty, 1), EBADF, "readv"); + TEST("write on it is EBADF, not EINVAL"); + EXPECT_ERRNO(write(fd, b, 4), EBADF, "write"); + TEST("pwrite on it is EBADF, not EINVAL"); + EXPECT_ERRNO(pwrite(fd, b, 4, 0), EBADF, "pwrite"); + TEST("an empty writev on it is EBADF"); + EXPECT_ERRNO(writev(fd, &empty, 1), EBADF, "writev"); + + struct { + uint32_t devnum; + uint32_t slow; + } ci = {0, 0}; + TEST("an ioctl on it is EPERM"); + EXPECT_EQ(io(fd, USBDEVFS_CONNECTINFO, &ci), -EPERM, "connectinfo"); + + /* The two the vfs answers before the gate are unaffected by it. */ + int one = 1; + TEST("FIONBIO on it is still 0"); + EXPECT_EQ(io(fd, FIONBIO, &one), 0, "fionbio"); + close(fd); +} + +/* usbfs_memory_mb is one allowance for every transfer in flight, not a per-call + * size cap. + * + * usbfs_increase_memory_usage charges len + sizeof(struct urb) against a + * module-global total and usbfs_decrease_memory_usage gives it back when the + * transfer settles (devio.c:145-178, charged at devio.c:1308). Read as a + * per-call ceiling it got both halves wrong: a single request of exactly the + * allowance was accepted, where Linux always refuses it because the URB itself + * is charged too, and nothing accumulated across calls or across fds at all -- + * measured before the counter, 32 threads each asking for 16 MB on its own fd: + * 32 accepted, none refused, and 32 refused and none accepted after. + * + * The three assertions below pin what one thread can decide on its own: the + * boundary a per-call cap gets wrong, and the two exits that owe the allowance + * back. Whether the sum across concurrent fds is bounded is the same counter + * seen from outside, and it needs overlapping transfers to observe, so it stays + * a measurement rather than a lane assertion. + * + * The device this needs is one whose interface can actually be claimed, which + * the service-less fixture has none of: the claim answers ENODEV and the length + * checks are never reached. So this runs where there is one -- the loopback + * model, and hardware -- and says so when there is not. + */ +#define ALLOWANCE (16u * 1024 * 1024) +#define LOOPBACK_NODE "/dev/bus/usb/003/001" +#define LOOPBACK_IFNUM 2 +#define LOOPBACK_EP_OUT 0x02 + +static long bulk_of(int fd, unsigned int len, void *data) +{ + struct bulktransfer bt = { + .ep = LOOPBACK_EP_OUT, .len = len, .timeout = 20000, .data = data}; + return io(fd, USBDEVFS_BULK, &bt); +} + +static void check_transfer_allowance(void) +{ + printf("\ntest-usbdev-ioctl: the in-flight transfer allowance\n"); + int fd = open(LOOPBACK_NODE, O_RDWR); + if (fd < 0) { + printf( + " no claimable device in this model, so the allowance is not\n" + " reachable here: the claim answers ENODEV before any length is\n" + " looked at. Covered by the loopback lane and on hardware.\n"); + return; + } + unsigned int ifn = LOOPBACK_IFNUM; + if (io(fd, USBDEVFS_CLAIMINTERFACE, &ifn) != 0) { + TEST("claim for the allowance checks"); + FAIL("claiminterface"); + close(fd); + return; + } + void *big = malloc(ALLOWANCE); + if (!big) { + TEST("buffer for the allowance checks"); + FAIL("malloc"); + close(fd); + return; + } + memset(big, 0xa5, ALLOWANCE); + + /* The URB is charged alongside the buffer, so the whole allowance never + * fits. A per-call size cap accepted this. + */ + TEST("a transfer of the whole allowance is ENOMEM"); + EXPECT_EQ(bulk_of(fd, ALLOWANCE, big), -ENOMEM, "len == allowance"); + + /* Just under it must still go through, twice: the second is the one that + * proves the first gave its charge back. + */ + unsigned int under = ALLOWANCE - 4096; + TEST("a transfer just under it is accepted"); + EXPECT_EQ(bulk_of(fd, under, big), (long) under, "len just under"); + TEST("and again, so the allowance came back"); + EXPECT_EQ(bulk_of(fd, under, big), (long) under, "second transfer"); + + /* The error arms owe it back too. A bad buffer fails after the charge is + * taken, so a leak here is invisible until the next large transfer. + */ + TEST("a faulting transfer is EFAULT"); + EXPECT_EQ(bulk_of(fd, under, (void *) 8), -EFAULT, "bad buffer"); + TEST("and the allowance came back from that too"); + EXPECT_EQ(bulk_of(fd, under, big), (long) under, "after the fault"); + + free(big); + close(fd); +} + +/* The side table is fixed and Linux's is not, so exhaustion is a deviation. + * Assert the two things that are still contract: the limit reports a + * kernel-side shortfall rather than the guest's own fd limit, and a closed fd + * gives its slot back. + */ +static void check_table_exhaustion(void) +{ + printf("\ntest-usbdev-ioctl: the side table's own limit\n"); + enum { CAP = 64 }; + int fds[CAP]; + int n = 0, err = 0; + for (; n < CAP; n++) { + fds[n] = open(NODE, O_RDWR); + if (fds[n] < 0) { + err = errno; + break; + } + } + TEST("simultaneous opens hit a bounded limit"); + EXPECT_TRUE(n > 0 && n < CAP, "table limit reached"); + TEST("exhaustion is ENOMEM, not the guest's EMFILE"); + EXPECT_EQ(err, ENOMEM, "exhaustion errno"); + + int reclaimed = -1; + if (n > 0) { + close(fds[0]); + reclaimed = open(NODE, O_RDWR); + fds[0] = reclaimed; + } + TEST("a closed fd gives its table slot back"); + EXPECT_TRUE(reclaimed >= 0, "slot reclaimed"); + for (int i = 0; i < n; i++) + if (fds[i] >= 0) + close(fds[i]); + printf(" XFAIL open-limit: Linux unbounded, elfuse %d simultaneous\n", n); +} + +/* The teardown hook is handed a bare fd number, and by the time it runs the + * fd-table slot is already free, so a sibling thread's open can be holding the + * same number with a second, live entry. Matching on the number alone tore down + * whichever entry sat lower in the table, about half the time the live one, and + * the thread that had just opened it saw EBADF. + */ +#define CHURN_THREADS 4 +#define CHURN_ROUNDS 2000 + +static int churn_bad; + +static void *churn(void *unused) +{ + (void) unused; + unsigned char buf[18]; + for (int i = 0; i < CHURN_ROUNDS; i++) { + int fd = open(NODE, O_RDWR); + if (fd < 0) + continue; /* table full for a moment; not what is under test */ + if (read(fd, buf, sizeof(buf)) < 0) + __atomic_fetch_add(&churn_bad, 1, __ATOMIC_RELAXED); + close(fd); + } + return NULL; +} + +static void check_close_identity(void) +{ + printf("\ntest-usbdev-ioctl: close tears down the entry it named\n"); + pthread_t t[CHURN_THREADS]; + int started = 0; + for (int i = 0; i < CHURN_THREADS; i++) + if (pthread_create(&t[i], NULL, churn, NULL) == 0) + started++; + for (int i = 0; i < started; i++) + pthread_join(t[i], NULL); + TEST("a freshly opened fd is never torn down"); + EXPECT_TRUE(started == CHURN_THREADS && churn_bad == 0, "churn EBADF"); + printf(" churn: %d threads x %d rounds, %d fds lost\n", started, + CHURN_ROUNDS, churn_bad); +} + +/* Deviations this stage keeps deliberately: printed with both values so the gap + * is in the lane's output rather than only in the commit message. + */ +static void print_known_gaps(void) +{ + printf("\ntest-usbdev-ioctl: recorded gaps\n"); + int fd = open(NODE, O_RDWR); + if (fd >= 0) { + long r = io(fd, USBDEVFS_RESET, NULL); + printf( + " XFAIL reset: Linux re-enumerates the port, elfuse clears " + "claimed pipes' stalls and returns %ld\n", + r); + long speed = io(fd, USBDEVFS_GET_SPEED, NULL); + printf( + " XFAIL disconnect-gate: Linux answers ENODEV for every ioctl " + "once the device is gone, elfuse still serves GET_SPEED, " + "CONNECTINFO, GET_CAPABILITIES and read() from the open-time " + "model (GET_SPEED here: %ld)\n", + speed); + close(fd); + } + printf( + " XFAIL driver-name: Linux GETDRIVER reports the driver's name " + "(cdc_acm), elfuse reports the IOKit class (AppleUSBACMControl), and " + "DISCONNECT_CLAIM's name filters compare against it\n"); + printf( + " XFAIL short-bulk-out: Linux reports the byte count actually sent, " + "elfuse has no length from WritePipeTO and reports EIO\n"); +} + +int main(void) +{ + printf("test-usbdev-ioctl: the usbdevfs fd contract without hardware\n\n"); + + /* Every expectation below is written against the model ELFUSE_USB_FIXTURE + * supplies, which is what makes the answers the same on any host. Run + * without it the layer enumerates whatever is plugged in, the node is a + * real device rather than the modeled one, and the checks that ask the + * model a question disagree for a reason that has nothing to do with what + * they test. Say the precondition once rather than eight times in the voice + * of a defect. + */ + const char *mode = getenv("ELFUSE_USB_FIXTURE"); + if (!mode) { + printf( + " this lane is written against ELFUSE_USB_FIXTURE=1, which\n" + " make check sets. Without it the answers come from whatever\n" + " is attached. Nothing below was run.\n"); + return 1; + } + + /* Five runs of this binary force one failure or one window each, and each + * asserts only what its knob is about: everything else below describes a + * healthy open on the default model and would be measuring the knob. + */ + const char *fault = getenv("ELFUSE_USBDEV_OPEN_FAULT"); + if (fault) { + check_open_failure_errno(fault); + SUMMARY("test-usbdev-ioctl"); + return fails > 0 ? 1 : 0; + } + if (getenv("ELFUSE_USBDEV_PUBLISH_DELAY_US")) { + check_publish_race(); + check_publish_window_reopen(); + SUMMARY("test-usbdev-ioctl"); + return fails > 0 ? 1 : 0; + } + if (getenv("ELFUSE_USBDEV_RETIRE_DELAY_US")) { + check_retire_window_race(); + SUMMARY("test-usbdev-ioctl"); + return fails > 0 ? 1 : 0; + } + if (!strcmp(mode, "badifnum")) { + check_malformed_interface_number(); + SUMMARY("test-usbdev-ioctl"); + return fails > 0 ? 1 : 0; + } + + check_open_agrees(); + check_write_family(); + check_seek(); + check_interface_bound(); + check_endpoint_arguments(); + check_offset_and_vector_edges(); + check_answers_without_a_device(); + check_vfs_ioctls(); + check_access_mode_three(); + check_transfer_allowance(); + check_table_exhaustion(); + check_close_identity(); + print_known_gaps(); + + SUMMARY("test-usbdev-ioctl"); + return fails > 0 ? 1 : 0; +}