Skip to content

Blocking DataFusion callbacks should preserve the active Tokio runtime #872

Description

@shyjsarah

Problem

block_on_with_runtime detects when it is called from an entered Tokio runtime, but currently runs the future on the process fallback runtime in a new OS thread and synchronously joins that thread.

Moving the future to a different runtime can stall work that depends on the active runtime. It also differs from the behavior of runtime() and await_with_runtime, which preserve an entered runtime.

Expected behavior

When called from a Tokio multi-thread runtime, the blocking helper should execute the future on that runtime while marking the blocking section so Tokio can continue scheduling work. Calls made without an entered runtime should continue to use the process fallback runtime. The current-thread runtime needs to retain a compatible fallback because block_in_place is not supported there.

Reproduction

A unit test can enter a multi-thread runtime, call block_on_with_runtime with a future that returns Handle::current().id(), and compare that value with the caller's runtime ID. The current implementation returns a different ID.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions