Skip to content

[C++][Dataset] Filtering dataset that has column with StringView type gives empty table as result #39637

Closed

Description

@jorisvandenbossche

Describe the bug, including details regarding any error messages, version, and platform.

Not a C++ reproducer, but with very preliminary (non-merged) Python bindings to illustrate it:

import pyarrow as pa
import pyarrow.dataset as ds

# create IPC file with StringView column
builder = pa.lib.StringViewBuilder()
builder.append("test")
builder.append("some long string that is not inlined")
builder.append(None)
arr = builder.finish()

table = pa.table({"a": [1, 2, 3], "b": arr})
ds.write_dataset(table, "test_string_view", format="ipc")

# read back as a dataset
dataset = ds.dataset("test_string_view/", format="ipc")

The dataset looks good, and can be read as a whole:

>>> dataset.schema
a: int64
b: string_view

>>> dataset.to_table()
pyarrow.Table
a: int64
b: string_view
----
a: [[1,2,3]]
b: [["test","some long string that is not inlined",null]]

But when reading it with a filter on the other integer column, it silently gives an empty table (I would have expected it to raise an error because filter/take is not yet implemented for string_view):

>>> dataset.to_table(filter=ds.field('a') > 1)
pyarrow.Table
a: int64
b: string_view
----
a: [[]]
b: [[]]

This doesn't seem to happen if the string array only has inlined strings (then I get the expected error).

Component(s)

C++

Activity

  1. added
    Critical FixBugfixes for security vulnerabilities, crashes, or invalid data.
    on Jan 16, 2024
  2. thisisnic commented on Nov 10, 2026

    @thisisnic
    Member

    I'm just going through older bug issues to see if we can close them out or we need further action.

    Could not reproduce on 26.0.1: the filter now raises ArrowNotImplementedError: Function 'array_filter' has no kernel matching input types (string_view, bool) instead of returning an empty table. The change is in 24.0.0 and later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions