[Niltra]

Rationale for this change

HivePartitioning::FormatValues always URI-encoded partition values, even when the partitioning was created with segment_encoding="none". The option therefore only affected parsing: writing My Fancy Product still produced part=My%20Fancy%20Product, as reported in GH-34905 and GH-41618.

Closes #34905.

What changes are included in this PR?

• HivePartitioning::FormatValues now honours segment_encoding: Uri keeps the current behaviour, None writes values verbatim.
• C++ tests for formatting and for writing a dataset with spaces and % in the partition value.
• Python test for ds.write_dataset with both encodings, including read-back.
• Clarified the segment_encoding documentation in the C++ header and the Python docstring.

The default remains "uri", so existing output is unchanged.

Are these changes tested?

• arrow-dataset-partition-test: 38/38 pass, including HivePartitioningFormat and the new WriteHiveWithoutUriEncoding.
• test_write_dataset_hive_segment_encoding: 2/2 pass against a local pyarrow built from this branch and the patched Arrow C++. Against pyarrow 26.0.0 the "none" case fails as expected, which reproduces the issue.
• clang-format 18.1.8, cpplint 1.6.1 and flake8 pass on the touched files.

Are there any user-facing changes?

Yes, a bug fix. This now works, without changing the default:

import pyarrow.dataset as ds
import pyarrow.parquet as pq

partitioning = ds.HivePartitioning(
    table.select(partition_cols).schema,
    segment_encoding="none",
)
pq.write_to_dataset(table, path, partitioning=partitioning, filesystem=fs)

part=My Fancy Product is written as-is; the default "uri" still writes part=My%20Fancy%20Product. With "none" the caller is responsible for using values that are valid path segments.

Was AI used for this PR?

PR code and description written by:

• AI

Reviewed before submission by:

• Human
• AI
• Not reviewed

This PR was human-directed and AI-assisted. The author selected the issue, set the requirements and quality bar (minimal change, no regressions, pass Arrow's checks), discussed the approach with the assistant, reviewed the test evidence, and authorised submission. The implementation, tests and description were produced with AI assistance and verified locally as described above. The change is limited to the existing segment_encoding option.

• GitHub Issue: [Python] unexpected URL encoded path (white spaces) when uploading to S3 #34905

[adamreeve]

Hi @Niltra, thanks for working on this. I have a PR open for the opposite scenario in directory and filename partitioning, which currently do no segment encoding when writing: #55423

As part of my PR, I've also made some changes to the no-encoding path so that it raises errors on invalid path segments.

I think we should wait until my PR is merged, and then re-use the validation here for Hive partitioning.