feat(storage): support Jindo for OSS scan planning

[XiaoHongbo-Hope]

Purpose

Add an opt-in, read-only JindoSDK backend for OSS scans. OpenDAL remains the default. PyPaimon can select it for native planning through apache/paimon#9124.

Changes

• add the optional storage-jindo feature, selected with fs.oss.impl=jindo
• dynamically load an installed JindoSDK without bundling its binary
• preserve HTTP/HTTPS endpoint protocol when adapting Jindo configuration
• support stat, paginated list, and ranged/full reads
• fetch list pages on demand and convert Jindo list exceptions into Rust errors
• initialize lazily and run blocking SDK calls on Tokio blocking workers
• reject unsupported write operations explicitly

Benchmark

Release-mode, partition-filtered scan planning on two fixed OSS-backed snapshots. This reads Paimon metadata and builds splits; it does not read data-file contents. Both backends ran on the same host with identical predicates, in reversed order and two independent processes per backend.

Workload	Resulting plan	OpenDAL planning time (warm median)	Jindo planning time (warm median)
A	~281k data-file references	1.55s	1.25s
B	~37k data-file references / ~27k splits	5.92s	2.19s

The file counts are references in the resulting scan plan, not individual data files opened or read. Both backends produced matching snapshot, split, file-count, and planned-byte results. Request count and peak OSS QPS were not measured.

Data read benchmark

Release-mode reads on two fixed OSS-backed snapshots. Planning completed before the timer started. Each backend ran in three independent processes on the same host, in alternating order.

Workload	Read result	OpenDAL data-read time (warm median)	Jindo data-read time (warm median)
A	indexed descriptor point read: 1 row / 313 B Arrow output	1.09s	1.26s
B	two-column read from one data file: 2,242,216 rows / 44.9 MiB Arrow output	0.49s	0.59s

Both backends produced matching row counts, Arrow sizes, and content checksums. Planning time is excluded. Network bytes and request count were not measured, so these results show end-to-end reader latency for these workloads, not raw OSS throughput.

Validation

• 6 Jindo storage tests and Python SDK discovery test
• default OSS and storage-jindo builds
• Clippy, formatting, dependency report, and diff checks
• read-only OSS REST-catalog test covering snapshot listing, planning, and row reading
• JindoSDK 6.10.4 local OSS-protocol probes: HTTPS emits TLS, list exceptions return errors, three-page listing is complete, and consuming one entry does not request later pages

Notes

No JindoSDK binary is packaged. Selecting Jindo without the feature or a loadable SDK returns a configuration error. A FileIO configured with Jindo rejects writes, deletes, renames, and copies.

[JingsongLi]

Have you compared Jindo and OpenDAL? What are the differences? (benchmark)

[XiaoHongbo-Hope]

Have you compared Jindo and OpenDAL? What are the differences? (benchmark)

yes, I did a test, Jindo performance is a little better, but there is a lot of qps issue during my testing, so may need another testing. And, I think this PR is best to have, not very urgent now. Because qps issue is addressed in other way.

[XiaoHongbo-Hope]

Have you compared Jindo and OpenDAL? What are the differences? (benchmark)

I updated the performance result in PR descrption.

[QuakeWang]

LGTM

[JingsongLi]

Requirement fit: SUPPORTED. Implementation: FINDINGS, reviewed at 45e887beb14cd1f67b1ae3ca5bfb10b26fdb21bc.

The opt-in planning backend has a concrete consumer in the merged apache/paimon#9124, and the supplied measurements demonstrate an end-to-end planning benefit for the two tested snapshots. Keeping the claim scoped to planning is appropriate: the reported data reads are slightly slower and request/QPS effects remain unmeasured.

[P2] Preserve explicit SDK environment selection before Python auto-discovery. In bindings/python/src/context.rs:122-124, a user who sets JINDOSDK_LIBRARY_PATH (or JINDOSDK_HOME) and has pyjindo installed gets fs.jindo.library.path injected from the Python package. library_candidates then treats that injected value as an exclusive explicit path and never consults the user's environment. Thus the documented SDK selection works for Rust callers but selects a different SDK through Python; if the package's library is not loadable, even a valid environment-selected SDK cannot be used. Please skip package discovery when either explicit SDK environment setting is present, or make package discovery a fallback after those settings, and cover this precedence in a binding test.

I traced configuration, lazy initialization, paginated listing, ranged reads, read-only operations, and the FileIO callers. cargo +1.94.0 test -p paimon --features storage-jindo --lib storage_jindo passes all 6 tests locally. I did not rerun the native SDK/OSS probes or the reported benchmarks; these unit tests do not validate the dynamic ABI or live storage behavior.