Read Linux file attributes through the FFM API

[vogella]

Bulk directory listing needs libfastlinuxfile, which only the x86_64 fragment ships, so aarch64, ppc64le and loongarch64 fall back to one stat per child through libunixfile. This binds opendir, readdir, statx, readlinkat and chmod through the Foreign Function & Memory API instead. struct statx has a kernel defined layout that is identical on every architecture, so one implementation covers all of them with no compiled artifact, and the native fragments lose their reason to exist. The buffers are held per thread and reused, so a listing allocates nothing per entry beyond the resulting FileInfo.

The bindings are generated by jextract using the script added with this change, as on Windows in #2908. Only statx is bound by hand, and only its MethodHandle: the descriptor and the symbol come from the generated code, with Linker.Option.captureCallState added, because errno is what tells a file that is not there from one that cannot be read and jextract does not emit that option. Note for anyone regenerating: --library c must be omitted, since it emits libraryLookup("libc.so"), which cannot be opened on glibc where the library is libc.so.6.

Listing 270027 entries in 45409 directories takes 439 to 464 ms against 537 to 563 ms for the JNI handler, with identical results for every entry: names, timestamps, lengths, permissions, symbolic links and link targets. -Declipse.filesystem.useFfm=false falls back to the native libraries.

One behaviour differs on purpose: a symbolic link whose target cannot be stated, such as a self referencing one, keeps its symlink attribute and link target in a directory listing. The JNI listing dropped both because its conversion returns early on an error, while its single file path already reported them, and that attribute is what the recursive link detection in UnifiedTree reads.

Moving this into PosixHandler and leaving fetchFileInfo/putFileInfo to Java NIO, as suggested in review, is left for a follow-up so that it can be sequenced with #2925.

[github-actions]

Test Results

    54 files  ±0      54 suites  ±0   57m 58s ⏱️ -28s
 4 810 tests ±0   4 788 ✅ ±0   22 💤 ±0  0 ❌ ±0 
12 321 runs  ±0  12 168 ✅ ±0  153 💤 ±0  0 ❌ ±0 

Results for commit c9c2df0. ± Comparison against base commit 31b3f6a.

♻️ This comment has been updated with latest results.

[HannesWell]

On the long run it would probably make most sense to use FFM under Linux/POSIX only to implement the NativeHandler methods listDirectoryNames() and listDirectoryAndGetFileInfos() in the existing PosixHandler. All other methods can probably be implemented using Java NIO with comparable performance.

The existing native implementations to fetch/put file information are intended to be removed via

• Replace native UnixFileHandler by Java NIO to fetch/put file-info #2925

If you complete a FFM based implementation of PosixHandler.listDirectoryNames() and PosixHandler.listDirectoryAndGetFileInfos(), all native code based implementations could be removed in #2925 and the performance of the 'final' state of PosixHandler could be compared with the existing native implementation.

The bindings are hand-written rather than generated, unlike #2908. I ran jextract over these headers to check: it produces 2743 lines against 246, and 174 errors under this bundle's compiler settings (154 non-externalized string literals, 20 unused imports), which is why #2908 downgrades nonExternalizedStringLiteral and unusedImport from error to warning. It also emits no captureCallState, so every call here would still need a hand-written downcall handle to read errno, which is what tells a missing file from a real I/O error. The generated layouts do agree exactly with the hand-written offsets, which is a useful independent check on them. jextract earns its keep on WIN32_FIND_DATAW; for four statx fields and one dirent field it does not.

Since #2908 is now merged, these errors are now warnings already. Furthermore it's correct that the generated code is not the most compact one. But I think deterministically generated code is better to maintain on the long run, than AI generated one.
And I hope that we develop some additional tooling on top of jextract when generating bindings for larger code bases like SWT that helps to remove unused code from the generated bindings and to avoid warnings like missing non-externalized string markers. To avoid the latter, we could even try to improve Eclipse itself to support individuals JDT settings per source folder.

[vogella]

Thanks, I have switched the bindings to jextract.

One exception: statx is bound by hand from the generated statx$address() and statx$descriptor(), with Linker.Option.captureCallState("errno") added, since jextract does not emit that option.

Without errno every failure looks the same and RefreshLocalVisitor deletes workspace resources whose statx failed with EACCES or EIO. The JNI handler reads errno today, so dropping the distinction would be a regression rather than a new limitation. That is similar to your adjustment in Lines 113-137 of the merged Win32Handler.

Since #2925 would change native code, I would rather not have both of us editing PosixHandler and LocalFileNativesManager at the same time: I can follow-up once #2925 is in.

[vogella]

@HannesWell how did you handle the additional warnings from the jextract generated code in your windows implementation? Just reset the quality gate? (How do I do this?)