zenoh-flat transition (integration branch)

.github/workflows/ci.yml

@@ -22,7 +22,34 @@ jobs:
         os: [ubuntu-latest, macos-latest]
 
     steps:
-      - uses: actions/checkout@v4
+      - name: Check out zenoh-java
+        uses: actions/checkout@v4
+        with:
+          path: zenoh-java
+
+      - name: Check out zenoh-flat-jni
+        uses: actions/checkout@v4
+        with:
+          repository: ZettaScaleLabs/zenoh-flat-jni
+          ref: 498ba268cf314b38bdca3faa131c2ee8402b2ac4
+          path: zenoh-flat-jni
+
+      - name: Check out zenoh-flat
+        uses: actions/checkout@v4
+        with:
+          repository: ZettaScaleLabs/zenoh-flat
+          ref: 3f431b6b581c49b6c4d0e94fab5922f860a98205
+          path: zenoh-flat
+
+      - name: Use prebindgen from GitHub main
+        shell: bash
+        working-directory: zenoh-flat-jni
+        run: |
+          sed -i.bak \
+            -e 's|prebindgen = { path = "../prebindgen/prebindgen" }|prebindgen = { git = "https://github.com/milyin/prebindgen.git", branch = "main" }|g' \
+            -e 's|prebindgen-proc-macro = { path = "../prebindgen/prebindgen-proc-macro",|prebindgen-proc-macro = { git = "https://github.com/milyin/prebindgen.git", branch = "main",|g' \
+            Cargo.toml ../zenoh-flat/Cargo.toml
+          rm Cargo.toml.bak ../zenoh-flat/Cargo.toml.bak
 
       - uses: actions/setup-java@v4
         with:
@@ -32,31 +59,29 @@ jobs:
       - name: Install Rust toolchain
         run: |
           rustup show
-          rustup component add rustfmt clippy
+          rustup component add rustfmt clippy --toolchain 1.93.0
 
       - name: Cargo Format
-        working-directory: zenoh-jni
-        run: cargo fmt --all --check
+        working-directory: zenoh-flat-jni
+        run: cargo fmt --check
 
-      - name: Clippy Check without Cargo.lock
-        working-directory: zenoh-jni
-        run: |
-          rm Cargo.lock
-          cargo clippy --all-targets --all-features -- -D warnings
-          git restore Cargo.lock
+      - name: Clippy Check
+        working-directory: zenoh-flat-jni
+        run: cargo clippy --all-targets --all-features -- -D warnings
 
       - name: Check for feature leaks
-        working-directory: zenoh-jni
+        working-directory: zenoh-flat-jni
         run: cargo test --no-default-features
 
-      - name: Build Zenoh-JNI
-        working-directory: zenoh-jni
+      - name: Build zenoh-flat-jni
+        working-directory: zenoh-flat-jni
         run: cargo build
 
       - name: Setup Gradle
         uses: gradle/actions/setup-gradle@v4
 
       - name: Gradle Test
+        working-directory: zenoh-java
         run: ./gradlew jvmTest --info
 
   markdown_lint:

ZENOH_FLAT_TRANSITION.md

@@ -0,0 +1,41 @@
+# zenoh-flat transition
+
+This branch (`zenoh-flat-transition`) is the **integration branch** for rebuilding
+zenoh-java on top of the generated JNI/Kotlin bindings, replacing the hand-written
+`zenoh-jni` layer. It exists so the transition can land as a series of reviewable
+PRs targeting this branch instead of `main`; when the transition is complete, this
+branch merges to `main` as a whole (and this file is removed).
+
+## Architecture
+
+```
+zenoh (Rust)
+  └─ zenoh-flat            flat #[prebindgen]-annotated Rust API
+       └─ zenoh-flat-jni   generated JNI externs + Kotlin classes (prebindgen lang::JniGen)
+            └─ zenoh-java  Kotlin SDK wrapper (this repo)
+```
+
+- **prebindgen** — <https://github.com/milyin/prebindgen> (generator)
+- **zenoh-flat** — <https://github.com/ZettaScaleLabs/zenoh-flat> (flat Rust API)
+- **zenoh-flat-jni** — <https://github.com/ZettaScaleLabs/zenoh-flat-jni> (generated bindings,
+  consumed as a sibling checkout in CI and via Gradle composite build locally;
+  as a Maven artifact once published)
+
+## Constituent PRs
+
+| PR | Scope | Status |
+| --- | --- | --- |
+| [#481](https://github.com/eclipse-zenoh/zenoh-java/pull/481) | Use receiver-style zenoh-flat-jni bindings (generated Session/Query/Publisher methods, split key-expr overloads, de-prefixed callback names); +61–83% subscriber throughput | merged |
+| [#484](https://github.com/eclipse-zenoh/zenoh-java/pull/484) | `Encoding`/`ZenohId.toString` become pure JVM values (fixes a per-message native leak); the conversion logic lives in the **shared tier** (`EncodingCodec`/`zidString` in zenoh-flat-jni, correspondence-tested against native, reusable by zenoh-kotlin) with only a thin facade here; publisher **default encoding set natively at declare** (plain puts cross no encoding data). Pairs with [zenoh-flat-jni#4](https://github.com/ZettaScaleLabs/zenoh-flat-jni/pull/4), [zenoh-flat#3](https://github.com/ZettaScaleLabs/zenoh-flat/pull/3) (merged), [prebindgen#80](https://github.com/milyin/prebindgen/pull/80) (merged) | in progress |
+| [#491](https://github.com/eclipse-zenoh/zenoh-java/pull/491) | `Session.undeclare` detaches the key-expr handle even when the native undeclare fails (the generated wrapper consumes it either way); found on the zenoh-kotlin port ([zenoh-kotlin#668](https://github.com/eclipse-zenoh/zenoh-kotlin/pull/668)) | merged |
+| shared-parameters | `Parameters` becomes a thin facade over the shared string-backed implementation in zenoh-flat-jni (Rust `parameters.rs` semantics: no percent-decoding, infallible parse, first-match-wins get) + `ParametersCorrespondenceTest` vs the native oracle. Pairs with [zenoh-flat#4](https://github.com/ZettaScaleLabs/zenoh-flat/pull/4), [zenoh-flat-jni#9](https://github.com/ZettaScaleLabs/zenoh-flat-jni/pull/9) | open |
+
+Companion PRs in the upstream repos are coordinated per constituent PR (e.g.
+[ZettaScaleLabs/zenoh-flat-jni#3](https://github.com/ZettaScaleLabs/zenoh-flat-jni/pull/3)
+pairs with #481 and merges first; CI here pins the exact upstream commits).
+
+## CI pinning
+
+`.github/workflows/ci.yml` on the constituent branches pins the exact
+`zenoh-flat-jni` / `zenoh-flat` commits the code was written against, while
+`prebindgen` resolves from its `main`. Pins are bumped as the upstream PRs land.

examples/build.gradle.kts

@@ -78,19 +78,13 @@ tasks {
 
     examples.forEach { example ->
         register(example, JavaExec::class) {
-            dependsOn("CompileZenohJNI")
+            dependsOn(":zenoh-java:buildZenohFlatJni")
             description = "Run the $example example"
             mainClass.set("io.zenoh.$example")
             classpath(sourceSets["main"].runtimeClasspath)
-            val zenohPaths = "../zenoh-jni/target/release"
+            val zenohPaths = "../zenoh-flat-jni/target/release"
             val defaultJvmArgs = arrayListOf("-Djava.library.path=$zenohPaths")
             jvmArgs(defaultJvmArgs)
         }
     }
 }
-
-tasks.register("CompileZenohJNI") {
-    project.exec {
-        commandLine("cargo", "build", "--release", "--manifest-path", "../zenoh-jni/Cargo.toml")
-    }
-}

examples/src/main/java/io/zenoh/ZQueryable.java

@@ -19,6 +19,8 @@
 import io.zenoh.query.Query;
 import io.zenoh.query.QueryableOptions;
 import io.zenoh.query.ReplyOptions;
+import io.zenoh.config.ZenohId;
+import io.zenoh.time.Timestamp;
 import org.apache.commons.net.ntp.TimeStamp;
 import picocli.CommandLine;
 
@@ -45,6 +47,9 @@ public Integer call() throws Exception {
 
         System.out.println("Opening session...");
         Session session = Zenoh.open(config);
+        // A timestamp is the pair (instant, id of the clock that produced it),
+        // so a reply stamps with THIS session's id.
+        this.zid = session.info().zid();
 
         // A Queryable can be implemented in multiple ways. Uncomment one to try:
         declareQueryableWithBlockingQueue(session, keyExpr);
@@ -98,13 +103,15 @@ private void handleQuery(Query query) {
             String valueInfo = query.getPayload() != null ? " with value '" + query.getPayload() + "'" : "";
             System.out.println(">> [Queryable] Received Query '" + query.getSelector() + "'" + valueInfo);
             var options = new ReplyOptions();
-            options.setTimeStamp(TimeStamp.getCurrentTime());
+            options.setTimeStamp(Timestamp.ofNtp64(TimeStamp.getCurrentTime().ntpValue(), zid));
             query.reply(query.getKeyExpr(), value, options);
         } catch (Exception e) {
             System.err.println(">> [Queryable] Error sending reply: " + e.getMessage());
         }
     }
 
+    private ZenohId zid;
+
     /**
      * ----- Example arguments and private fields -----
      */

examples/src/main/java/io/zenoh/ZSubThr.java

@@ -17,6 +17,7 @@
 import io.zenoh.exceptions.ZError;
 import io.zenoh.keyexpr.KeyExpr;
 import io.zenoh.pubsub.Subscriber;
+import io.zenoh.sample.Sample;
 import picocli.CommandLine;
 
 import java.util.List;
@@ -40,7 +41,7 @@ public Integer call() throws Exception {
         System.out.println("Opening Session");
         try (Session session = Zenoh.open(config)) {
             try (KeyExpr keyExpr = KeyExpr.tryFrom("test/thr")) {
-                subscriber = session.declareSubscriber(keyExpr, sample -> listener(number));
+                subscriber = session.declareSubscriber(keyExpr, sample -> listener(sample, number));
                 System.out.println("Press CTRL-C to quit...");
 
                 while (subscriber.isValid()) {
@@ -51,7 +52,12 @@ public Integer call() throws Exception {
         return 0;
     }
 
-    private void listener(long number) {
+    private void listener(Sample sample, long number) {
+        // Touch the delivered data so it actually reaches the Java surface
+        // (payload bytes + key-expression string): keeps lazy bindings honest
+        // and prevents either side from skipping materialization costs.
+        dataSink += sample.getPayload().toBytes().length
+                + sample.getKeyExpr().toString().length();
         if (batchCount > samples) {
             closeSubscriber();
             report();
@@ -86,8 +92,8 @@ private void report() {
         double elapsedTimeSecs = (double) (end - globalStartTimestampNs) / NANOS_TO_SEC;
         double averageMessagesPerSec = totalMessages / elapsedTimeSecs;
 
-        System.out.printf("Received %d messages in %.2f seconds: averaged %.2f msgs/sec%n",
-                totalMessages, elapsedTimeSecs, averageMessagesPerSec);
+        System.out.printf("Received %d messages in %.2f seconds: averaged %.2f msgs/sec (sink %d)%n",
+                totalMessages, elapsedTimeSecs, averageMessagesPerSec, dataSink);
     }
 
     private void closeSubscriber() {
@@ -114,6 +120,7 @@ private void closeSubscriber() {
     private static final long NANOS_TO_SEC = 1_000_000_000L;
     private long batchCount = 0;
     private long count = 0;
+    private long dataSink = 0;
     private long startTimestampNs = 0;
     private long globalStartTimestampNs = 0;
 

settings.gradle.kts

@@ -23,7 +23,11 @@ rootProject.name = "zenoh-java"
 
 include(":zenoh-java")
 include(":examples")
-include(":zenoh-jni")
+
+// zenoh-flat-jni is now a separate repository (https://github.com/ZettaScaleLabs/zenoh-flat-jni)
+// and is consumed as a Maven artifact: org.eclipse.zenoh:zenoh-flat-jni:VERSION
+// For local development with coordinated changes, use Gradle composite build:
+includeBuild("../zenoh-flat-jni")
 
 plugins {
     id("org.gradle.toolchains.foojay-resolver-convention") version("0.4.0")