Skip to content

[#231] Move the schema to Protocol Buffers edition 2023 - #646

Merged
wes-johnson merged 19 commits into
eclipse-sparkplug:developfrom
wes-johnson:edition-2023
Oct 29, 2026
Merged

wes-johnson merged 19 commits into
eclipse-sparkplug:developfrom
wes-johnson:edition-2023

Conversation

@wes-johnson

Copy link
Copy Markdown
Contributor

Moves the Protocol Buffers schema from syntax = "proto2" to edition = "2023".

Stacked on #645. The schema lines it touches are the ones #645 changes, so it is based on that
branch and should merge after it. The diff below is only this change.

Why editions, and why not proto3

proto2 and proto3 are both frozen syntax modes. Editions superseded them in protoc 27 and are where
protobuf development happens.

proto3 was not an option: it bans extension ranges outright, and its replacement — unknown field
preservation — does not work in JSON, where an unrecognized key is dropped on output and rejected on
input. That would make the extension story worse in the encoding that most needs a good one.

Edition 2024 was rejected on evidence. It enables enforce_naming_style, which produces 36 errors
against this schema because every DataType value is PascalCase. Those names are the JSON wire
format under #645, so complying would rename Int32 to INT32 and UInt32 to U_INT32 in every
payload, example and spec sentence. STYLE_LEGACY avoids that but means adopting 2024 and opting
straight back out of its headline change.

Edition 2024 also changes the generated Java layout: nest_in_file_class defaults to flat, so
SparkplugBProto.Payload becomes a top-level Payload, breaking every Java implementation's
imports. Restoring the nested layout is not possible at file scope —
Option pb.JavaFeatures.nest_in_file_class cannot be set on an entity of type file — so it would
require a Java-specific option annotated on every top-level type, inside a schema that is
language-neutral and shared by C, C++, Python and Go implementations.

The conversion

  • syntax = "proto2" becomes edition = "2023"
  • the optional label is dropped from 48 fields; EXPLICIT presence is the editions default
  • 6 [packed=...] options are dropped; packed is a feature now, and the default for repeated scalars
  • DataType and MetricOperation are marked features.enum_type = CLOSED, keeping proto2 enum
    semantics so neither needs a zero value

The two DataType listings in the chapter are kept textually identical, as before.

Not a wire change

A payload exercising templates, datasets, packed arrays and property sets serializes to
byte-identical output under both schemas, and each decodes the other's bytes:

proto2 bytes       : 181
edition 2023 bytes : 181
BYTE-IDENTICAL     : True
proto2 decodes edition : True
edition decodes proto2 : True

On edition 2023 aging

Low risk. Today's protoc still compiles proto2, a syntax from 2001, so nothing has been dropped in
twenty years. Editions are designed so a file can stay on one indefinitely and migrate per-feature on
its own schedule. Moving to a later edition later is a one-line change plus a runtime pin.

Verification

Build passes; the schema compiles clean under protoc. Broken-anchor (10) and tck-id-anomaly (7)
baselines unchanged. No assertion changes.

Adds a normative JSON encoding of the Sparkplug payload model, published on the
spJv4.0 namespace alongside the Protocol Buffers encoding on spCv4.0.

The mapping is derived from the Google Protocol Buffers JSON Mapping so that most
of it can be produced by library facilities, with Sparkplug's deviations stated
explicitly: TypedValue is flattened to a bare 'value', PropertySet is a JSON
object keyed by property name, array DataTypes are JSON arrays rather than base64
packed bytes, and a DataSet.Column carries one 'values' array.

64-bit integers are encoded as JSON strings. A v4 nanosecond timestamp is roughly
165 times larger than the 2^53 limit a JSON number is guaranteed to carry, so a
numeric form is silently rounded by any parser that decodes into a double.

Also generalizes three Ch. 5 statements that required a Protobuf encoded payload,
notes that topics written with spCv4.0 apply equally to spJv4.0, and renames the
'dataType' key in the Ch. 6 examples to 'datatype' and the metric-level 'template'
key to 'value' so the examples match the encoding rules.
…ing-encoded 64-bit values

The note claimed a Sparkplug sequence number is always small, which is wrong: seq,
bd_seq, and descriptor_seq each count to the maximum value of their declared type
before rolling over to zero. They are therefore subject to the same precision loss
as timestamps, and a numeric form would be correct early in a session and lossy
late in one.
The three message sequence numbers move from uint64 to int64 in the Protocol
Buffers schema, with the range and rollover point changing from
18446744073709551615 to 9223372036854775807 across all 38 statements that bound
them in Chapters 4, 5, and 6.

Protobuf uint64 is a signed long in Java, so the upper half of the uint64 range
was never reachable by an implementation built on it. Declaring int64 makes the
specified range the representable one.

MetaData.seq, which sequences the parts of a multi-part file transfer, is a
different field and is unchanged.
… rsn arithmetic

Header.timestamp and Metric.timestamp move from uint64 to int64, for the same
reason as the sequence numbers: protobuf uint64 is a signed long in Java, so the
upper half of the unsigned range was never reachable. The range note in Chapter 5
is rewritten accordingly - the ceiling is 2262-04-11, stated directly rather than
as a caveat on an unsigned range that no implementation could use.

The NREBIRTH rsn calculation said to add the maximum sequence number to a negative
result. That was off by one, since a counter cycling 0..MAX has a period of MAX+1,
and MAX+1 is not representable as an int64. Restated as two cases so no
intermediate is ever negative or out of range:

  if pds >= 1 + dc:  rsn = pds - (1 + dc)
  otherwise:         rsn = 9223372036854775807 - (dc - pds)

Verified equivalent to (pds - (1 + dc)) mod 2^63 over the full range.
…ing seq as unsigned

The seq, bd_seq, and descriptor_seq bullets in the Header section still described
each field as an unsigned 64-bit integer after the schema moved them to int64.
…ormance profiles

An Edge Node MUST support at least one encoding and MAY support both, selected by
configuration, but MUST use a single encoding and namespace for the whole of an
MQTT session - seq is Edge-Node-wide and bd_seq scopes the session, so splitting
one session across both namespaces would produce two unusable halves of one
sequence stream.

Host Applications and Sparkplug Aware MQTT Servers MUST support both encodings.
A Host Application cannot choose the encoding of the Edge Nodes it consumes, and
an Aware Server decodes payloads rather than treating them as opaque.

A Sparkplug Compliant MQTT Server has no payload requirements at all, which is now
stated non-normatively so the asymmetry with the Aware profile is not read as an
omission.

Also generalizes the Edge Node profile's reference to the spCv4.0/# namespace.
…g the capped range

seq, bd_seq, and descriptor_seq go back to uint64 while keeping the rollover point
at 9223372036854775807. timestamp stays int64.

An unsigned type is right for a counter that can never be negative, and it keeps
the three consistent with alias, MetaData.size, and Status.code. Capping the range
below the type's maximum still guarantees every value a conformant Edge Node can
produce fits a signed 64-bit integer, so an implementation in a language without
an unsigned 64-bit type never needs unsigned comparison. Neither bound is
reachable: at a million messages per second the capped range lasts ~292,000 years.

timestamp keeps int64 because every mainstream time representation it interoperates
with is signed - protobuf's own Timestamp, OPC UA DateTime, java.time.Instant - and
the 2262 ceiling it accepts is one none of those exceed either.

The repeated "(max value of a UINT64)" parenthetical is dropped rather than
rescaled, since the cap is now a deliberate restriction rather than the type's
maximum. It is explained once in the Header section instead.
…ary-support claim

The oneof was named 'content' and its uncompressed arm was named 'plain', so a
Protocol Buffers JSON implementation emitted the key "plain" while every example
in the chapter used "content". A field may not share the name of its containing
oneof, so the oneof is renamed to payload_content and the arm becomes 'content'.
The examples were right and the schema now agrees with them, and a stock library
emits the correct key with no post-processing. Three prose references to the
'plain' field are updated.

The non-normative comment claimed an implementation could produce most of the JSON
encoding with library facilities. Measured against the compiled schema, that is
not true: the structural mappings reshape every Metric, so an encoder writes its
own traversal. What the derivation does settle is the scalar rules and the header,
which the comment now says instead.
…er extension fields

The section leaned on the Google Protocol Buffers JSON Mapping for part of its
meaning. That mapping is vendor documentation rather than a versioned standard,
it is specified for proto3 while the Sparkplug schema is proto2, and it does not
cover extensions at all. The section now states its rules itself and says so.

- The lowerCamelCase form a consumer must accept is defined here - remove each
  underscore and upper case the letter after it - so the assertion no longer
  depends on an external document for its meaning.
- New Extension Fields section: an extension is keyed by its fully qualified name
  in square brackets, its value follows the ordinary rules for its type, and an
  unrecognized extension is ignored. The Sparkplug schema reserves an extension
  range in most messages, so leaving this undefined left a field that is
  well-defined on the wire undefined in JSON. Includes a note that unlike the
  binary encoding, where an unknown field is retained and re-emitted, an
  unrecognized JSON key does not survive a round trip.
- Floating point values must be written with enough digits to parse back exactly:
  up to 9 significant decimal digits for a float and 17 for a double.
…e change time

Metrics in a payload are ordered oldest first, so a consuming application that
applies them in array order arrives at the correct current value without
comparing timestamps. This matters when one payload carries several values for
the same metric, as a payload of historical values does. Metrics remain
identified by name or alias, so position still carries no identity.

A metric that omits its timestamp sorts as though its value change time were the
payload timestamp, which is the time it is already taken to have changed.

Metrics that share a value change time may appear in any order and a consumer
must not reorder them. Multi-part file publishing depends on that: it identifies
a metric by position, and every metric of a multi-part publish carries the same
timestamp, so they tie and the publisher's order is carried unchanged by both
encodings. Non-normative comments in both sections record the interaction.

Also splits the JSON PropertySet statement. Uniqueness of property names is
observable in a payload and stays testable; the absence of meaning in key order
is not observable and becomes non-testable, with a note that RFC 8259 defines a
JSON object as unordered so the requirement is on the publisher.
1. The conformance note justifying the session scoping rule was denser than the
   rule it explained. Reduced to the plain consequence: an Edge Node has one
   sequence number stream, so splitting a session across namespaces leaves a
   consumer of either with gaps in it.

2. Removed the requirement that a consuming application accept the lowerCamelCase
   form of a field name. A conformant publisher emits the Protocol Buffers field
   name, so the rule required consumers to accept a form no conformant publisher
   may produce - specifying leniency toward non-conformance and creating a second
   de facto wire form for every field. Replaced with implementer guidance: a
   Protocol Buffers JSON library converts to lowerCamelCase by default and must be
   configured to preserve the original names.

3. Explained why an absent field is omitted rather than written as null or zero.
   Zero is a legitimate value for seq, bd_seq, descriptor_seq, alias, and quality,
   so writing an absent field as 0 would be indistinguishable from one set to
   zero; and null would be a second, conflicting way to express what is_null
   already expresses.

4. Scoped the ignore-unknown rule to the names of fields within a message. It was
   open to being read as covering an unrecognized metric, which is a different
   situation entirely - a mismatch against the metric structure the application
   holds, resolved by the metadata rules and a Rebirth Request rather than by
   ignoring it. Both cases are now stated and contrasted.
A DateTime value is a nanosecond count since epoch, the same quantity the payload
and metric timestamps carry, so it follows them to int64 rather than staying
unsigned. Leaving it as uint64 meant a DateTime value and a payload timestamp
differed in signedness despite meaning the same thing.

This moves a DateTime value from the uint64_value arm of TypedValue to
int64_value, so unlike the timestamp field changes it is a wire change. A DateTime
in a DataSet.Column is unaffected: there is no signed repeated field, so it stays
in long_values alongside UInt64 and is carried as its two's complement
representation, which the DataSet.Column section already requires for signed
values.
…PropertySet TypedValues

Review found two defects in the normative JSON Schema and its prose.

nonNegative64 was "type": ["string","integer"] with "pattern": "^[0-9]+$". JSON
Schema applies pattern only to strings, so a negative JSON number validated
cleanly, as did a string far beyond the 64-bit cap. That governs seq, bd_seq,
descriptor_seq, timestamp, alias, MetaData size and seq, and Status code - the
fields whose range the rest of the specification relies on. Now bounded on both
arms. The earlier negative tests missed this because they used the string form.

The TypedValue flattening rule named only Metric and Template.Parameter as
datatype carriers, but PropertySet.values is a repeated TypedValue, so a property
value had no enclosing field to carry its datatype and could not be encoded by
following the rule. The JSON Schema already resolved this the other way, nesting
an object with its own datatype - which the rule forbade. The rule is now scoped
to Metric and Template.Parameter, and the PropertySet case is stated to match the
schema. Also corrects a reference to PropertyValues, which is not a v4 message.

@ckienle ckienle left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the change at b6edfc3 (the diff over #645). The conversion looks correct apart from one behaviour change inline. Caveat up front: I couldn't run protoc 27+ here, so this is a static comparison against the proto2 -> edition 2023 feature defaults, not a descriptor-set diff.

Checked clean: field numbers, types, oneofs and extension ranges are unchanged; presence defaults to EXPLICIT, matching proto2 optional; all five packable repeated scalars were already [packed=true], so the PACKED default doesn't change the wire format (and dropping [packed=false] on string_values is right); both enums are marked CLOSED; the duplicated DataType listing at :1619 matches the schema.

Questions:

  • Edition 2023 also changes json_format from LEGACY_BEST_EFFORT to ALLOW, so JSON-name conflicts become protoc errors. The description says the schema compiles clean; which protoc version did you use?
  • Does a current nanopb generate from an edition-2023 file? Tahu C uses nanopb, and if it can't, C implementers will have to hand-convert back to proto2.

Nits:

  • :3579 "Every field ... has explicit presence": repeated fields have no presence; "Every singular field".
  • :3581 is 109 characters; the reworded paragraph wasn't re-wrapped to the chapter's ~100.
  • Dropping optional left the field names and // comments out of column alignment with the repeated lines in the same messages (e.g. :235).

//

syntax = "proto2";
edition = "2023";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should fix (or document): this silently changes UTF-8 handling. proto2 files resolve to features.utf8_validation = NONE; edition 2023 defaults to VERIFY, and the behaviour-preserving conversion adds option features.utf8_validation = NONE; at file level.

Without it, decoders generated from this schema in C++, Python (upb), Go and other upb-based runtimes reject the whole payload if any string field holds invalid UTF-8, where today the same bytes decode. Java is unaffected (it follows its own utf8 option). The "not a wire change" equivalence test can't show this because its sample payload has only valid strings.

The spec already says payload strings are UTF-8, so VERIFY only rejects payloads that were already non-conformant, but it turns one bad field into a dropped message, so it should be a deliberate choice. Either add the option, or keep VERIFY and say so (e.g. "a decoder MAY reject a payload whose string fields are not valid UTF-8").

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 97e5f89, but resolved the opposite way to my first attempt — worth reading, since the outcome reverses what I initially committed.

You were right that this silently changed UTF-8 handling. I first set utf8_validation = NONE to preserve the proto2 behaviour. On review that reasoning was wrong: v4 has not shipped, so there is no byte compatibility to preserve, and the specification already requires string fields to be UTF-8. The schema now keeps VERIFY, which enforces what is already normative rather than accepting payloads the specification calls non-conformant.

Two statements were added to the specification text so the rule belongs to Sparkplug rather than to a Protocol Buffers default that changed between proto2 and edition 2023 — payloads-string-utf8 and payloads-string-utf8-reject. The JSON encoding inherits the same rule from RFC 8259 instead, so stating it once in the payload section covers both.

Rejecting the whole payload rather than the offending field is deliberate and now explained: a payload is the unit a sequence number orders, so accepting part of one would leave a consuming application unable to say whether it had applied that message.

On your "Java follows its own utf8 option" note — I checked. features.(pb.java).utf8_validation defaults to respecting the file level setting, so one option gives every language the same behaviour. I tried setting the Java feature explicitly as well and reverted it: it requires importing java_features.proto, which makes generated code in every other language depend on that file, and the Python runtime does not ship it.

Verified: a payload with an invalid UTF-8 string is rejected, a valid one round-trips.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — added in 2ba1c64.

The rebirth loop is real and worth saying out loud, because the obvious reaction to a rejected BIRTH is to ask for another one. The non-normative comment now says that republishing cannot make a non-conformant payload valid, so a Host Application that rejects an NBIRTH or DBIRTH on this basis gains nothing from a Rebirth Request and will receive the same payload back. It should report the condition and treat the Edge Node as unusable until it is corrected, rather than requesting Rebirths indefinitely.

Agreed with your framing that this is arguably just what a non-conformant Edge Node gets — the value is in telling the implementer what to expect rather than leaving them to discover it from a loop in production.

// To compile:
// cd client_libraries/java
// protoc --proto_path=../../ --java_out=src/main/java ../../sparkplug_c.proto
// Requires protoc 27 or later for edition 2023 support.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: worth stating the runtime floor too, not just protoc. Code generated by protoc 27+ for an edition-2023 file needs a matching runtime (protobuf-java >= 4.27, Python protobuf >= 5.27, etc.). Tahu Java is on protobuf-java 3.25.5, so implementers regenerating from the spec need a runtime upgrade as well.

Suggest: "Requires protoc 27 or later and a Protocol Buffers runtime that supports edition 2023".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 66d8e5f, using your wording. The compile comment now reads:

// Requires protoc 27 or later and a Protocol Buffers runtime that supports edition 2023
// (protobuf-java 4.27 or later, Python protobuf 5.27 or later, and equivalents).

Worth noting alongside it: #647 moves the TCK from protobuf-java 3.17.3 to 4.36.2, so the TCK itself clears that floor. Tahu Java on 3.25.5 will need the same bump when it regenerates from this schema, which is exactly the point you were making.

…essing, and schema bounds

Required:

- The namespace substitution note made STATE ambiguous. STATE is pinned to
  spCv4.0/STATE/sparkplug_host_id, so a JSON Edge Node following the note would
  have waited on spJv4.0/STATE while the Host published on spCv4.0/STATE and never
  seen it come online. STATE is now stated as the one exception, with the reason.

- Nothing said which namespace a Host Application uses to address a particular
  Edge Node. A Host supports both encodings, so it could send an NCMD or a Rebirth
  Request on a namespace the Edge Node does not subscribe to; the MQTT Server
  accepts it and it is never delivered, so Rebirth recovery fails silently. A Host
  now publishes to an Edge Node on the namespace of that Edge Node's NBIRTH, and
  an Edge Node subscribes to its command topics on the namespace it publishes on.

Also:

- nonNegative64 capped every 64-bit field at 2^63-1, but alias, MetaData.size and
  Status.code are uncapped uint64, so a legal spCv4.0 payload could not be
  expressed as a schema-valid spJv4.0 one - contradicting the lossless conversion
  this section claims. Split into uint64 and sequence64, with the string branch of
  sequence64 bounded by length and its limit stated.
- The Sparkplug Aware MQTT Server encoding requirement is now conditional on
  replacing NDEATH timestamps, which is the only payload-touching behavior
  required of it and is itself a MAY.
- Flattening drops the TypedValue's own datatype and is_null. The two copies must
  now agree in the Protocol Buffers encoding, the single JSON key represents both,
  and a null Template.Parameter has a JSON form: is_null is added to the parameter
  keys and to the schema.
- The metric ordering MUST is scoped to payloads carrying more than one value for
  the same metric, with a SHOULD elsewhere so a BIRTH in browse order stays
  conformant, and it now says it does not reach Template or Command metrics.
- Nits: INT64 rollover wording, a stale (max value of an INT64), DateTimeArray
  elements described as signed, and a MUST NOT inside a non-normative comment.
@wes-johnson

Copy link
Copy Markdown
Contributor Author

Both questions answered, and the UTF-8 one is now resolved the other way.

protoc version: 4.36.2, fetched by Gradle via the protobuf plugin in #647, so no local protoc
install is involved. json_format moving to ALLOW produced no errors — there are no JSON-name
conflicts in this schema.

nanopb: yes. I said earlier I couldn't test this; that was wrong, and I have now run it.
nanopb 0.4.9.1 generates from the edition 2023 file without complaint:

$ nanopb_generator proto/sparkplug_c.proto
Writing to proto/sparkplug_c.pb.h and proto/sparkplug_c.pb.c

The output carries the v4 constructs (Payload_Header, Payload_Content, DataSet_Column,
descriptor_seq, quality, MetricOperation) and preserves explicit presence — 27 has_ flags,
including has_seq, has_bd_seq, has_timestamp, has_descriptor_seq and has_quality. It still
generates after the utf8 change below. Caveat: the pip package ships only the generator, not the C
runtime headers, so I could not compile the generated .c here. Generation with correct structure
is strong evidence, not proof.

utf8_validation is now VERIFY, not NONE. Your point stood but the resolution went further: v4
has not shipped, so preserving proto2 decoding behaviour is not a goal, and the specification
already requires string fields to be UTF-8. VERIFY enforces what is already normative. The schema
now sets it explicitly with the reason, and two statements were added to the specification text so
the requirement belongs to Sparkplug rather than to a Protocol Buffers default that changed between
proto2 and edition 2023 — the JSON encoding inherits the same rule from RFC 8259 instead.

A payload with an invalid UTF-8 string is rejected whole rather than per field, because a payload is
the unit a sequence number orders; accepting part of one would leave a consumer unable to say
whether it had applied that message.

One thing I checked because of your "Java follows its own utf8 option" note: Java's
features.(pb.java).utf8_validation defaults to respecting the file level setting, so one option
gives every language the same behaviour. I tried setting the Java feature explicitly as well and
reverted it — it requires importing java_features.proto, which makes generated code in every
other language depend on that file, and the Python runtime does not ship it.

Also addressed: the runtime floor is now stated alongside the protoc one, "every singular field",
the paragraph is re-wrapped, and the schema is realigned after the optional label was dropped —
whitespace only, asserted by comparing the token streams before and after.

@ckienle ckienle left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 97e5f89. Keeping VERIFY and making it normative (payloads-string-utf8 / -reject) is a better resolution than the NONE I suggested: v4 hasn't shipped, the spec already required UTF-8, and stating it in the spec rather than leaving it to a protobuf feature default is right. Thanks for confirming protoc 4.36.2 and nanopb 0.4.9.1, and for the runtime floor in the compile comment. Approving.

One optional thought: with the reject rule, a Host that rejects an NBIRTH (or DBIRTH) because of an invalid UTF-8 string will request a Rebirth and get the same BIRTH back, indefinitely. That's arguably just what a non-conformant Edge Node gets, but a sentence in the non-normative comment saying what a consumer should expect (or that it shouldn't keep requesting rebirths for a payload it rejected) might save an implementer some confusion.

…gth-bounded string form

MetaData.seq sequences the parts of a multi-part file transfer and is an uncapped
uint64 in both the schema and the prose. The earlier split pointed it at
sequence64 because the bulk rename matched on the field name, which the two
fields share, so it was capped at 2^63-1 by accident - the same class of mistake
as the regex that produced the recursion.

Also gives the uint64 definition the same note sequence64 carries, since its
string branch is bounded by length rather than by value, and re-wraps the
DateTimeArray entry left split mid-phrase by the earlier edit.
proto2 and proto3 are both frozen syntax modes; editions superseded them in
protoc 27 and is where protobuf development happens. Moving to proto3 was not an
option - it bans extension ranges, and its replacement, unknown field
preservation, does not work in JSON, which would make the extension story worse
in the encoding that most needs a good one.

Edition 2024 was rejected: it enables enforce_naming_style, which produces 36
errors against this schema because every DataType value is PascalCase. Those
names are the JSON wire format under this specification, so complying would
rename Int32 to INT32 and UInt32 to U_INT32 in every payload and example. The
alternative, features.enforce_naming_style = STYLE_LEGACY, means adopting 2024
and opting straight back out of it.

The conversion is mechanical:
- syntax = "proto2" becomes edition = "2023"
- the optional label is dropped from 48 fields; EXPLICIT presence is the
  editions default
- 6 [packed=...] options are dropped; packed is a feature, and the default for
  repeated scalars
- DataType and MetricOperation are marked features.enum_type = CLOSED to keep
  proto2 enum semantics, so neither needs a zero value

This is not a wire change. A payload exercising templates, datasets, packed
arrays, and property sets serializes to byte-identical output under both
schemas, and each decodes the other's bytes.

The TCK's protobuf runtime pin is left alone. Edition 2023 needs protobuf-java
4.27+, but the checked-in SparkplugBProto.java was generated by protoc 3.17 and
does not compile against a 4.x runtime, so the bump is inseparable from the
regeneration tracked in eclipse-sparkplug#641.
…iour under edition 2023

proto2 files resolve to features.utf8_validation = NONE; edition 2023 defaults to
VERIFY. Without pinning it, decoders generated from this schema on upb-based
runtimes - C++, Python, Go - would reject a whole payload when any string field
holds invalid UTF-8, where the Sparkplug 3 schema decoded it. The byte-identical
equivalence test could not surface this because its sample carried only valid
strings. NONE is set explicitly so the conversion stays behaviour preserving, and
the reason is recorded in the schema.

Also states the runtime floor alongside the protoc one, since generated code for
an edition 2023 file needs protobuf-java 4.27 or later and equivalents.

Nits: repeated fields have no presence, so "every singular field"; the reworded
paragraph is re-wrapped to the chapter width; and the schema is realigned, since
dropping the optional label left singular fields out of column with the repeated
ones. The realignment changes whitespace only, asserted by comparing the token
streams before and after.
…to2 behaviour

Reverses the earlier choice of utf8_validation = NONE. That was made to preserve
Sparkplug 3 decoding behaviour, which is not a goal: v4 has not shipped, so there
is no byte compatibility to keep, and the specification already requires string
fields to be UTF-8. VERIFY enforces what is already normative instead of
accepting payloads the specification calls non-conformant.

The Java runtime's own utf8_validation feature defaults to respecting the file
level setting, so one option gives every language the same behaviour. Setting the
Java feature explicitly was tried and reverted: it requires importing
java_features.proto, which makes generated code in every other language depend on
that file, and the Python runtime does not ship it.

Adds the requirement to the specification text rather than leaving it implicit in
the encoding, since the Protocol Buffers default changed between proto2 and
edition 2023 and the JSON encoding inherits the rule from RFC 8259 instead. A
non-conformant payload is rejected whole rather than per field, because a payload
is the unit a sequence number orders.

Verified: a payload with an invalid UTF-8 string is rejected and a valid one round
trips, and nanopb still generates from the schema.
…it rejects

With the UTF-8 reject rule, a Host Application that rejects an NBIRTH or DBIRTH
would otherwise issue a Rebirth Request and receive the same non-conformant
payload back, indefinitely. The non-normative comment now says that republishing
cannot make a payload valid, so the Host should report the condition and treat
the Edge Node as unusable rather than requesting Rebirths until it is corrected.
@wes-johnson
wes-johnson merged commit a004163 into eclipse-sparkplug:develop Oct 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants