From 4198e90c5e45d1177d43c94102d4f5efaef62670 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Tue, 20 Oct 2026 20:40:00 +0000 Subject: [PATCH 01/15] [#231] Define the spJv4.0 JSON payload encoding Adds a normative JSON encoding of the Sparkplug payload model, published on the spJv4.0 namespace alongside the Protocol Buffers encoding on spCv4.0. The mapping is derived from the Google Protocol Buffers JSON Mapping so that most of it can be produced by library facilities, with Sparkplug's deviations stated explicitly: TypedValue is flattened to a bare 'value', PropertySet is a JSON object keyed by property name, array DataTypes are JSON arrays rather than base64 packed bytes, and a DataSet.Column carries one 'values' array. 64-bit integers are encoded as JSON strings. A v4 nanosecond timestamp is roughly 165 times larger than the 2^53 limit a JSON number is guaranteed to carry, so a numeric form is silently rounded by any parser that decodes into a double. Also generalizes three Ch. 5 statements that required a Protobuf encoded payload, notes that topics written with spCv4.0 apply equally to spJv4.0, and renames the 'dataType' key in the Ch. 6 examples to 'datatype' and the metric-level 'template' key to 'value' so the examples match the encoding rules. --- .../asciidoc/chapters/Sparkplug_4_Topics.adoc | 6 + .../Sparkplug_5_Operational_Behavior.adoc | 9 +- .../chapters/Sparkplug_6_Payloads.adoc | 582 +++++++++++++++++- 3 files changed, 562 insertions(+), 35 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc index 1593157c..3d3925de 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc @@ -72,6 +72,12 @@ and spJv4.0 namespaces, which differ only in payload encoding. For earlier 'Spar 'Sparkplug B' topic namespace definitions and concepts, see the earlier versions of the Sparkplug Specification. +Non-normative comment: Topics are written with the spCv4.0 namespace element throughout the +remainder of this specification. Every such topic applies equally to the spJv4.0 namespace with +that element substituted, because the two namespaces share one topic structure. The encoding of the +payload carried on a topic is determined by the namespace element of that topic, as required in the +link:#payloads_json_encoding_selection[Encoding Selection] section. + [[topics_topic_prefix_element]] ==== topic_prefix Element diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc index c5a6d539..5bd893de 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc @@ -470,7 +470,8 @@ Edge Node's MQTT Will Message's topic MUST be of the form '[topic_prefix/]spCv4. where the optional topic_prefix is the Sparkplug Topic Prefix and the edge_node_id is the Sparkplug Edge Node ID for this Edge Node*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-will-message-payload]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-will-message-payload] The -Edge Node's MQTT Will Message's payload MUST be a Sparkplug Google Protobuf encoded payload.*# +Edge Node's MQTT Will Message's payload MUST be a Sparkplug payload encoded as defined for the +namespace on which it is published.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-will-message-payload-bdSeq]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-will-message-payload-bdSeq] The Edge Node's MQTT Will Message's payload MUST include a bd_seq number in the payload and the value MUST be incremented by one from the value in the previous MQTT CONNECT packet unless the value would @@ -566,7 +567,8 @@ Edge Node's NBIRTH MQTT topic MUST be of the form '[topic_prefix/]spCv4.0/NBIRTH/edge_node_id' where the optional topic_prefix is the Sparkplug Topic Prefix and the edge_node_id is the Sparkplug Edge Node ID for this Edge Node*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-nbirth-payload]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-nbirth-payload] The -Edge Node's NBIRTH payload MUST be a Sparkplug Google Protobuf encoded payload.*# +Edge Node's NBIRTH payload MUST be a Sparkplug payload encoded as defined for the namespace on +which it is published.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-nbirth-payload-bdSeq]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-nbirth-payload-bdSeq] The Edge Node's NBIRTH payload MUST include a bd_seq number in the payload and the value MUST be the same as the bd_seq number value specified in the immediately previous MQTT CONNECT packet.*# @@ -851,7 +853,8 @@ identify the Sparkplug Device.*# Device's DBIRTH MQTT topic topic_prefix and edge_node_id MUST match the topic_prefix and edge_node_id that were sent in the prior NBIRTH message for the Edge Node this Device is associated with.*# * [tck-testable tck-id-message-flow-device-birth-publish-dbirth-payload]#[yellow-background]*[tck-id-message-flow-device-birth-publish-dbirth-payload] The -Device's DBIRTH payload MUST be a Sparkplug Google Protobuf encoded payload.*# +Device's DBIRTH payload MUST be a Sparkplug payload encoded as defined for the namespace on which +it is published.*# * [tck-testable tck-id-message-flow-device-birth-publish-dbirth-qos]#[yellow-background]*[tck-id-message-flow-device-birth-publish-dbirth-qos] The Device's DBIRTH MQTT QoS MUST be 0.*# * [tck-testable tck-id-message-flow-device-birth-publish-dbirth-retained]#[yellow-background]*[tck-id-message-flow-device-birth-publish-dbirth-retained] The diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index c52650ac..b294f6c5 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -378,7 +378,7 @@ JSON as follows: "name": , "alias": , "timestamp": , - "dataType": , + "datatype": , "value": }] } @@ -401,7 +401,7 @@ A simple Sparkplug B payload with values would be represented as follows: "name": "My Metric", "alias": 1, "timestamp": 1479123452194000000, - "dataType": "String", + "datatype": "String", "value": "Test" }] } @@ -1255,8 +1255,8 @@ Acme/Dublin/Packaging/spCv4.0/NMETA/EdgeNode1 "content": { "metrics": [{ "name": "MotorType", - "dataType": "Template", - "template": { + "datatype": "Template", + "value": { "version": "1.0", "is_definition": true, "parameters": [{ @@ -1266,13 +1266,13 @@ Acme/Dublin/Packaging/spCv4.0/NMETA/EdgeNode1 }], "metrics": [{ "name": "RPM", - "dataType": "Int32" + "datatype": "Int32" }, { "name": "AmpDraw", - "dataType": "Float" + "datatype": "Float" }, { "name": "Running", - "dataType": "Boolean" + "datatype": "Boolean" }] } }] @@ -1300,16 +1300,16 @@ Acme/Dublin/Packaging/spCv4.0/DMETA/EdgeNode1/Line3 "content": { "metrics": [{ "name": "Motors/Conveyor", - "dataType": "Template", - "template": { + "datatype": "Template", + "value": { "version": "1.0", "is_definition": false, "template_ref": "MotorType" } }, { "name": "Motors/Filler", - "dataType": "Template", - "template": { + "datatype": "Template", + "value": { "version": "1.0", "is_definition": false, "template_ref": "MotorType" @@ -1339,8 +1339,8 @@ Acme/Dublin/Packaging/spCv4.0/DBIRTH/EdgeNode1/Line3 "content": { "metrics": [{ "name": "Motors/Conveyor", - "dataType": "Template", - "template": { + "datatype": "Template", + "value": { "version": "1.0", "is_definition": false, "template_ref": "MotorType", @@ -1351,36 +1351,36 @@ Acme/Dublin/Packaging/spCv4.0/DBIRTH/EdgeNode1/Line3 }], "metrics": [{ "name": "RPM", - "dataType": "Int32", + "datatype": "Int32", "value": 1780 }, { "name": "AmpDraw", - "dataType": "Float", + "datatype": "Float", "value": 12.4 }, { "name": "Running", - "dataType": "Boolean", + "datatype": "Boolean", "value": true }] } }, { "name": "Motors/Filler", - "dataType": "Template", - "template": { + "datatype": "Template", + "value": { "version": "1.0", "is_definition": false, "template_ref": "MotorType", "metrics": [{ "name": "RPM", - "dataType": "Int32", + "datatype": "Int32", "value": 900 }, { "name": "AmpDraw", - "dataType": "Float", + "datatype": "Float", "value": 7.1 }, { "name": "Running", - "dataType": "Boolean", + "datatype": "Boolean", "value": false }] } @@ -1408,13 +1408,13 @@ Acme/Dublin/Packaging/spCv4.0/DDATA/EdgeNode1/Line3 "content": { "metrics": [{ "name": "Motors/Conveyor", - "dataType": "Template", - "template": { + "datatype": "Template", + "value": { "is_definition": false, "template_ref": "MotorType", "metrics": [{ "name": "RPM", - "dataType": "Int32", + "datatype": "Int32", "value": 1802 }] } @@ -2640,7 +2640,7 @@ Consider the following Sparkplug B payload in the NCMD message shown above: "metrics": [{ "name": "DIGITAL_OUT_1", "timestamp": 1486144502122000000, - "dataType": "Boolean", + "datatype": "Boolean", "value": true }] } @@ -2671,12 +2671,12 @@ The following is a representation of an NCMD payload that invokes the 'Calibrate "metrics": [{ "name": "Calibrate", "timestamp": 1486144502122000000, - "dataType": "Command", + "datatype": "Command", "command": { "correlation_id": "5d7c1f6e-9c2a-4f1b-8f3a-1c9d2e6b4a01", "arguments": [{ "name": "channel", - "dataType": "Int32", + "datatype": "Int32", "value": 1 }] } @@ -2735,12 +2735,12 @@ Consider the following Sparkplug B payload in the DCMD message shown above: "metrics": [{ "name": "Motor/Run", "timestamp": 1486144502122000000, - "dataType": "Boolean", + "datatype": "Boolean", "value": true }, { "name": "Motor/Speed Setpoint", "timestamp": 1486144502122000000, - "dataType": "Int32", + "datatype": "Int32", "value": 120 }] } @@ -2799,7 +2799,7 @@ Consider the following Sparkplug B payload in the NRESP message shown above: "metrics": [{ "name": "Calibrate", "timestamp": 1486144502122000000, - "dataType": "Command", + "datatype": "Command", "command": { "correlation_id": "5d7c1f6e-9c2a-4f1b-8f3a-1c9d2e6b4a01", "status": { @@ -2808,7 +2808,7 @@ Consider the following Sparkplug B payload in the NRESP message shown above: }, "results": [{ "name": "offset", - "dataType": "Double", + "datatype": "Double", "value": 0.125 }] } @@ -2863,7 +2863,7 @@ Consider the following Sparkplug B payload in the DRESP message shown above: "metrics": [{ "name": "Reset", "timestamp": 1486144502122000000, - "dataType": "Command", + "datatype": "Command", "command": { "correlation_id": "a1b2c3d4-0001", "status": { @@ -3444,3 +3444,521 @@ on the publishing (transmission) side, and 5xx indicate an error on the receivin |Receiving Side Error |Error Renaming Partial File |=== + +[[payloads_json]] +=== Sparkplug JSON Payload Definition + +[[payloads_json_overview]] +==== Overview + +Sparkplug defines two payload encodings of the same data model. The Google Protocol Buffers +encoding is defined in the link:#payloads_sparkplug_c_mqtt_payload_definition[Sparkplug B MQTT +Payload Definition] section above and is published on the spCv4.0 namespace. The JSON encoding is +defined in this section and is published on the spJv4.0 namespace. + +The two encodings carry identical information. Every component defined in the sections above - +Header, Content, Metric, Quality, MetaData, PropertySet, DataSet, Template, Command, and the +DataTypes - applies unchanged to the JSON encoding, and every normative statement made about those +components applies to both encodings. This section defines only how those components are +represented as JSON text. All operational behavior, topic structure, sequence numbering, and +session management defined elsewhere in this specification is encoding independent. + +Non-normative comment: Because the two encodings are two representations of one model, a message +may be converted from one to the other without loss and without consulting anything outside the +message itself. This is what allows a bridging application to republish an spCv4.0 namespace onto +spJv4.0, or the reverse, as a mechanical transformation. + +Non-normative comment: The JSON encoding is derived from the JSON Mapping defined by Google +Protocol Buffers, which most Protocol Buffers libraries implement directly. Where this +specification deviates from that mapping it does so explicitly, and those deviations are collected +in the link:#payloads_json_structural[Structural Mappings] section. An implementation that already +has the Sparkplug Protocol Buffers schema can therefore produce most of the JSON encoding with +library facilities rather than with hand-written code. + +[[payloads_json_encoding_selection]] +==== Encoding Selection + +[tck-testable tck-id-payloads-json-namespace-encoding]#[yellow-background]*[tck-id-payloads-json-namespace-encoding] The +payload of any Sparkplug message published on the spJv4.0 namespace MUST be encoded as defined in +this section.*# + +[tck-testable tck-id-payloads-protobuf-namespace-encoding]#[yellow-background]*[tck-id-payloads-protobuf-namespace-encoding] The +payload of any Sparkplug message published on the spCv4.0 namespace MUST be encoded using the +Google Protocol Buffers schema defined in the +link:#payloads_c_google_protocol_buffer_schema[Google Protocol Buffer Schema] section.*# + +[tck-testable tck-id-payloads-json-utf8]#[yellow-background]*[tck-id-payloads-json-utf8] A Sparkplug +JSON payload MUST be a JSON text as defined by RFC 8259, encoded as UTF-8 with no byte order +mark.*# + +[tck-testable tck-id-payloads-json-object]#[yellow-background]*[tck-id-payloads-json-object] The +top level JSON value of a Sparkplug JSON payload MUST be a JSON object representing the Payload +message.*# + +Non-normative comment: An Edge Node publishes on one namespace, not both. An Edge Node that +publishes on spJv4.0 is a complete Sparkplug Edge Node and is not required to implement the +Protocol Buffers encoding at all. The choice is a deployment decision; see the +link:#payloads_json_choosing[Choosing a Payload Encoding] section. + +[[payloads_json_field_naming]] +==== Field Naming + +[tck-testable tck-id-payloads-json-field-names]#[yellow-background]*[tck-id-payloads-json-field-names] Each +JSON object key MUST be the name of the corresponding field exactly as it appears in the Sparkplug +Google Protocol Buffers schema, except where the +link:#payloads_json_structural[Structural Mappings] section names a different key.*# + +[tck-testable tck-id-payloads-json-field-names-accept]#[yellow-background]*[tck-id-payloads-json-field-names-accept] A +consuming application MUST also accept the lowerCamelCase form of a field name, because that is the +form a Google Protocol Buffers JSON Mapping implementation emits by default.*# + +[tck-testable tck-id-payloads-json-omit-absent]#[yellow-background]*[tck-id-payloads-json-omit-absent] A +field that is not present MUST be omitted from the JSON object rather than written with a JSON null +or a zero value.*# + +[tck-testable tck-id-payloads-json-ignore-unknown]#[yellow-background]*[tck-id-payloads-json-ignore-unknown] A +consuming application MUST ignore any JSON object key it does not recognize, and MUST NOT treat the +payload as malformed on that basis.*# + +Non-normative comment: The naming rule keeps a JSON payload visually identical to the field names +in the Protocol Buffers schema, so the two encodings can be read against one another and against +the component definitions above without a translation table. Ignoring unrecognized keys is the JSON +counterpart of the extension ranges reserved in the Protocol Buffers schema, and serves the same +purpose for third party extensions. + +[[payloads_json_scalar]] +==== Scalar Value Mapping + +The following table defines how each Protocol Buffers scalar type is represented in JSON. + +[cols="2,2,4"] +|=== +|*Protocol Buffers Type* +|*JSON Type* +|*Representation* + +|int32, uint32, sint32, fixed32, sfixed32 +|number +|A JSON number with no fractional part or exponent. + +|int64, uint64, sint64, fixed64, sfixed64 +|string +|The value written in decimal, as a JSON string. For example "1486144502122847913". + +|float, double +|number or string +|A JSON number, or one of the strings "NaN", "Infinity", or "-Infinity". + +|bool +|true or false +|A JSON literal, never the strings "true" or "false" and never 0 or 1. + +|string +|string +|The string value. + +|bytes +|string +|The bytes, base64 encoded as defined in RFC 4648 Section 4, including padding. + +|enum +|string +|The name of the enumerated value, for example "Int32", "Template", or "ADD". +|=== + +[tck-testable tck-id-payloads-json-64bit-string]#[yellow-background]*[tck-id-payloads-json-64bit-string] A +publishing application MUST represent every 64-bit integer field as a JSON string containing the +decimal value, including when the value is small enough to be represented exactly as a JSON +number.*# + +[tck-testable tck-id-payloads-json-64bit-accept]#[yellow-background]*[tck-id-payloads-json-64bit-accept] A +consuming application MUST accept a 64-bit integer field represented as either a JSON string or a +JSON number.*# + +[tck-testable tck-id-payloads-json-bytes-base64]#[yellow-background]*[tck-id-payloads-json-bytes-base64] A +field of Protocol Buffers type bytes MUST be represented as a JSON string containing the base64 +encoding of those bytes as defined in RFC 4648 Section 4, including padding.*# + +[tck-testable tck-id-payloads-json-enum-name]#[yellow-background]*[tck-id-payloads-json-enum-name] A +publishing application MUST represent an enumerated value, including a DataType and a +MetricOperation, as a JSON string containing the name of that value.*# + +[tck-testable tck-id-payloads-json-enum-accept]#[yellow-background]*[tck-id-payloads-json-enum-accept] A +consuming application MUST accept an enumerated value represented as either the name of that value +or the integer value assigned to it in the Google Protocol Buffers schema.*# + +[tck-testable tck-id-payloads-json-float-nonfinite]#[yellow-background]*[tck-id-payloads-json-float-nonfinite] A +non-finite float or double value MUST be represented as one of the JSON strings "NaN", "Infinity", +or "-Infinity", because RFC 8259 provides no numeric form for these values.*# + +Non-normative comment: 64-bit integers are represented as strings because a JSON number is not +required by RFC 8259 to carry more precision than an IEEE 754 double, which holds integers exactly +only up to 2^53^. Several widely used JSON parsers, including the one built into every web browser, +decode JSON numbers into doubles unconditionally. A Sparkplug timestamp is a nanosecond count and +is therefore roughly 165 times larger than that limit, so a timestamp written as a JSON number +would be silently rounded, typically to the nearest 256 nanoseconds. Metric values of type Int64, +UInt64, and DateTime have the same exposure. + +Non-normative comment: The rule is driven by the declared type of the field rather than by the +magnitude of the value, so seq, bd_seq, and descriptor_seq are written as strings even though a +Sparkplug sequence number is always small. This keeps the mapping mechanical: an encoder derives +it from the schema alone, and no value can lose precision because of how large it happened to be. + +[[payloads_json_structural]] +==== Structural Mappings + +A Protocol Buffers message is represented as a JSON object and a repeated field as a JSON array. +The following cases are where the Sparkplug JSON encoding deviates from a direct mapping of the +Protocol Buffers schema, in each case so that the JSON encoding reads as JSON rather than as a +transliteration of a binary format. + +[[payloads_json_typed_values]] +===== Typed Values + +In the Protocol Buffers schema a Metric carries its value in a TypedValue submessage, which repeats +the datatype and selects one of several value fields. In JSON the datatype is already carried by +the enclosing object and the value field is not needed to disambiguate. + +[tck-testable tck-id-payloads-json-typedvalue-flat]#[yellow-background]*[tck-id-payloads-json-typedvalue-flat] A +TypedValue MUST be represented as the value itself, under the key 'value', rather than as a nested +object, and its datatype MUST be carried by the 'datatype' key of the enclosing Metric or the +'type' key of the enclosing Template.Parameter.*# + +[tck-testable tck-id-payloads-json-typedvalue-parameter-keys]#[yellow-background]*[tck-id-payloads-json-typedvalue-parameter-keys] A +Template.Parameter MUST therefore be represented with the keys 'name', 'type', and 'value', and +MUST NOT use the key 'typed_value'.*# + +Non-normative comment: These are the two places where a JSON key is not a Protocol Buffers field +name. 'value' is the name of the oneof within TypedValue rather than of any field in it, because +flattening replaces the whole oneof with the single value it holds; a Metric's own field is also +called value, so a Metric reads the same either way. A Template.Parameter's field is typed_value, +and flattening replaces it with the 'type' and 'value' pair used for parameters throughout this +specification. + +[tck-testable tck-id-payloads-json-null]#[yellow-background]*[tck-id-payloads-json-null] A value +that is null MUST be represented by setting 'is_null' to true and omitting the 'value' key.*# + +[[payloads_json_propertyset]] +===== PropertySet + +In the Protocol Buffers schema a PropertySet is a pair of parallel arrays of keys and values. In +JSON it is represented as an object, which is both smaller and directly usable by a consuming +application. + +[tck-testable tck-id-payloads-json-propertyset-object]#[yellow-background]*[tck-id-payloads-json-propertyset-object] A +PropertySet MUST be represented as a JSON object whose keys are the property names and whose values +are the corresponding PropertyValues.*# + +[tck-testable tck-id-payloads-json-propertyset-unique]#[yellow-background]*[tck-id-payloads-json-propertyset-unique] Property +names within a single PropertySet MUST be unique, and the order of the keys MUST NOT be +significant.*# + +[[payloads_json_array_types]] +===== Array Types + +In the Protocol Buffers encoding the array DataTypes are carried as packed little-endian bytes, as +described in the link:#payloads_c_datatype_array[Array Types] section. Carrying those same packed +bytes in JSON would make an array value opaque to a JSON consumer. + +[tck-testable tck-id-payloads-json-array-values]#[yellow-background]*[tck-id-payloads-json-array-values] A +value of an array DataType MUST be represented as a JSON array of the element values, not as a +base64 encoded packed byte array.*# + +[tck-testable tck-id-payloads-json-array-element-type]#[yellow-background]*[tck-id-payloads-json-array-element-type] Each +element of that JSON array MUST be represented according to the scalar mapping for the element +type, so that the elements of an Int64Array and a DateTimeArray are JSON strings and the elements +of an Int32Array are JSON numbers.*# + +[[payloads_json_dataset]] +===== DataSet + +In the Protocol Buffers schema a DataSet.Column carries its values in one of six parallel arrays, +one per underlying Protocol Buffers type, of which exactly one is used. In JSON the column's +datatype already identifies the element type. + +[tck-testable tck-id-payloads-json-dataset-values]#[yellow-background]*[tck-id-payloads-json-dataset-values] A +DataSet.Column MUST carry its values as a single JSON array under the key 'values', rather than as +one of the typed value arrays named in the Google Protocol Buffers schema, and each element MUST be +represented according to the scalar mapping for the column's datatype.*# + +[[payloads_json_content]] +===== Content and Compression + +[tck-testable tck-id-payloads-json-content-key]#[yellow-background]*[tck-id-payloads-json-content-key] An +uncompressed payload MUST carry its content under the key 'content', and a compressed payload MUST +carry it under the key 'compressed', matching the two arms of the content oneof in the Google +Protocol Buffers schema.*# + +[tck-testable tck-id-payloads-json-compressed-base64]#[yellow-background]*[tck-id-payloads-json-compressed-base64] The +value of 'compressed' MUST be a JSON string containing the base64 encoding of the compressed bytes, +and those bytes MUST be the compressed form of the UTF-8 JSON text of the Content object.*# + +Non-normative comment: The header of a compressed JSON payload is a sibling of the 'compressed' key +and remains readable without decompressing, exactly as in the Protocol Buffers encoding. + +[[payloads_json_schema]] +==== JSON Schema + +The following JSON Schema describes a Sparkplug JSON payload. It is normative for the structure and +types it expresses. Requirements stated elsewhere in this specification that a JSON Schema cannot +express, such as the relationship between a metric's datatype and the form of its value, apply in +addition. + +[tck-testable tck-id-payloads-json-schema-valid]#[yellow-background]*[tck-id-payloads-json-schema-valid] A +Sparkplug JSON payload MUST validate against the JSON Schema below.*# + +---- +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://sparkplug.eclipse.org/specification/4.0.0/sparkplug_j.json", + "title": "Sparkplug JSON Payload", + "type": "object", + "properties": { + "header": { "$ref": "#/$defs/header" }, + "content": { "$ref": "#/$defs/content" }, + "compressed": { "$ref": "#/$defs/base64" } + }, + "not": { "required": [ "content", "compressed" ] }, + "$defs": { + "uint64": { "type": [ "string", "integer" ], "pattern": "^[0-9]+$" }, + "int64": { "type": [ "string", "integer" ], "pattern": "^-?[0-9]+$" }, + "base64": { "type": "string", "contentEncoding": "base64" }, + "number": { + "oneOf": [ + { "type": "number" }, + { "enum": [ "NaN", "Infinity", "-Infinity" ] } + ] + }, + "datatype": { + "oneOf": [ + { "type": "string" }, + { "type": "integer", "minimum": 0 } + ] + }, + "header": { + "type": "object", + "properties": { + "timestamp": { "$ref": "#/$defs/uint64" }, + "seq": { "$ref": "#/$defs/uint64" }, + "bd_seq": { "$ref": "#/$defs/uint64" }, + "descriptor_seq": { "$ref": "#/$defs/uint64" }, + "definition_id": { "type": "string" }, + "previous_definition_id": { "type": "string" }, + "source": { "type": "string" }, + "uuid": { "type": "string" }, + "compression_algorithm": { "type": "string" } + } + }, + "content": { + "type": "object", + "properties": { + "metrics": { "type": "array", "items": { "$ref": "#/$defs/metric" } }, + "body": { "$ref": "#/$defs/base64" } + } + }, + "value": { + "anyOf": [ + { "type": [ "string", "boolean" ] }, + { "$ref": "#/$defs/number" }, + { "type": "array" }, + { "$ref": "#/$defs/dataset" }, + { "$ref": "#/$defs/template" }, + { "$ref": "#/$defs/propertyset" }, + { "type": "array", "items": { "$ref": "#/$defs/propertyset" } } + ] + }, + "metric": { + "type": "object", + "properties": { + "name": { "type": "string" }, + "alias": { "$ref": "#/$defs/uint64" }, + "timestamp": { "$ref": "#/$defs/uint64" }, + "datatype": { "$ref": "#/$defs/datatype" }, + "quality": { "type": "integer", "minimum": 0 }, + "quality_context": { "type": "string" }, + "source": { "type": "string" }, + "is_historical": { "type": "boolean" }, + "is_transient": { "type": "boolean" }, + "is_null": { "type": "boolean" }, + "metadata": { "$ref": "#/$defs/metadata" }, + "properties": { "$ref": "#/$defs/propertyset" }, + "value": { "$ref": "#/$defs/value" }, + "command": { "$ref": "#/$defs/command" }, + "operation": { "enum": [ "ADD", "UPDATE", "DELETE", 1, 2, 3 ] } + } + }, + "propertyset": { + "type": "object", + "additionalProperties": { "$ref": "#/$defs/propertyvalue" } + }, + "propertyvalue": { + "type": "object", + "properties": { + "datatype": { "$ref": "#/$defs/datatype" }, + "is_null": { "type": "boolean" }, + "value": { "$ref": "#/$defs/value" } + } + }, + "metadata": { + "type": "object", + "properties": { + "is_multi_part": { "type": "boolean" }, + "content_type": { "type": "string" }, + "size": { "$ref": "#/$defs/uint64" }, + "seq": { "$ref": "#/$defs/uint64" }, + "file_name": { "type": "string" }, + "file_type": { "type": "string" }, + "md5": { "type": "string" }, + "description": { "type": "string" }, + "engUnit": { "type": "string" } + } + }, + "dataset": { + "type": "object", + "properties": { + "columns": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { "type": "string" }, + "datatype": { "$ref": "#/$defs/datatype" }, + "values": { "type": "array" } + } + } + } + } + }, + "template": { + "type": "object", + "properties": { + "version": { "type": "string" }, + "template_ref": { "type": "string" }, + "is_definition": { "type": "boolean" }, + "metrics": { "type": "array", "items": { "$ref": "#/$defs/metric" } }, + "parameters": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { "type": "string" }, + "type": { "$ref": "#/$defs/datatype" }, + "value": { "$ref": "#/$defs/value" } + } + } + } + } + }, + "command": { + "type": "object", + "properties": { + "arguments": { "type": "array", "items": { "$ref": "#/$defs/metric" } }, + "results": { "type": "array", "items": { "$ref": "#/$defs/metric" } }, + "correlation_id": { "type": "string" }, + "status": { + "type": "object", + "properties": { + "code": { "$ref": "#/$defs/uint64" }, + "message": { "type": "string" } + } + } + } + } + } +} +---- + +[[payloads_json_example]] +==== JSON Payload Example + +The following is a non-normative example of one DDATA message shown in both encodings, so the two +may be compared directly. The payload reports a single aliased Float metric. + +The spCv4.0 form, shown as a structural illustration of the Protocol Buffers message: + +---- +Acme/Dublin/Packaging/spCv4.0/DDATA/EdgeNode1/Line3 +---- + +---- +{ + "header": { + "timestamp": 1486144502122847913, + "seq": 4, + "bd_seq": 0, + "descriptor_seq": 2, + "definition_id": "a1c9...77" + }, + "content": { + "metrics": [{ + "alias": 1, + "datatype": "Float", + "value": 1802.5 + }] + } +} +---- + +The same message on the spJv4.0 namespace. The payload below is the literal UTF-8 text published as +the MQTT message payload, with the 64-bit fields represented as strings: + +---- +Acme/Dublin/Packaging/spJv4.0/DDATA/EdgeNode1/Line3 +---- + +---- +{ + "header": { + "timestamp": "1486144502122847913", + "seq": "4", + "bd_seq": "0", + "descriptor_seq": "2", + "definition_id": "a1c9...77" + }, + "content": { + "metrics": [{ + "alias": "1", + "datatype": "Float", + "value": 1802.5 + }] + } +} +---- + +Non-normative comment: The examples given throughout the rest of this chapter use the same JSON +notation to illustrate the structure of Protocol Buffers messages, and predate the JSON encoding. +They show field names and nesting rather than the exact text of a published spJv4.0 payload, and in +particular they write 64-bit values as JSON numbers. A published spJv4.0 payload follows the rules +in this section. + +[[payloads_json_choosing]] +==== Choosing a Payload Encoding + +Non-normative comment: The two encodings are interchangeable in terms of the information they +carry, so the choice between them is made on operational grounds. The Protocol Buffers encoding +produces smaller messages and is cheaper to encode and decode. The JSON encoding is readable +without tooling and can be consumed by any application that has a JSON parser, which is most of +them. + +Non-normative comment: The size difference is largest on small messages and narrows as messages +grow, because a larger message amortizes the per-message overhead over more metrics while the +per-metric overhead of JSON keys stays proportional. For representative payloads the JSON encoding +is roughly two to three times the size of the Protocol Buffers encoding. The difference in +encode and decode cost is of roughly one order of magnitude, and unlike the size difference it does +not narrow with message size. + +Non-normative comment: The size difference largely disappears when the content section is +compressed, as described in the link:#payloads_c_compression[Compression] section. The repeated +field names that make JSON large are highly compressible, so for payloads of more than a few +metrics the two encodings compress to within a few percent of one another. On a bandwidth +constrained link where compression is already in use, message size is therefore not a strong reason +to prefer one encoding over the other; encode and decode cost remains one. + +Non-normative comment: Taken together, the Protocol Buffers encoding is the better fit for Edge +Nodes on constrained hardware, on metered or low bandwidth links, and wherever messages are small +and frequent, which is the case Sparkplug was originally designed around. The JSON encoding is the +better fit where Sparkplug data is consumed directly by applications that cannot readily add a +Protocol Buffers dependency, and where readability during development and troubleshooting is worth +more than the bytes. A deployment may use both, with a bridging application republishing one +namespace onto the other, so that constrained Edge Nodes publish spCv4.0 while enterprise consumers +subscribe to spJv4.0. From 16855090742a0357aac9e8d8a70cb3f34708bf39 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Wed, 21 Oct 2026 17:08:24 +0000 Subject: [PATCH 02/15] [#231] Correct the sequence number rationale for string-encoded 64-bit values The note claimed a Sparkplug sequence number is always small, which is wrong: seq, bd_seq, and descriptor_seq each count to the maximum value of their declared type before rolling over to zero. They are therefore subject to the same precision loss as timestamps, and a numeric form would be correct early in a session and lossy late in one. --- .../main/asciidoc/chapters/Sparkplug_6_Payloads.adoc | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index b294f6c5..2577d9fa 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -3599,10 +3599,15 @@ is therefore roughly 165 times larger than that limit, so a timestamp written as would be silently rounded, typically to the nearest 256 nanoseconds. Metric values of type Int64, UInt64, and DateTime have the same exposure. +Non-normative comment: The sequence numbers are subject to this for the same reason. A Sparkplug +seq, bd_seq, and descriptor_seq each count up to the maximum value of their declared type before +rolling over to zero, as required in the link:#payloads_c_payload_header[Header] section, so they +reach magnitudes a JSON number cannot carry exactly. A sequence number is small early in a session +and large late in one, which would make a numeric form correct in testing and lossy in production. + Non-normative comment: The rule is driven by the declared type of the field rather than by the -magnitude of the value, so seq, bd_seq, and descriptor_seq are written as strings even though a -Sparkplug sequence number is always small. This keeps the mapping mechanical: an encoder derives -it from the schema alone, and no value can lose precision because of how large it happened to be. +magnitude of a particular value, so an encoder derives it from the schema alone and no value can +lose precision because of how large it happened to be. [[payloads_json_structural]] ==== Structural Mappings From 999e5eadae455c9dcaafca0f6f73d83423530139 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Wed, 21 Oct 2026 17:51:06 +0000 Subject: [PATCH 03/15] [#231] Make seq, bd_seq, and descriptor_seq int64 The three message sequence numbers move from uint64 to int64 in the Protocol Buffers schema, with the range and rollover point changing from 18446744073709551615 to 9223372036854775807 across all 38 statements that bound them in Chapters 4, 5, and 6. Protobuf uint64 is a signed long in Java, so the upper half of the uint64 range was never reachable by an implementation built on it. Declaring int64 makes the specified range the representable one. MetaData.seq, which sequences the parts of a multi-part file transfer, is a different field and is unchanged. --- .../asciidoc/chapters/Sparkplug_4_Topics.adoc | 22 +++--- .../Sparkplug_5_Operational_Behavior.adoc | 20 +++--- .../chapters/Sparkplug_6_Payloads.adoc | 69 +++++++++---------- 3 files changed, 55 insertions(+), 56 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc index 3d3925de..22d48a64 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc @@ -330,7 +330,7 @@ definitions MUST be published in the NMETA.*# include a sequence number. When the NMETA is published at the start of a BIRTH sequence it is the first message and its sequence number MUST be 0. When the NMETA is published as an update (accompanying an NMMETA), its sequence number MUST be one greater than the sequence number of the -previous message from this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the +previous message from this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-nmeta-bdseq]#[yellow-background]*[tck-id-topics-nmeta-bdseq] The NMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -374,7 +374,7 @@ NMMETA MUST include only the metrics affected by the change, and each metric MUS of ADD, UPDATE, or DELETE.*# * [tck-testable tck-id-topics-nmmeta-seq]#[yellow-background]*[tck-id-topics-nmmeta-seq] The NMMETA MUST include a sequence number that is one greater than the sequence number of the previous message -published by this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the sequence +published by this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-nmmeta-bdseq]#[yellow-background]*[tck-id-topics-nmmeta-bdseq] The NMMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -439,7 +439,7 @@ immediately prior MQTT CONNECT packet's Will Message.*# matching bd_seq number allows Host Applications to correlate NBIRTHs to NDEATHs.*# * [tck-testable tck-id-topics-nbirth-bdseq-increment]#[yellow-background]*[tck-id-topics-nbirth-bdseq-increment] The bd_seq number MUST start at zero and increment by one on every new MQTT CONNECT packet until it -reaches a maximum of 18446744073709551615 (max value of a UINT64). At this point, the following +reaches a maximum of 9223372036854775807 (max value of an INT64). At this point, the following bd_seq number MUST be zero.*# The NBIRTH message can also include additional Node Control payload components. These are used by a @@ -534,7 +534,7 @@ payload.*# ** This allows Host Applications to correlate NREBIRTHs to NDEATHs. * [tck-testable tck-id-topics-nrebirth-bdseq-increment]#[yellow-background]*[tck-id-topics-nrebirth-bdseq-increment] The bd_seq number MUST start at zero and increment by one on every new MQTT CONNECT packet until it -reaches a maximum of 18446744073709551615 (max value of a UINT64). At this point, the following +reaches a maximum of 9223372036854775807 (max value of an INT64). At this point, the following bd_seq number MUST be zero.*# The NREBIRTH message can also include additional Node Control payload components. These are used by @@ -594,7 +594,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-ndata-seq-num]#[yellow-background]*[tck-id-topics-ndata-seq-num] The NDATA MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-ndata-timestamp-1]#[yellow-background]*[tck-id-topics-ndata-timestamp-1] The NDATA MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -756,7 +756,7 @@ metadata MAY also be included. Aliases are optional.*# DMETA MUST NOT be used to convey current metric values.*# * [tck-testable tck-id-topics-dmeta-seq]#[yellow-background]*[tck-id-topics-dmeta-seq] The DMETA MUST include a sequence number that is one greater than the sequence number of the previous message from -this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the sequence number MUST +this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the sequence number MUST be 0. A DMETA is never the first message of a BIRTH sequence, as it follows the NMETA and NBIRTH.*# * [tck-testable tck-id-topics-dmeta-bdseq]#[yellow-background]*[tck-id-topics-dmeta-bdseq] The DMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -798,7 +798,7 @@ DMMETA MUST include only the metrics affected by the change, and each metric MUS of ADD, UPDATE, or DELETE.*# * [tck-testable tck-id-topics-dmmeta-seq]#[yellow-background]*[tck-id-topics-dmmeta-seq] The DMMETA MUST include a sequence number that is one greater than the sequence number of the previous message -published by this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the sequence +published by this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-dmmeta-bdseq]#[yellow-background]*[tck-id-topics-dmmeta-bdseq] The DMMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -840,7 +840,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-dbirth-seq]#[yellow-background]*[tck-id-topics-dbirth-seq] The DBIRTH MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-dbirth-timestamp-1]#[yellow-background]*[tck-id-topics-dbirth-timestamp-1] The DBIRTH MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -928,7 +928,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-drebirth-seq]#[yellow-background]*[tck-id-topics-drebirth-seq] The DREBIRTH MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-drebirth-timestamp-1]#[yellow-background]*[tck-id-topics-drebirth-timestamp-1] The DREBIRTH MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -1006,7 +1006,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-ddata-seq-num]#[yellow-background]*[tck-id-topics-ddata-seq-num] The DDATA MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-ddata-timestamp-1]#[yellow-background]*[tck-id-topics-ddata-timestamp-1] The DDATA MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -1052,7 +1052,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-ddeath-seq-num]#[yellow-background]*[tck-id-topics-ddeath-seq-num] The DDEATH MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-ddeath-bdseq]#[yellow-background]*[tck-id-topics-ddeath-bdseq] The DDEATH MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc index 5bd893de..1e55aab2 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc @@ -475,8 +475,8 @@ namespace on which it is published.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-will-message-payload-bdSeq]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-will-message-payload-bdSeq] The Edge Node's MQTT Will Message's payload MUST include a bd_seq number in the payload and the value MUST be incremented by one from the value in the previous MQTT CONNECT packet unless the value would -be greater than 18446744073709551615 (max value of a UINT64). If in the previous NBIRTH a value of -18446744073709551615 was sent, the next MQTT Connect packet Will Message payload bd_seq number value +be greater than 9223372036854775807 (max value of an INT64). If in the previous NBIRTH a value of +9223372036854775807 was sent, the next MQTT Connect packet Will Message payload bd_seq number value MUST have a value of 0.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-will-message-qos]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-will-message-qos] The Edge Node's MQTT Will Message's MQTT QoS MUST be 1.*# @@ -577,11 +577,11 @@ Edge Node's NBIRTH MQTT QoS MUST be 0.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-nbirth-retained]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-nbirth-retained] The Edge Node's NBIRTH retained flag MUST be set to false.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-nbirth-payload-seq]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-nbirth-payload-seq] The -Edge Node's NBIRTH payload MUST include a 'seq' number that is between 0 and 18446744073709551615 +Edge Node's NBIRTH payload MUST include a 'seq' number that is between 0 and 9223372036854775807 (inclusive).*# ** This will become the starting sequence number which all following messages will include a -sequence number that is one more than the previous up to 18446744073709551615 (max value of a -UINT64) where it wraps back to zero. +sequence number that is one more than the previous up to 9223372036854775807 (max value of an +INT64) where it wraps back to zero. * [tck-testable tck-id-message-flow-edge-node-birth-publish-phid-offline]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-phid-offline] If the Edge Node is configured to wait for a Primary Host Application, it is connected to the MQTT Server, and receives a STATE message on its configured Primary Host, the timestamp value in the @@ -860,7 +860,7 @@ Device's DBIRTH MQTT QoS MUST be 0.*# * [tck-testable tck-id-message-flow-device-birth-publish-dbirth-retained]#[yellow-background]*[tck-id-message-flow-device-birth-publish-dbirth-retained] The Device's DBIRTH retained flag MUST be set to false.*# * [tck-testable tck-id-message-flow-device-birth-publish-dbirth-payload-seq]#[yellow-background]*[tck-id-message-flow-device-birth-publish-dbirth-payload-seq] The -Device's DBIRTH payload MUST include a 'seq' number that is between 0 and 18446744073709551615 +Device's DBIRTH payload MUST include a 'seq' number that is between 0 and 9223372036854775807 (inclusive) and be one more than was included in the prior Sparkplug message sent from the Edge Node associated with this Device.*# @@ -1265,7 +1265,7 @@ values stop making sense. A Host Application also has to establish where in a descriptor sequence stream it is starting from. A BIRTH begins a stream and carries a descriptor_seq of 0. A REBIRTH does not begin a stream; it slip-streams the Host Application into a stream that is already running and carries the next -incrementing value, which is only 0 in the case of a UINT64 rollover. +incrementing value, which is only 0 in the case of an INT64 rollover. * [tck-testable tck-id-operational-behavior-host-partial-descriptor-seq-anchor]#[yellow-background]*[tck-id-operational-behavior-host-partial-descriptor-seq-anchor] A Host Application MUST anchor its expected descriptor_seq for a Sparkplug Descriptor on the @@ -1746,9 +1746,9 @@ Application getting in sync with the data stream. NREBIRTH payload MUST account for sequence number rollover if required based on the next pending DATA message and the number of devices under the scope of the Edge Node.*# ** For example, if the next pending DATA message sequence number will be 1 and the Edge Node has -two devices, the NBIRTHs sequence number value must be 18446744073709551614. This means the sequence -numbers will be: NBIRTH: 18446744073709551614, DBIRTH 1: 18446744073709551615, DBIRTH 2: 0, DATA: 1 -** The above example is based on the fact that the max value of a UINT64 is 18446744073709551615 +two devices, the NBIRTHs sequence number value must be 9223372036854775806. This means the sequence +numbers will be: NBIRTH: 9223372036854775806, DBIRTH 1: 9223372036854775807, DBIRTH 2: 0, DATA: 1 +** The above example is based on the fact that the max value of an INT64 is 9223372036854775807 * [tck-testable tck-id-operational-behavior-rebirth-action-6]#[yellow-background]*[tck-id-operational-behavior-rebirth-action-6] Each DREBIRTH payload MUST use a seq number that is incremented by one from the previous message.*# * [tck-testable tck-id-operational-behavior-rebirth-action-7]#[yellow-background]*[tck-id-operational-behavior-rebirth-action-7] Each diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 2577d9fa..00a907cd 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -327,9 +327,9 @@ message Payload { message Header { optional uint64 timestamp = 1; // Timestamp at message sending time - optional uint64 seq = 2; // Message sequence number - ordered stream of all messages from the Edge Node - optional uint64 bd_seq = 3; // The Birth/Death sequence number - scopes this message to the MQTT session it was published in - optional uint64 descriptor_seq = 4; // Descriptor sequence number - ordered stream per Sparkplug Descriptor (the Edge Node Descriptor for Edge Node messages, or a Device Descriptor for Device messages) + optional int64 seq = 2; // Message sequence number - ordered stream of all messages from the Edge Node + optional int64 bd_seq = 3; // The Birth/Death sequence number - scopes this message to the MQTT session it was published in + optional int64 descriptor_seq = 4; // Descriptor sequence number - ordered stream per Sparkplug Descriptor (the Edge Node Descriptor for Edge Node messages, or a Device Descriptor for Device messages) optional string definition_id = 5; // Definition identifier: content hash of the metric definitions this message reports against optional string previous_definition_id = 6; // NMMETA/DMMETA only: the definition_id in effect before the change optional string source = 7; // Source for CMD and REBIRTH messages - the Host Application ID that sent the message @@ -517,11 +517,11 @@ Node does. This allows Host Applications to easily distinguish an NDEATH deliver from an NDEATH published by the Edge Node itself. **** [tck-testable tck-id-payloads-sequence-num-req-nbirth]#[yellow-background]*[tck-id-payloads-sequence-num-req-nbirth] A NBIRTH message from an Edge Node MUST always contain a sequence number between 0 and -18446744073709551615 (inclusive).*# +9223372036854775807 (inclusive).*# **** [tck-testable tck-id-payloads-sequence-num-incrementing]#[yellow-background]*[tck-id-payloads-sequence-num-incrementing] All subsequent messages after an NBIRTH from an Edge Node MUST contain a sequence number that is continually increasing by one in each message from that Edge Node until a value of -18446744073709551615 (max value of a UINT64) is reached. At that point, the sequence number of the +9223372036854775807 (max value of an INT64) is reached. At that point, the sequence number of the following message MUST be zero.*# ** _bd_seq_ *** This is the birth/death sequence number which is an unsigned 64-bit integer. It scopes a message to @@ -545,7 +545,7 @@ that Host Application and is unrelated to the bd_seq number of any Edge Node. Se link:#payloads_c_state_bd_seq[STATE bd_seq Number] section. **** [tck-testable tck-id-payloads-bd-sequence-num-req-nbirth]#[yellow-background]*[tck-id-payloads-bd-sequence-num-req-nbirth] Every birth/death sequence number included in the payload from an Edge Node MUST always be between 0 and -18446744073709551615 (inclusive).*# +9223372036854775807 (inclusive).*# **** [tck-testable tck-id-payloads-bd-sequence-num-session-constant]#[yellow-background]*[tck-id-payloads-bd-sequence-num-session-constant] Every message an Edge Node publishes within a single MQTT session MUST carry the same birth/death sequence number, and that value MUST be the one registered in the Will Message payload of that session's MQTT @@ -555,7 +555,7 @@ message from an Edge Node can be attributed to the session that produced it, and discard a message belonging to a session it has already replaced. **** [tck-testable tck-id-payloads-bd-sequence-num-incrementing]#[yellow-background]*[tck-id-payloads-bd-sequence-num-incrementing] The birth/death sequence number MUST start at zero and MUST increment by one on every new MQTT CONNECT -packet sent by the Edge Node, until a value of 18446744073709551615 (max value of a UINT64) is reached. +packet sent by the Edge Node, until a value of 9223372036854775807 (max value of an INT64) is reached. At that point the birth/death sequence number of the following MQTT CONNECT packet MUST be zero.*# ***** Non-normative comment: the bd_seq number is not incremented from message to message within a session, and it is not incremented by an NREBIRTH, because an NREBIRTH does not establish a new MQTT @@ -584,7 +584,7 @@ sequence (see the Reconnect Optimization), the NBIRTH or DBIRTH is the first mes **** [tck-testable tck-id-payloads-descriptor-seq-incrementing]#[yellow-background]*[tck-id-payloads-descriptor-seq-incrementing] Each subsequent message in a descriptor sequence stream MUST contain a descriptor sequence number that is one greater than the descriptor sequence number of the previous message in that same stream, until a -value of 18446744073709551615 (max value of a UINT64) is reached, after which the next descriptor +value of 9223372036854775807 (max value of an INT64) is reached, after which the next descriptor sequence number MUST be 0.*# **** [tck-testable tck-id-payloads-descriptor-seq-rebirth]#[yellow-background]*[tck-id-payloads-descriptor-seq-rebirth] An NREBIRTH or DREBIRTH slip-streams a Host Application into the existing data stream and MUST NOT reset @@ -2109,11 +2109,11 @@ alias was assigned to it in the referenced NMETA, or otherwise by its metric nam NOT be used to convey metric datatype, properties, or metadata.*# * [tck-testable tck-id-payloads-nrebirth-seq-1]#[yellow-background]*[tck-id-payloads-nrebirth-seq-1] Every NREBIRTH message MUST include a sequence number and it MUST have a value between 0 and -18446744073709551615 (inclusive).*# +9223372036854775807 (inclusive).*# * [tck-testable tck-id-payloads-nrebirth-seq-2]#[yellow-background]*[tck-id-payloads-nrebirth-seq-2] Every NREBIRTH message MUST include a sequence number that is the following value:*# ** rsn = pds - (1 + dc) -** If the rsn results in a negative number, add 18446744073709551615 (max value of a UINT64) to it. +** If the rsn results in a negative number, add 9223372036854775807 (max value of an INT64) to it. *** rsn = Rebirth Sequence Number value *** pds = Next pending DATA publish seq number *** dc = Device count (number of Sparkplug Devices under the scope of the Edge Node) @@ -2230,8 +2230,8 @@ NOT be used to convey metric datatype, properties, or metadata.*# DBIRTH message MUST include a sequence number.*# * [tck-testable tck-id-payloads-dbirth-seq-inc]#[yellow-background]*[tck-id-payloads-dbirth-seq-inc] Every DBIRTH message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 18446744073709551615 (max value of a -UINT64). If in the previous sequence number sent by the Edge Node was 18446744073709551615, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an +INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# * [tck-testable tck-id-payloads-dbirth-order]#[yellow-background]*[tck-id-payloads-dbirth-order] All DBIRTH messages sent by an Edge Node MUST be sent immediately after the NBIRTH and before any NDATA @@ -2333,8 +2333,8 @@ NOT be used to convey metric datatype, properties, or metadata.*# DREBIRTH message MUST include a sequence number.*# * [tck-testable tck-id-payloads-drebirth-seq-inc]#[yellow-background]*[tck-id-payloads-drebirth-seq-inc] Every DREBIRTH message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 18446744073709551615 (max value of a -UINT64). If in the previous sequence number sent by the Edge Node was 18446744073709551615, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an +INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST be a value of 0.*# * [tck-testable tck-id-payloads-drebirth-order]#[yellow-background]*[tck-id-payloads-drebirth-order] All DREBIRTH messages sent by an Edge Node MUST be sent immediately after the NREBIRTH and before any @@ -2433,8 +2433,8 @@ published.*# NDATA message MUST include a sequence number.*# * [tck-testable tck-id-payloads-ndata-seq-inc]#[yellow-background]*[tck-id-payloads-ndata-seq-inc] Every NDATA message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 18446744073709551615 (max value of a -UINT64). If in the previous sequence number sent by the Edge Node was 18446744073709551615, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an +INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# * [tck-testable tck-id-payloads-ndata-order]#[yellow-background]*[tck-id-payloads-ndata-order] All NDATA messages sent by an Edge Node MUST NOT be sent until all the NBIRTH and all DBIRTH messages @@ -2500,8 +2500,8 @@ published.*# DDATA message MUST include a sequence number.*# * [tck-testable tck-id-payloads-ddata-seq-inc]#[yellow-background]*[tck-id-payloads-ddata-seq-inc] Every DDATA message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 18446744073709551615 (max value of a -UINT64). If in the previous sequence number sent by the Edge Node was 18446744073709551615, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an +INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# * [tck-testable tck-id-payloads-ddata-order]#[yellow-background]*[tck-id-payloads-ddata-order] All DDATA messages sent by an Edge Node MUST NOT be sent until all the NBIRTH and all DBIRTH messages @@ -2903,7 +2903,7 @@ message. payload compared to what was in the previous CONNECT packet. This ensures that any Host Applications will be able to distinguish between current and old bd_seq numbers in the event that messages are delivered out of order. When incrementing the bd_seq number, if the previous value was -18446744073709551615 (max value of a UINT64), the next value must be zero. +9223372036854775807 (max value of an INT64), the next value must be zero. * [tck-testable tck-id-payloads-ndeath-will-message-publisher]#[yellow-background]*[tck-id-payloads-ndeath-will-message-publisher] An NDEATH message SHOULD be published by the Edge Node before it intentionally disconnects from the MQTT Server.*# @@ -2971,8 +2971,8 @@ published.*# DDEATH message MUST include a sequence number.*# * [tck-testable tck-id-payloads-ddeath-seq-inc]#[yellow-background]*[tck-id-payloads-ddeath-seq-inc] Every DDEATH message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 18446744073709551615 (max value of a -UINT64). If in the previous sequence number sent by the Edge Node was 18446744073709551615, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an +INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# The following is a representation of a simple DDEATH message on the topic: @@ -3061,15 +3061,15 @@ Certificate to be matched to the Birth Certificate of the session it terminates. bd_seq number MUST be included in every STATE message published by or on behalf of a Sparkplug Host Application, including the Death Certificate registered as the MQTT Will Message.*# * [tck-testable tck-id-payloads-state-bdseq-range]#[yellow-background]*[tck-id-payloads-state-bdseq-range] The -bd_seq number in a STATE payload MUST always be between 0 and 18446744073709551615 (inclusive).*# +bd_seq number in a STATE payload MUST always be between 0 and 9223372036854775807 (inclusive).*# * [tck-testable tck-id-payloads-state-bdseq-session-constant]#[yellow-background]*[tck-id-payloads-state-bdseq-session-constant] Every STATE message a Sparkplug Host Application publishes within a single MQTT session MUST carry the same bd_seq number, and that value MUST be the one registered in the Will Message payload of that session's MQTT CONNECT packet.*# * [tck-testable tck-id-payloads-state-bdseq-increment]#[yellow-background]*[tck-id-payloads-state-bdseq-increment] The bd_seq number MUST start at zero and MUST increment by one on every new MQTT CONNECT packet the -Sparkplug Host Application sends to a given MQTT Server, until a value of 18446744073709551615 (max -value of a UINT64) is reached. At that point the bd_seq number of the following MQTT CONNECT packet +Sparkplug Host Application sends to a given MQTT Server, until a value of 9223372036854775807 (max +value of an INT64) is reached. At that point the bd_seq number of the following MQTT CONNECT packet MUST be zero.*# ** Non-normative comment: these are the same rules that apply to an Edge Node's bd_seq number. As with an Edge Node, the bd_seq number is not incremented from message to message within a session. @@ -3725,8 +3725,7 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# }, "not": { "required": [ "content", "compressed" ] }, "$defs": { - "uint64": { "type": [ "string", "integer" ], "pattern": "^[0-9]+$" }, - "int64": { "type": [ "string", "integer" ], "pattern": "^-?[0-9]+$" }, + "nonNegative64": { "type": [ "string", "integer" ], "pattern": "^[0-9]+$" }, "base64": { "type": "string", "contentEncoding": "base64" }, "number": { "oneOf": [ @@ -3743,10 +3742,10 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "header": { "type": "object", "properties": { - "timestamp": { "$ref": "#/$defs/uint64" }, - "seq": { "$ref": "#/$defs/uint64" }, - "bd_seq": { "$ref": "#/$defs/uint64" }, - "descriptor_seq": { "$ref": "#/$defs/uint64" }, + "timestamp": { "$ref": "#/$defs/nonNegative64" }, + "seq": { "$ref": "#/$defs/nonNegative64" }, + "bd_seq": { "$ref": "#/$defs/nonNegative64" }, + "descriptor_seq": { "$ref": "#/$defs/nonNegative64" }, "definition_id": { "type": "string" }, "previous_definition_id": { "type": "string" }, "source": { "type": "string" }, @@ -3776,8 +3775,8 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "type": "object", "properties": { "name": { "type": "string" }, - "alias": { "$ref": "#/$defs/uint64" }, - "timestamp": { "$ref": "#/$defs/uint64" }, + "alias": { "$ref": "#/$defs/nonNegative64" }, + "timestamp": { "$ref": "#/$defs/nonNegative64" }, "datatype": { "$ref": "#/$defs/datatype" }, "quality": { "type": "integer", "minimum": 0 }, "quality_context": { "type": "string" }, @@ -3809,8 +3808,8 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "properties": { "is_multi_part": { "type": "boolean" }, "content_type": { "type": "string" }, - "size": { "$ref": "#/$defs/uint64" }, - "seq": { "$ref": "#/$defs/uint64" }, + "size": { "$ref": "#/$defs/nonNegative64" }, + "seq": { "$ref": "#/$defs/nonNegative64" }, "file_name": { "type": "string" }, "file_type": { "type": "string" }, "md5": { "type": "string" }, @@ -3863,7 +3862,7 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "status": { "type": "object", "properties": { - "code": { "$ref": "#/$defs/uint64" }, + "code": { "$ref": "#/$defs/nonNegative64" }, "message": { "type": "string" } } } From 1b39c7248af24a6f1e858942b18aaf9f9e9e58c8 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Wed, 21 Oct 2026 18:37:57 +0000 Subject: [PATCH 04/15] [#231] Make timestamp int64 and correct the NREBIRTH rsn arithmetic Header.timestamp and Metric.timestamp move from uint64 to int64, for the same reason as the sequence numbers: protobuf uint64 is a signed long in Java, so the upper half of the unsigned range was never reachable. The range note in Chapter 5 is rewritten accordingly - the ceiling is 2262-04-11, stated directly rather than as a caveat on an unsigned range that no implementation could use. The NREBIRTH rsn calculation said to add the maximum sequence number to a negative result. That was off by one, since a counter cycling 0..MAX has a period of MAX+1, and MAX+1 is not representable as an int64. Restated as two cases so no intermediate is ever negative or out of range: if pds >= 1 + dc: rsn = pds - (1 + dc) otherwise: rsn = 9223372036854775807 - (dc - pds) Verified equivalent to (pds - (1 + dc)) mod 2^63 over the full range. --- .../chapters/Sparkplug_5_Operational_Behavior.adoc | 12 ++++++------ .../asciidoc/chapters/Sparkplug_6_Payloads.adoc | 13 +++++++------ 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc index 1e55aab2..f2604bf9 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc @@ -66,12 +66,12 @@ an implementation to have a nanosecond resolution time source. An implementation is coarser than one nanosecond expresses its timestamps in nanoseconds all the same, scaled to that denomination. For example, an implementation whose time source has a resolution of one millisecond expresses a timestamp as the number of milliseconds since epoch multiplied by 1000000. -** Non-normative comment: an unsigned 64-bit integer of nanoseconds since epoch covers a range of -approximately 584 years, reaching the year 2554. Implementers should note that this range is only -available to a genuinely unsigned representation. The Google Protocol Buffers uint64 type is -represented as a signed 64-bit long in Java, and several widely used nanosecond time representations -are signed 64-bit as well, which reduces the reachable range to approximately 292 years, ending on -2262-04-11. This is the practical ceiling for such implementations. +** Non-normative comment: a signed 64-bit integer of nanoseconds since epoch covers a range of +approximately 292 years, ending on 2262-04-11, which is the ceiling for a Sparkplug timestamp. A +signed type is specified rather than an unsigned one because the Google Protocol Buffers uint64 +type is represented as a signed 64-bit long in Java, and several widely used nanosecond time +representations are signed 64-bit as well, so the upper half of an unsigned range was never +reachable in practice. Non-normative comment: the resolution of a timestamp is not the same as its accuracy. Denominating a timestamp in nanoseconds says how finely a value can be expressed, not how closely it corresponds to diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 00a907cd..f9ae23a4 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -309,7 +309,7 @@ message Payload { optional string name = 1; // Metric name - Required in BIRTH messages, optional in DATA messages if using aliases optional uint64 alias = 2; // Metric alias - tied to name on BIRTH and included in all later DATA messages - optional uint64 timestamp = 3; // Metric value change time - the time this metric's value last changed - if omitted, the overall payload timestamp is assumed as the metric value change time + optional int64 timestamp = 3; // Metric value change time - the time this metric's value last changed - if omitted, the overall payload timestamp is assumed as the metric value change time optional DataType datatype = 4; // DataType of the metric/tag value optional uint32 quality = 5; // Quality code of the metric/tag value - severity in bits 30-31 optional string quality_context = 6; // The optional quality context of the metric/tag value @@ -326,7 +326,7 @@ message Payload { } message Header { - optional uint64 timestamp = 1; // Timestamp at message sending time + optional int64 timestamp = 1; // Timestamp at message sending time optional int64 seq = 2; // Message sequence number - ordered stream of all messages from the Edge Node optional int64 bd_seq = 3; // The Birth/Death sequence number - scopes this message to the MQTT session it was published in optional int64 descriptor_seq = 4; // Descriptor sequence number - ordered stream per Sparkplug Descriptor (the Edge Node Descriptor for Edge Node messages, or a Device Descriptor for Device messages) @@ -489,7 +489,7 @@ The header section includes the following components. * *header* ** _timestamp_ -*** This is the timestamp in the form of an unsigned 64-bit integer representing the number of +*** This is the timestamp in the form of a signed 64-bit integer representing the number of nanoseconds since epoch (Jan 1, 1970). *** [tck-testable tck-id-payloads-timestamp-in-UTC]#[yellow-background]*[tck-id-payloads-timestamp-in-UTC] This timestamp MUST be in UTC.*# @@ -666,7 +666,7 @@ and DBIRTH messages MUST include both a metric name and alias.*# *** [tck-testable tck-id-payloads-alias-data-cmd-requirement]#[yellow-background]*[tck-id-payloads-alias-data-cmd-requirement] NDATA, DDATA, NCMD, and DCMD messages MUST only include an alias and the metric name MUST be excluded.*# * *timestamp* -** This is the timestamp in the form of an unsigned 64-bit integer representing the number of +** This is the timestamp in the form of a signed 64-bit integer representing the number of nanoseconds since epoch (Jan 1, 1970). ** [tck-not-testable]#[yellow-background]*In an NBIRTH, NREBIRTH, DBIRTH, DREBIRTH, NDATA, or DDATA message, the timestamp of a Metric MUST represent the metric value change time, meaning the time at @@ -2112,8 +2112,9 @@ NREBIRTH message MUST include a sequence number and it MUST have a value between 9223372036854775807 (inclusive).*# * [tck-testable tck-id-payloads-nrebirth-seq-2]#[yellow-background]*[tck-id-payloads-nrebirth-seq-2] Every NREBIRTH message MUST include a sequence number that is the following value:*# -** rsn = pds - (1 + dc) -** If the rsn results in a negative number, add 9223372036854775807 (max value of an INT64) to it. +** If pds is greater than or equal to (1 + dc), then rsn = pds - (1 + dc) +** Otherwise the sequence number has rolled over since that pending DATA publish, and +rsn = 9223372036854775807 - (dc - pds) *** rsn = Rebirth Sequence Number value *** pds = Next pending DATA publish seq number *** dc = Device count (number of Sparkplug Devices under the scope of the Edge Node) From ea2cc004c3631bbf71d9f378a8c6bfc9a49e5b6e Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Wed, 21 Oct 2026 20:13:30 +0000 Subject: [PATCH 05/15] [#231] Correct three field descriptions left describing seq as unsigned The seq, bd_seq, and descriptor_seq bullets in the Header section still described each field as an unsigned 64-bit integer after the schema moved them to int64. --- .../src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index f9ae23a4..31957812 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -502,7 +502,7 @@ timestamp is the point at which the reported values were current, and the timest is the time that metric's value last changed. A consuming application needs both to know what a value is and how long it has held that value. ** _seq_ -*** This is the sequence number which is an unsigned 64-bit integer. +*** This is the sequence number which is a signed 64-bit integer. **** [tck-testable tck-id-payloads-sequence-num-always-included]#[yellow-background]*[tck-id-payloads-sequence-num-always-included] A sequence number MUST be included in the payload of every Sparkplug MQTT message from an Edge Node except an NDEATH message that is delivered as the Edge Node's registered MQTT Will Message.*# @@ -524,7 +524,7 @@ continually increasing by one in each message from that Edge Node until a value 9223372036854775807 (max value of an INT64) is reached. At that point, the sequence number of the following message MUST be zero.*# ** _bd_seq_ -*** This is the birth/death sequence number which is an unsigned 64-bit integer. It scopes a message to +*** This is the birth/death sequence number which is a signed 64-bit integer. It scopes a message to the MQTT session in which it was published. Unlike seq and descriptor_seq, which advance from message to message, the bd_seq number is constant for the lifetime of an MQTT session and changes only when a new session is established. @@ -561,7 +561,7 @@ At that point the birth/death sequence number of the following MQTT CONNECT pack session, and it is not incremented by an NREBIRTH, because an NREBIRTH does not establish a new MQTT session. ** _descriptor_seq_ -*** This is the descriptor sequence number which is an unsigned 64-bit integer. It is an ordered +*** This is the descriptor sequence number which is a signed 64-bit integer. It is an ordered sequence number stream scoped to a single Sparkplug Descriptor (see the link:#introduction_sparkplug_ids[Sparkplug Identifiers] section), either the Edge Node Descriptor for an Edge Node's messages (NBIRTH, NDATA) or a Device Descriptor for a Device's messages (DBIRTH, From 8b35e0e9a8c245c8b546473ca6dc437ceaf1cd50 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Mon, 26 Oct 2026 15:24:59 +0000 Subject: [PATCH 06/15] [#231] Add payload encoding requirements to the conformance profiles An Edge Node MUST support at least one encoding and MAY support both, selected by configuration, but MUST use a single encoding and namespace for the whole of an MQTT session - seq is Edge-Node-wide and bd_seq scopes the session, so splitting one session across both namespaces would produce two unusable halves of one sequence stream. Host Applications and Sparkplug Aware MQTT Servers MUST support both encodings. A Host Application cannot choose the encoding of the Edge Nodes it consumes, and an Aware Server decodes payloads rather than treating them as opaque. A Sparkplug Compliant MQTT Server has no payload requirements at all, which is now stated non-normatively so the asymmetry with the Aware profile is not read as an omission. Also generalizes the Edge Node profile's reference to the spCv4.0/# namespace. --- .../chapters/Sparkplug_10_Conformance.adoc | 45 ++++++++++++++++++- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc index 5c3475f9..ddcb86e1 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc @@ -35,8 +35,25 @@ typically be both an Edge Node and a Host Application. ==== Sparkplug Edge Node A Sparkplug Edge Node is typically an 'Edge Gateway'. It sends and receives data to an MQTT Server -using the spCv4.0/# namespace. Edge Nodes typically interact with physical devices to gather data -and also write to device outputs. +using a Sparkplug namespace, either spCv4.0/# or spJv4.0/#. Edge Nodes typically interact with +physical devices to gather data and also write to device outputs. + +Sparkplug defines two payload encodings, as described in the +link:#payloads_json_encoding_selection[Encoding Selection] section. An Edge Node is not required to +implement both. + +* [tck-testable tck-id-conformance-edge-node-encoding-one]#[yellow-background]*[tck-id-conformance-edge-node-encoding-one] A +Sparkplug Edge Node MUST support at least one of the two Sparkplug payload encodings, and MUST +publish on the namespace that corresponds to the encoding it uses.*# +* [tck-not-testable tck-id-conformance-edge-node-encoding-both]#[yellow-background]*[tck-id-conformance-edge-node-encoding-both] A +Sparkplug Edge Node MAY support both payload encodings, in which case the encoding it uses is +selected by configuration.*# +* [tck-testable tck-id-conformance-edge-node-encoding-session]#[yellow-background]*[tck-id-conformance-edge-node-encoding-session] A +Sparkplug Edge Node MUST publish every message of an MQTT session using a single payload encoding on +the corresponding single namespace.*# +** Non-normative comment: the sequence number is Edge-Node-wide and the bd_seq number scopes an MQTT +session, so an Edge Node that split one session across both namespaces would produce two interleaved +halves of one sequence number stream and neither would be consumable on its own. Support for strongly typed commands is optional. An Edge Node that exposes commands must do so in a conformant manner. @@ -74,6 +91,20 @@ NREBIRTH, and NDEATH of every Edge Node reporting a Device of interest, which ca state that encapsulates the state of its Devices) and its use of the descriptor sequence number in place of the Edge-Node-wide sequence number. +A Sparkplug Host Application consumes data produced by Edge Nodes that it does not control, and so +cannot assume which payload encoding those Edge Nodes use. + +* [tck-testable tck-id-conformance-host-encoding-both]#[yellow-background]*[tck-id-conformance-host-encoding-both] A +Sparkplug Host Application MUST support both Sparkplug payload encodings, and MUST be able to +consume messages on both the spCv4.0 and spJv4.0 namespaces.*# +* [tck-testable tck-id-conformance-host-encoding-publish]#[yellow-background]*[tck-id-conformance-host-encoding-publish] When +a Sparkplug Host Application publishes a message, it MUST encode that message using the encoding +that corresponds to the namespace it publishes on.*# +** Non-normative comment: which namespaces a Host Application subscribes to remains a deployment +decision. The requirement is that it is capable of decoding either, so that an Edge Node's choice of +encoding does not determine which Host Applications can consume it. This is the property that makes +the two encodings interoperable rather than two separate ecosystems. + [tck-testable tck-id-conformance-primary-host]#[yellow-background]*[tck-id-conformance-primary-host] Sparkplug Host Applications MUST publish 'STATE' messages that represent its Birth and Death Certificates.*# @@ -116,6 +147,10 @@ Sparkplug conformant MQTT Server MUST support all aspects of Will Messages inclu * [tck-testable tck-id-conformance-mqtt-retained]#[yellow-background]*[tck-id-conformance-mqtt-retained] A Sparkplug conformant MQTT Server MUST support all aspects of the 'retain flag'*# +Non-normative comment: a Sparkplug Compliant MQTT Server has no payload requirements of any kind. It +treats a Sparkplug payload as an opaque sequence of bytes, so both payload encodings are carried by +the same server with no additional support. + [[conformance_sparkplug_aware_mqtt_server]] ==== Sparkplug Aware MQTT Server @@ -132,6 +167,12 @@ continue to be published with the MQTT retain flag set to false and are not reta * [tck-testable tck-id-conformance-mqtt-aware-basic]#[yellow-background]*[tck-id-conformance-mqtt-aware-basic] A Sparkplug Aware MQTT Server MUST support all aspects of a Sparkplug Compliant MQTT Server*# +* [tck-testable tck-id-conformance-mqtt-aware-encoding-both]#[yellow-background]*[tck-id-conformance-mqtt-aware-encoding-both] A +Sparkplug Aware MQTT Server MUST support both Sparkplug payload encodings on both the spCv4.0 and +spJv4.0 namespaces.*# +** Non-normative comment: unlike a Sparkplug Compliant MQTT Server, a Sparkplug Aware MQTT Server +decodes and re-encodes payloads in order to perform the behavior required of it below, so it cannot +treat a payload as opaque and must therefore implement both encodings. * [tck-testable tck-id-conformance-mqtt-aware-ndeath-timestamp]#[yellow-background]*[tck-id-conformance-mqtt-aware-ndeath-timestamp] A Sparkplug Aware MQTT Server MAY replace the timestmap of NDEATH messages. If it does, it MUST set the timestamp to the UTC time at which it attempts to deliver the NDEATH to subscribed clients*# From de911961c9d29bc62dc6a10fa693e395c23ec98e Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Mon, 26 Oct 2026 16:06:43 +0000 Subject: [PATCH 07/15] [#231] Return the sequence numbers to uint64, keeping the capped range seq, bd_seq, and descriptor_seq go back to uint64 while keeping the rollover point at 9223372036854775807. timestamp stays int64. An unsigned type is right for a counter that can never be negative, and it keeps the three consistent with alias, MetaData.size, and Status.code. Capping the range below the type's maximum still guarantees every value a conformant Edge Node can produce fits a signed 64-bit integer, so an implementation in a language without an unsigned 64-bit type never needs unsigned comparison. Neither bound is reachable: at a million messages per second the capped range lasts ~292,000 years. timestamp keeps int64 because every mainstream time representation it interoperates with is signed - protobuf's own Timestamp, OPC UA DateTime, java.time.Instant - and the 2262 ceiling it accepts is one none of those exceed either. The repeated "(max value of a UINT64)" parenthetical is dropped rather than rescaled, since the cap is now a deliberate restriction rather than the type's maximum. It is explained once in the Header section instead. --- .../asciidoc/chapters/Sparkplug_4_Topics.adoc | 26 +++++---- .../Sparkplug_5_Operational_Behavior.adoc | 9 +-- .../chapters/Sparkplug_6_Payloads.adoc | 57 +++++++++++-------- 3 files changed, 53 insertions(+), 39 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc index 22d48a64..2d5d0986 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc @@ -330,7 +330,8 @@ definitions MUST be published in the NMETA.*# include a sequence number. When the NMETA is published at the start of a BIRTH sequence it is the first message and its sequence number MUST be 0. When the NMETA is published as an update (accompanying an NMMETA), its sequence number MUST be one greater than the sequence number of the -previous message from this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the +previous message from this Edge Node, unless that message contained a value of 9223372036854775807, +in which case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-nmeta-bdseq]#[yellow-background]*[tck-id-topics-nmeta-bdseq] The NMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -374,7 +375,8 @@ NMMETA MUST include only the metrics affected by the change, and each metric MUS of ADD, UPDATE, or DELETE.*# * [tck-testable tck-id-topics-nmmeta-seq]#[yellow-background]*[tck-id-topics-nmmeta-seq] The NMMETA MUST include a sequence number that is one greater than the sequence number of the previous message -published by this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the sequence +published by this Edge Node, unless that message contained a value of 9223372036854775807, in which +case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-nmmeta-bdseq]#[yellow-background]*[tck-id-topics-nmmeta-bdseq] The NMMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -439,7 +441,7 @@ immediately prior MQTT CONNECT packet's Will Message.*# matching bd_seq number allows Host Applications to correlate NBIRTHs to NDEATHs.*# * [tck-testable tck-id-topics-nbirth-bdseq-increment]#[yellow-background]*[tck-id-topics-nbirth-bdseq-increment] The bd_seq number MUST start at zero and increment by one on every new MQTT CONNECT packet until it -reaches a maximum of 9223372036854775807 (max value of an INT64). At this point, the following +reaches a maximum of 9223372036854775807. At this point, the following bd_seq number MUST be zero.*# The NBIRTH message can also include additional Node Control payload components. These are used by a @@ -534,7 +536,7 @@ payload.*# ** This allows Host Applications to correlate NREBIRTHs to NDEATHs. * [tck-testable tck-id-topics-nrebirth-bdseq-increment]#[yellow-background]*[tck-id-topics-nrebirth-bdseq-increment] The bd_seq number MUST start at zero and increment by one on every new MQTT CONNECT packet until it -reaches a maximum of 9223372036854775807 (max value of an INT64). At this point, the following +reaches a maximum of 9223372036854775807. At this point, the following bd_seq number MUST be zero.*# The NREBIRTH message can also include additional Node Control payload components. These are used by @@ -594,7 +596,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-ndata-seq-num]#[yellow-background]*[tck-id-topics-ndata-seq-num] The NDATA MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807. In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-ndata-timestamp-1]#[yellow-background]*[tck-id-topics-ndata-timestamp-1] The NDATA MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -756,7 +758,8 @@ metadata MAY also be included. Aliases are optional.*# DMETA MUST NOT be used to convey current metric values.*# * [tck-testable tck-id-topics-dmeta-seq]#[yellow-background]*[tck-id-topics-dmeta-seq] The DMETA MUST include a sequence number that is one greater than the sequence number of the previous message from -this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the sequence number MUST +this Edge Node, unless that message contained a value of 9223372036854775807, in which case the +sequence number MUST be 0. A DMETA is never the first message of a BIRTH sequence, as it follows the NMETA and NBIRTH.*# * [tck-testable tck-id-topics-dmeta-bdseq]#[yellow-background]*[tck-id-topics-dmeta-bdseq] The DMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -798,7 +801,8 @@ DMMETA MUST include only the metrics affected by the change, and each metric MUS of ADD, UPDATE, or DELETE.*# * [tck-testable tck-id-topics-dmmeta-seq]#[yellow-background]*[tck-id-topics-dmmeta-seq] The DMMETA MUST include a sequence number that is one greater than the sequence number of the previous message -published by this Edge Node, unless that message contained a value of 9223372036854775807 (max value of an INT64), in which case the sequence +published by this Edge Node, unless that message contained a value of 9223372036854775807, in which +case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-dmmeta-bdseq]#[yellow-background]*[tck-id-topics-dmmeta-bdseq] The DMMETA MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# @@ -840,7 +844,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-dbirth-seq]#[yellow-background]*[tck-id-topics-dbirth-seq] The DBIRTH MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807. In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-dbirth-timestamp-1]#[yellow-background]*[tck-id-topics-dbirth-timestamp-1] The DBIRTH MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -928,7 +932,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-drebirth-seq]#[yellow-background]*[tck-id-topics-drebirth-seq] The DREBIRTH MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807. In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-drebirth-timestamp-1]#[yellow-background]*[tck-id-topics-drebirth-timestamp-1] The DREBIRTH MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -1006,7 +1010,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-ddata-seq-num]#[yellow-background]*[tck-id-topics-ddata-seq-num] The DDATA MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807. In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-ddata-timestamp-1]#[yellow-background]*[tck-id-topics-ddata-timestamp-1] The DDATA MUST include the overall payload timestamp denoting the date and time the message was sent from the Edge Node.*# @@ -1052,7 +1056,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# * [tck-testable tck-id-topics-ddeath-seq-num]#[yellow-background]*[tck-id-topics-ddeath-seq-num] The DDEATH MUST include a sequence number in the payload and it MUST have a value of one greater than the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a -value of 9223372036854775807 (max value of an INT64). In this case the sequence number MUST be 0.*# +value of 9223372036854775807. In this case the sequence number MUST be 0.*# * [tck-testable tck-id-topics-ddeath-bdseq]#[yellow-background]*[tck-id-topics-ddeath-bdseq] The DDEATH MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc index f2604bf9..6802639c 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc @@ -475,7 +475,7 @@ namespace on which it is published.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-will-message-payload-bdSeq]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-will-message-payload-bdSeq] The Edge Node's MQTT Will Message's payload MUST include a bd_seq number in the payload and the value MUST be incremented by one from the value in the previous MQTT CONNECT packet unless the value would -be greater than 9223372036854775807 (max value of an INT64). If in the previous NBIRTH a value of +be greater than 9223372036854775807. If in the previous NBIRTH a value of 9223372036854775807 was sent, the next MQTT Connect packet Will Message payload bd_seq number value MUST have a value of 0.*# * [tck-testable tck-id-message-flow-edge-node-birth-publish-will-message-qos]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-will-message-qos] The @@ -580,8 +580,8 @@ Edge Node's NBIRTH retained flag MUST be set to false.*# Edge Node's NBIRTH payload MUST include a 'seq' number that is between 0 and 9223372036854775807 (inclusive).*# ** This will become the starting sequence number which all following messages will include a -sequence number that is one more than the previous up to 9223372036854775807 (max value of an -INT64) where it wraps back to zero. +sequence number that is one more than the previous up to 9223372036854775807 where it wraps back to +zero. * [tck-testable tck-id-message-flow-edge-node-birth-publish-phid-offline]#[yellow-background]*[tck-id-message-flow-edge-node-birth-publish-phid-offline] If the Edge Node is configured to wait for a Primary Host Application, it is connected to the MQTT Server, and receives a STATE message on its configured Primary Host, the timestamp value in the @@ -1748,7 +1748,8 @@ DATA message and the number of devices under the scope of the Edge Node.*# ** For example, if the next pending DATA message sequence number will be 1 and the Edge Node has two devices, the NBIRTHs sequence number value must be 9223372036854775806. This means the sequence numbers will be: NBIRTH: 9223372036854775806, DBIRTH 1: 9223372036854775807, DBIRTH 2: 0, DATA: 1 -** The above example is based on the fact that the max value of an INT64 is 9223372036854775807 +** The above example is based on the fact that the maximum Sparkplug sequence number is +9223372036854775807 * [tck-testable tck-id-operational-behavior-rebirth-action-6]#[yellow-background]*[tck-id-operational-behavior-rebirth-action-6] Each DREBIRTH payload MUST use a seq number that is incremented by one from the previous message.*# * [tck-testable tck-id-operational-behavior-rebirth-action-7]#[yellow-background]*[tck-id-operational-behavior-rebirth-action-7] Each diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 31957812..b9fee2fe 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -327,9 +327,9 @@ message Payload { message Header { optional int64 timestamp = 1; // Timestamp at message sending time - optional int64 seq = 2; // Message sequence number - ordered stream of all messages from the Edge Node - optional int64 bd_seq = 3; // The Birth/Death sequence number - scopes this message to the MQTT session it was published in - optional int64 descriptor_seq = 4; // Descriptor sequence number - ordered stream per Sparkplug Descriptor (the Edge Node Descriptor for Edge Node messages, or a Device Descriptor for Device messages) + optional uint64 seq = 2; // Message sequence number - ordered stream of all messages from the Edge Node + optional uint64 bd_seq = 3; // The Birth/Death sequence number - scopes this message to the MQTT session it was published in + optional uint64 descriptor_seq = 4; // Descriptor sequence number - ordered stream per Sparkplug Descriptor (the Edge Node Descriptor for Edge Node messages, or a Device Descriptor for Device messages) optional string definition_id = 5; // Definition identifier: content hash of the metric definitions this message reports against optional string previous_definition_id = 6; // NMMETA/DMMETA only: the definition_id in effect before the change optional string source = 7; // Source for CMD and REBIRTH messages - the Host Application ID that sent the message @@ -502,7 +502,16 @@ timestamp is the point at which the reported values were current, and the timest is the time that metric's value last changed. A consuming application needs both to know what a value is and how long it has held that value. ** _seq_ -*** This is the sequence number which is a signed 64-bit integer. +*** This is the sequence number which is an unsigned 64-bit integer. +*** Non-normative comment: the Sparkplug sequence numbers - seq, bd_seq, and descriptor_seq - are +declared as unsigned 64-bit integers but roll over at 9223372036854775807, which is half of what +that type can hold. The restriction is deliberate. An unsigned type is used because a sequence +number is a counter and can never be negative, and the range is capped so that every value a +conformant Edge Node can produce also fits a signed 64-bit integer. Several widely used languages, +including Java, have no unsigned 64-bit type, so an implementation in one of them would otherwise +have to compare sequence numbers with unsigned arithmetic to remain correct near the top of the +range. Neither bound is reachable in practice: at one million messages per second the capped range +takes roughly 292,000 years to exhaust. **** [tck-testable tck-id-payloads-sequence-num-always-included]#[yellow-background]*[tck-id-payloads-sequence-num-always-included] A sequence number MUST be included in the payload of every Sparkplug MQTT message from an Edge Node except an NDEATH message that is delivered as the Edge Node's registered MQTT Will Message.*# @@ -521,10 +530,10 @@ NBIRTH message from an Edge Node MUST always contain a sequence number between 0 **** [tck-testable tck-id-payloads-sequence-num-incrementing]#[yellow-background]*[tck-id-payloads-sequence-num-incrementing] All subsequent messages after an NBIRTH from an Edge Node MUST contain a sequence number that is continually increasing by one in each message from that Edge Node until a value of -9223372036854775807 (max value of an INT64) is reached. At that point, the sequence number of the +9223372036854775807 is reached. At that point, the sequence number of the following message MUST be zero.*# ** _bd_seq_ -*** This is the birth/death sequence number which is a signed 64-bit integer. It scopes a message to +*** This is the birth/death sequence number which is an unsigned 64-bit integer. It scopes a message to the MQTT session in which it was published. Unlike seq and descriptor_seq, which advance from message to message, the bd_seq number is constant for the lifetime of an MQTT session and changes only when a new session is established. @@ -555,13 +564,13 @@ message from an Edge Node can be attributed to the session that produced it, and discard a message belonging to a session it has already replaced. **** [tck-testable tck-id-payloads-bd-sequence-num-incrementing]#[yellow-background]*[tck-id-payloads-bd-sequence-num-incrementing] The birth/death sequence number MUST start at zero and MUST increment by one on every new MQTT CONNECT -packet sent by the Edge Node, until a value of 9223372036854775807 (max value of an INT64) is reached. +packet sent by the Edge Node, until a value of 9223372036854775807 is reached. At that point the birth/death sequence number of the following MQTT CONNECT packet MUST be zero.*# ***** Non-normative comment: the bd_seq number is not incremented from message to message within a session, and it is not incremented by an NREBIRTH, because an NREBIRTH does not establish a new MQTT session. ** _descriptor_seq_ -*** This is the descriptor sequence number which is a signed 64-bit integer. It is an ordered +*** This is the descriptor sequence number which is an unsigned 64-bit integer. It is an ordered sequence number stream scoped to a single Sparkplug Descriptor (see the link:#introduction_sparkplug_ids[Sparkplug Identifiers] section), either the Edge Node Descriptor for an Edge Node's messages (NBIRTH, NDATA) or a Device Descriptor for a Device's messages (DBIRTH, @@ -584,7 +593,7 @@ sequence (see the Reconnect Optimization), the NBIRTH or DBIRTH is the first mes **** [tck-testable tck-id-payloads-descriptor-seq-incrementing]#[yellow-background]*[tck-id-payloads-descriptor-seq-incrementing] Each subsequent message in a descriptor sequence stream MUST contain a descriptor sequence number that is one greater than the descriptor sequence number of the previous message in that same stream, until a -value of 9223372036854775807 (max value of an INT64) is reached, after which the next descriptor +value of 9223372036854775807 is reached, after which the next descriptor sequence number MUST be 0.*# **** [tck-testable tck-id-payloads-descriptor-seq-rebirth]#[yellow-background]*[tck-id-payloads-descriptor-seq-rebirth] An NREBIRTH or DREBIRTH slip-streams a Host Application into the existing data stream and MUST NOT reset @@ -2231,8 +2240,8 @@ NOT be used to convey metric datatype, properties, or metadata.*# DBIRTH message MUST include a sequence number.*# * [tck-testable tck-id-payloads-dbirth-seq-inc]#[yellow-background]*[tck-id-payloads-dbirth-seq-inc] Every DBIRTH message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an -INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807. If in the previous +sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# * [tck-testable tck-id-payloads-dbirth-order]#[yellow-background]*[tck-id-payloads-dbirth-order] All DBIRTH messages sent by an Edge Node MUST be sent immediately after the NBIRTH and before any NDATA @@ -2334,8 +2343,8 @@ NOT be used to convey metric datatype, properties, or metadata.*# DREBIRTH message MUST include a sequence number.*# * [tck-testable tck-id-payloads-drebirth-seq-inc]#[yellow-background]*[tck-id-payloads-drebirth-seq-inc] Every DREBIRTH message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an -INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807. If in the previous +sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST be a value of 0.*# * [tck-testable tck-id-payloads-drebirth-order]#[yellow-background]*[tck-id-payloads-drebirth-order] All DREBIRTH messages sent by an Edge Node MUST be sent immediately after the NREBIRTH and before any @@ -2434,8 +2443,8 @@ published.*# NDATA message MUST include a sequence number.*# * [tck-testable tck-id-payloads-ndata-seq-inc]#[yellow-background]*[tck-id-payloads-ndata-seq-inc] Every NDATA message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an -INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807. If in the previous +sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# * [tck-testable tck-id-payloads-ndata-order]#[yellow-background]*[tck-id-payloads-ndata-order] All NDATA messages sent by an Edge Node MUST NOT be sent until all the NBIRTH and all DBIRTH messages @@ -2501,8 +2510,8 @@ published.*# DDATA message MUST include a sequence number.*# * [tck-testable tck-id-payloads-ddata-seq-inc]#[yellow-background]*[tck-id-payloads-ddata-seq-inc] Every DDATA message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an -INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807. If in the previous +sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# * [tck-testable tck-id-payloads-ddata-order]#[yellow-background]*[tck-id-payloads-ddata-order] All DDATA messages sent by an Edge Node MUST NOT be sent until all the NBIRTH and all DBIRTH messages @@ -2904,7 +2913,7 @@ message. payload compared to what was in the previous CONNECT packet. This ensures that any Host Applications will be able to distinguish between current and old bd_seq numbers in the event that messages are delivered out of order. When incrementing the bd_seq number, if the previous value was -9223372036854775807 (max value of an INT64), the next value must be zero. +9223372036854775807, the next value must be zero. * [tck-testable tck-id-payloads-ndeath-will-message-publisher]#[yellow-background]*[tck-id-payloads-ndeath-will-message-publisher] An NDEATH message SHOULD be published by the Edge Node before it intentionally disconnects from the MQTT Server.*# @@ -2972,8 +2981,8 @@ published.*# DDEATH message MUST include a sequence number.*# * [tck-testable tck-id-payloads-ddeath-seq-inc]#[yellow-background]*[tck-id-payloads-ddeath-seq-inc] Every DDEATH message MUST include a sequence number value that is one greater than the previous sequence -number sent by the Edge Node. This value MUST never exceed 9223372036854775807 (max value of an -INT64). If in the previous sequence number sent by the Edge Node was 9223372036854775807, the next +number sent by the Edge Node. This value MUST never exceed 9223372036854775807. If in the previous +sequence number sent by the Edge Node was 9223372036854775807, the next sequence number sent MUST have a value of 0.*# The following is a representation of a simple DDEATH message on the topic: @@ -3601,10 +3610,10 @@ would be silently rounded, typically to the nearest 256 nanoseconds. Metric valu UInt64, and DateTime have the same exposure. Non-normative comment: The sequence numbers are subject to this for the same reason. A Sparkplug -seq, bd_seq, and descriptor_seq each count up to the maximum value of their declared type before -rolling over to zero, as required in the link:#payloads_c_payload_header[Header] section, so they -reach magnitudes a JSON number cannot carry exactly. A sequence number is small early in a session -and large late in one, which would make a numeric form correct in testing and lossy in production. +seq, bd_seq, and descriptor_seq each count up to 9223372036854775807 before rolling over to zero, as +required in the link:#payloads_c_payload_header[Header] section, so they reach magnitudes a JSON +number cannot carry exactly. A sequence number is small early in a session and large late in one, +which would make a numeric form correct in testing and lossy in production. Non-normative comment: The rule is driven by the declared type of the field rather than by the magnitude of a particular value, so an encoder derives it from the schema alone and no value can From 86d291a240ffa560d6793888cd07a13ce53c6f59 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Mon, 26 Oct 2026 16:27:34 +0000 Subject: [PATCH 08/15] [#231] Rename the content oneof and correct the library-support claim The oneof was named 'content' and its uncompressed arm was named 'plain', so a Protocol Buffers JSON implementation emitted the key "plain" while every example in the chapter used "content". A field may not share the name of its containing oneof, so the oneof is renamed to payload_content and the arm becomes 'content'. The examples were right and the schema now agrees with them, and a stock library emits the correct key with no post-processing. Three prose references to the 'plain' field are updated. The non-normative comment claimed an implementation could produce most of the JSON encoding with library facilities. Measured against the compiled schema, that is not true: the structural mappings reshape every Metric, so an encoder writes its own traversal. What the derivation does settle is the scalar rules and the header, which the comment now says instead. --- .../chapters/Sparkplug_6_Payloads.adoc | 32 ++++++++++++------- 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index b9fee2fe..2d5bbd3c 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -346,8 +346,8 @@ message Payload { optional Header header = 1; // Always present and always uncompressed - oneof content { - Content plain = 2; // The Content of this message, uncompressed + oneof payload_content { + Content content = 2; // The Content of this message, uncompressed bytes compressed = 3; // The compressed bytes of a serialized Content } @@ -3152,13 +3152,14 @@ Compression applies to the content section of a payload only. The header section compressed, so a consuming application can read the header of a compressed payload without decompressing anything. -A compressed payload carries its content in the compressed field of the content oneof rather than in -the plain field. Which of the two is present is therefore evident from the payload itself, and no +A compressed payload carries its content in the compressed field of the payload_content oneof +rather than in the content field. Which of the two is present is therefore evident from the payload +itself, and no separate marker is required. * [tck-testable tck-id-payloads-compression-content-only]#[yellow-background]*[tck-id-payloads-compression-content-only] When using Sparkplug compression, the compressed field of a payload MUST contain the compressed bytes of -a serialized Content message, and the plain field MUST NOT be present.*# +a serialized Content message, and the content field MUST NOT be present.*# * [tck-testable tck-id-payloads-compression-header-uncompressed]#[yellow-background]*[tck-id-payloads-compression-header-uncompressed] The header section of a payload MUST NOT be compressed.*# ** Non-normative comment: this is what allows a Host Application to evaluate the sequence numbers @@ -3175,7 +3176,7 @@ before decompressing it. In earlier versions of this specification the algorithm metric inside the payload being compressed, which required the payload to be decompressed in order to discover how to decompress it. ** Non-normative comment: the value names the algorithm only. Whether the content is compressed at -all is already carried by which field of the content oneof is present, so no marker value is +all is already carried by which field of the payload_content oneof is present, so no marker value is required in this field. * [tck-testable tck-id-payloads-compression-algorithm-support]#[yellow-background]*[tck-id-payloads-compression-algorithm-support] A consuming application that is delivered a compressed payload whose algorithm it does not support @@ -3195,7 +3196,8 @@ the original, and the header and framing are not compressible in any case. The following is the NDATA message shown earlier in this chapter, with its content section compressed. The header is identical to the uncompressed form apart from the addition of the -compression_algorithm, the plain field is absent, and the compressed field holds the deflated bytes +compression_algorithm, the content field is absent, and the compressed field holds the deflated +bytes of the serialized Content: ---- @@ -3481,9 +3483,15 @@ spJv4.0, or the reverse, as a mechanical transformation. Non-normative comment: The JSON encoding is derived from the JSON Mapping defined by Google Protocol Buffers, which most Protocol Buffers libraries implement directly. Where this specification deviates from that mapping it does so explicitly, and those deviations are collected -in the link:#payloads_json_structural[Structural Mappings] section. An implementation that already -has the Sparkplug Protocol Buffers schema can therefore produce most of the JSON encoding with -library facilities rather than with hand-written code. +in the link:#payloads_json_structural[Structural Mappings] section. + +Non-normative comment: Deriving the encoding this way settles the scalar questions rather than +reopening them, so the rules for 64-bit integers, enumerated values, bytes, and non-finite floating +point values in the link:#payloads_json_scalar[Scalar Value Mapping] section are the ones a +Protocol Buffers JSON implementation already applies, and a payload's header needs no adjustment at +all. It does not mean a conformant encoder can be assembled from library calls. The structural +mappings reshape every Metric, so an implementation writes its own traversal and uses the library, +if at all, for the scalar formatting within it. [[payloads_json_encoding_selection]] ==== Encoding Selection @@ -3701,8 +3709,8 @@ represented according to the scalar mapping for the column's datatype.*# [tck-testable tck-id-payloads-json-content-key]#[yellow-background]*[tck-id-payloads-json-content-key] An uncompressed payload MUST carry its content under the key 'content', and a compressed payload MUST -carry it under the key 'compressed', matching the two arms of the content oneof in the Google -Protocol Buffers schema.*# +carry it under the key 'compressed', matching the two arms of the payload_content oneof in the +Google Protocol Buffers schema.*# [tck-testable tck-id-payloads-json-compressed-base64]#[yellow-background]*[tck-id-payloads-json-compressed-base64] The value of 'compressed' MUST be a JSON string containing the base64 encoding of the compressed bytes, From 18d02e6325236108b1448abf7afb82997f542a7a Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Mon, 26 Oct 2026 16:36:17 +0000 Subject: [PATCH 09/15] [#231] Make the JSON encoding self-contained and cover extension fields The section leaned on the Google Protocol Buffers JSON Mapping for part of its meaning. That mapping is vendor documentation rather than a versioned standard, it is specified for proto3 while the Sparkplug schema is proto2, and it does not cover extensions at all. The section now states its rules itself and says so. - The lowerCamelCase form a consumer must accept is defined here - remove each underscore and upper case the letter after it - so the assertion no longer depends on an external document for its meaning. - New Extension Fields section: an extension is keyed by its fully qualified name in square brackets, its value follows the ordinary rules for its type, and an unrecognized extension is ignored. The Sparkplug schema reserves an extension range in most messages, so leaving this undefined left a field that is well-defined on the wire undefined in JSON. Includes a note that unlike the binary encoding, where an unknown field is retained and re-emitted, an unrecognized JSON key does not survive a round trip. - Floating point values must be written with enough digits to parse back exactly: up to 9 significant decimal digits for a float and 17 for a double. --- .../chapters/Sparkplug_6_Payloads.adoc | 73 +++++++++++++++---- 1 file changed, 59 insertions(+), 14 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 2d5bbd3c..98d67fd5 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -3480,18 +3480,20 @@ may be converted from one to the other without loss and without consulting anyth message itself. This is what allows a bridging application to republish an spCv4.0 namespace onto spJv4.0, or the reverse, as a mechanical transformation. -Non-normative comment: The JSON encoding is derived from the JSON Mapping defined by Google -Protocol Buffers, which most Protocol Buffers libraries implement directly. Where this -specification deviates from that mapping it does so explicitly, and those deviations are collected -in the link:#payloads_json_structural[Structural Mappings] section. - -Non-normative comment: Deriving the encoding this way settles the scalar questions rather than -reopening them, so the rules for 64-bit integers, enumerated values, bytes, and non-finite floating -point values in the link:#payloads_json_scalar[Scalar Value Mapping] section are the ones a -Protocol Buffers JSON implementation already applies, and a payload's header needs no adjustment at -all. It does not mean a conformant encoder can be assembled from library calls. The structural -mappings reshape every Metric, so an implementation writes its own traversal and uses the library, -if at all, for the scalar formatting within it. +This section defines the JSON encoding completely. Conformance to it does not depend on any +document outside this specification. + +Non-normative comment: The scalar rules below were chosen to agree with the JSON Mapping that +Google Protocol Buffers defines for proto3, so the representation of 64-bit integers, enumerated +values, bytes, and non-finite floating point values will be familiar, and a Protocol Buffers JSON +implementation produces those forms directly. That agreement is a convenience and not a normative +dependency. The Sparkplug schema is proto2, that mapping is specified for proto3, it does not +define how extension fields are represented, and it is not a versioned standard, so this +specification states the rules itself rather than incorporating them by reference. + +Non-normative comment: Agreement on the scalar forms does not mean a conformant encoder can be +assembled from library calls. The structural mappings reshape every Metric, so an implementation +writes its own traversal and uses a library, if at all, for the scalar formatting within it. [[payloads_json_encoding_selection]] ==== Encoding Selection @@ -3527,8 +3529,14 @@ Google Protocol Buffers schema, except where the link:#payloads_json_structural[Structural Mappings] section names a different key.*# [tck-testable tck-id-payloads-json-field-names-accept]#[yellow-background]*[tck-id-payloads-json-field-names-accept] A -consuming application MUST also accept the lowerCamelCase form of a field name, because that is the -form a Google Protocol Buffers JSON Mapping implementation emits by default.*# +consuming application MUST also accept the lowerCamelCase form of a field name, which is the name +with each underscore removed and the letter following it converted to upper case, and every other +character left unchanged.*# + +Non-normative comment: The lowerCamelCase form is required on read because it is what a Google +Protocol Buffers JSON implementation emits by default. Under the rule above bd_seq becomes bdSeq, +previous_definition_id becomes previousDefinitionId, and is_multi_part becomes isMultiPart, while +names with no underscore, such as seq, md5, and engUnit, are unchanged. [tck-testable tck-id-payloads-json-omit-absent]#[yellow-background]*[tck-id-payloads-json-omit-absent] A field that is not present MUST be omitted from the JSON object rather than written with a JSON null @@ -3609,6 +3617,11 @@ or the integer value assigned to it in the Google Protocol Buffers schema.*# non-finite float or double value MUST be represented as one of the JSON strings "NaN", "Infinity", or "-Infinity", because RFC 8259 provides no numeric form for these values.*# +[tck-testable tck-id-payloads-json-float-precision]#[yellow-background]*[tck-id-payloads-json-float-precision] A +finite float or double value MUST be written with enough decimal digits that parsing it back +reproduces the original value exactly, which requires up to 9 significant decimal digits for a +float and up to 17 for a double.*# + Non-normative comment: 64-bit integers are represented as strings because a JSON number is not required by RFC 8259 to carry more precision than an IEEE 754 double, which holds integers exactly only up to 2^53^. Several widely used JSON parsers, including the one built into every web browser, @@ -3627,6 +3640,38 @@ Non-normative comment: The rule is driven by the declared type of the field rath magnitude of a particular value, so an encoder derives it from the schema alone and no value can lose precision because of how large it happened to be. +[[payloads_json_extensions]] +==== Extension Fields + +The Sparkplug Google Protocol Buffers schema reserves an extension range in most of its messages so +that a third party can add fields without colliding with future Sparkplug fields. The JSON encoding +represents those fields as well. + +[tck-testable tck-id-payloads-json-extension-key]#[yellow-background]*[tck-id-payloads-json-extension-key] A +field carried in a message's extension range MUST be represented with a JSON object key that is the +fully qualified name of that extension enclosed in square brackets, for example +'[com.example.vendor_tag]'.*# + +[tck-testable tck-id-payloads-json-extension-value]#[yellow-background]*[tck-id-payloads-json-extension-value] The +value of an extension field MUST be represented according to the same rules as any other field of +its type, as defined in the link:#payloads_json_scalar[Scalar Value Mapping] and +link:#payloads_json_structural[Structural Mappings] sections.*# + +[tck-not-testable tck-id-payloads-json-extension-ignore]#[yellow-background]*[tck-id-payloads-json-extension-ignore] A +consuming application that does not recognize an extension MUST ignore it, as it ignores any other +unrecognized key, and MUST NOT treat the payload as malformed.*# + +Non-normative comment: The bracketed form is the one Google Protocol Buffers JSON implementations +already use for extensions, so it is specified here rather than invented. It is stated normatively +because the Protocol Buffers JSON Mapping is defined for proto3, which has no extensions, and so +does not cover this case at all. + +Non-normative comment: An extension does not survive a round trip through an application that does +not know it. In the Google Protocol Buffers encoding an unknown field is retained and re-emitted, +but in JSON an unrecognized key is not. A deployment that depends on extensions reaching a +consumer through an intermediary should confirm that the intermediary understands them, rather +than assuming they are carried through. + [[payloads_json_structural]] ==== Structural Mappings From 4fad7093ae86b46a0e64d02427bf59ceb613612a Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Mon, 26 Oct 2026 17:04:16 +0000 Subject: [PATCH 10/15] [#231] Require payload metrics to be ordered by value change time Metrics in a payload are ordered oldest first, so a consuming application that applies them in array order arrives at the correct current value without comparing timestamps. This matters when one payload carries several values for the same metric, as a payload of historical values does. Metrics remain identified by name or alias, so position still carries no identity. A metric that omits its timestamp sorts as though its value change time were the payload timestamp, which is the time it is already taken to have changed. Metrics that share a value change time may appear in any order and a consumer must not reorder them. Multi-part file publishing depends on that: it identifies a metric by position, and every metric of a multi-part publish carries the same timestamp, so they tie and the publisher's order is carried unchanged by both encodings. Non-normative comments in both sections record the interaction. Also splits the JSON PropertySet statement. Uniqueness of property names is observable in a payload and stays testable; the absence of meaning in key order is not observable and becomes non-testable, with a note that RFC 8259 defines a JSON object as unordered so the requirement is on the publisher. --- .../chapters/Sparkplug_6_Payloads.adoc | 39 ++++++++++++++++++- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 98d67fd5..85ec0515 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -635,6 +635,25 @@ link:#payloads_c_compression[Compression] section. ** _metrics_ *** This is an array of metrics representing key/value/datatype values. Metrics are further defined link:#payloads_c_metric[here]. +*** [tck-testable tck-id-payloads-metrics-ordered]#[yellow-background]*[tck-id-payloads-metrics-ordered] The +metrics of a payload MUST be ordered by metric value change time, oldest first.*# +*** [tck-testable tck-id-payloads-metrics-ordered-no-timestamp]#[yellow-background]*[tck-id-payloads-metrics-ordered-no-timestamp] A +metric that omits its timestamp MUST be ordered as though its value change time were the payload +timestamp, which is the time that metric is taken to have changed as described in the +link:#payloads_c_metric[Metric] section.*# +*** [tck-not-testable tck-id-payloads-metrics-ordered-ties]#[yellow-background]*[tck-id-payloads-metrics-ordered-ties] Metrics +that share a value change time MAY appear in any order relative to one another, and a consuming +application MUST NOT reorder them.*# +*** Non-normative comment: Ordering by value change time means a consuming application that applies +the metrics in array order arrives at the correct current value without comparing timestamps, which +matters when one payload carries several values for the same metric, as a payload of historical +values does. Metrics remain identified by name or alias, so position still carries no identity. +*** Non-normative comment: The order among metrics that share a value change time is the publisher's +and is preserved by both encodings, because a JSON array and a Google Protocol Buffers repeated +field are both ordered. Multi-part file publishing depends on this: every metric of a multi-part +publish carries the same timestamp, as required in the +link:#payloads_c_file_publishing[File Publishing] section, so they tie, and the rules that +distinguish the first metric of the publish from the rest continue to apply. ** _body_ *** This is an array of bytes which can be used for any custom binary encoded data. @@ -3282,6 +3301,14 @@ metric MetaData file_name MUST be included and set to the String value of the fi When publishing in multi-part form, the following rules apply regarding the first metric associated with the file publish. +Non-normative comment: The rules below identify a metric by its position in the payload, which is +the one place in this specification where the position of a metric carries meaning. This does not +conflict with the requirement that metrics be ordered by value change time, stated in the +link:#payloads_c_payload_content[Content] section, because every metric of a multi-part file +publish carries the same timestamp. They therefore tie, the ordering requirement does not constrain +their relative order, and the order the publishing Edge Node or Device chooses is carried unchanged +by both payload encodings. A consuming application MUST NOT reorder them. + * [tck-testable tck-id-payloads-file-multi-metadata-size-first]#[yellow-background]*[tck-id-payloads-file-multi-metadata-size-first] The first metric regarding the file publish's metric MetaData size MUST be included and set to the uint64 value of the total number of separate metrics that make up the full multi-part file publish.*# @@ -3718,8 +3745,16 @@ PropertySet MUST be represented as a JSON object whose keys are the property nam are the corresponding PropertyValues.*# [tck-testable tck-id-payloads-json-propertyset-unique]#[yellow-background]*[tck-id-payloads-json-propertyset-unique] Property -names within a single PropertySet MUST be unique, and the order of the keys MUST NOT be -significant.*# +names within a single PropertySet MUST be unique.*# + +[tck-not-testable tck-id-payloads-json-propertyset-unordered]#[yellow-background]*[tck-id-payloads-json-propertyset-unordered] The +order of the keys of a PropertySet MUST NOT be significant, and an application MUST NOT encode +meaning in it.*# + +Non-normative comment: RFC 8259 defines a JSON object as an unordered collection, so a parser is +free to present the keys in any order and key order cannot be relied upon in any case. The +requirement is on the publisher not to attach meaning to it. Uniqueness is stated separately +because it is observable in a payload, whereas the absence of meaning in the ordering is not. [[payloads_json_array_types]] ===== Array Types From a10f5c0d52c10dc3a9fe3533a58f1a8010fafee1 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Mon, 26 Oct 2026 20:01:06 +0000 Subject: [PATCH 11/15] [#231] Address review comments on the JSON encoding 1. The conformance note justifying the session scoping rule was denser than the rule it explained. Reduced to the plain consequence: an Edge Node has one sequence number stream, so splitting a session across namespaces leaves a consumer of either with gaps in it. 2. Removed the requirement that a consuming application accept the lowerCamelCase form of a field name. A conformant publisher emits the Protocol Buffers field name, so the rule required consumers to accept a form no conformant publisher may produce - specifying leniency toward non-conformance and creating a second de facto wire form for every field. Replaced with implementer guidance: a Protocol Buffers JSON library converts to lowerCamelCase by default and must be configured to preserve the original names. 3. Explained why an absent field is omitted rather than written as null or zero. Zero is a legitimate value for seq, bd_seq, descriptor_seq, alias, and quality, so writing an absent field as 0 would be indistinguishable from one set to zero; and null would be a second, conflicting way to express what is_null already expresses. 4. Scoped the ignore-unknown rule to the names of fields within a message. It was open to being read as covering an unrecognized metric, which is a different situation entirely - a mismatch against the metric structure the application holds, resolved by the metadata rules and a Rebirth Request rather than by ignoring it. Both cases are now stated and contrasted. --- .../chapters/Sparkplug_10_Conformance.adoc | 6 +-- .../chapters/Sparkplug_6_Payloads.adoc | 41 ++++++++++++++----- 2 files changed, 33 insertions(+), 14 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc index ddcb86e1..e290e74c 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc @@ -51,9 +51,9 @@ selected by configuration.*# * [tck-testable tck-id-conformance-edge-node-encoding-session]#[yellow-background]*[tck-id-conformance-edge-node-encoding-session] A Sparkplug Edge Node MUST publish every message of an MQTT session using a single payload encoding on the corresponding single namespace.*# -** Non-normative comment: the sequence number is Edge-Node-wide and the bd_seq number scopes an MQTT -session, so an Edge Node that split one session across both namespaces would produce two interleaved -halves of one sequence number stream and neither would be consumable on its own. +** Non-normative comment: an Edge Node has one sequence number stream, so publishing part of a +session on one namespace and part on the other would leave a consumer of either namespace with gaps +in that stream. Support for strongly typed commands is optional. An Edge Node that exposes commands must do so in a conformant manner. diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 85ec0515..0c60cb74 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -3555,23 +3555,42 @@ JSON object key MUST be the name of the corresponding field exactly as it appear Google Protocol Buffers schema, except where the link:#payloads_json_structural[Structural Mappings] section names a different key.*# -[tck-testable tck-id-payloads-json-field-names-accept]#[yellow-background]*[tck-id-payloads-json-field-names-accept] A -consuming application MUST also accept the lowerCamelCase form of a field name, which is the name -with each underscore removed and the letter following it converted to upper case, and every other -character left unchanged.*# - -Non-normative comment: The lowerCamelCase form is required on read because it is what a Google -Protocol Buffers JSON implementation emits by default. Under the rule above bd_seq becomes bdSeq, -previous_definition_id becomes previousDefinitionId, and is_multi_part becomes isMultiPart, while -names with no underscore, such as seq, md5, and engUnit, are unchanged. +Non-normative comment: A Google Protocol Buffers JSON implementation converts field names to +lowerCamelCase by default, emitting bdSeq for bd_seq and isMultiPart for is_multi_part. That is not +the Sparkplug form, so an implementation that generates JSON with such a library must configure it +to preserve the original field names. In the Java and Python implementations this is the +preservingProtoFieldNames printer option. [tck-testable tck-id-payloads-json-omit-absent]#[yellow-background]*[tck-id-payloads-json-omit-absent] A field that is not present MUST be omitted from the JSON object rather than written with a JSON null or a zero value.*# +Non-normative comment: Every field of the Sparkplug schema is optional, so a field carries the +distinction between having been set and never having been set, and the JSON encoding has to keep +that distinction. Zero is a legitimate value for seq, bd_seq, descriptor_seq, alias, and quality, +so writing an absent field as 0 would make it indistinguishable from one deliberately set to zero. +A JSON null is excluded for a different reason: a null metric value is expressed by setting is_null +to true and omitting the value key, as required in the +link:#payloads_json_typed_values[Typed Values] section, so a null appearing as a value would be a +second and conflicting way to say the same thing. + [tck-testable tck-id-payloads-json-ignore-unknown]#[yellow-background]*[tck-id-payloads-json-ignore-unknown] A -consuming application MUST ignore any JSON object key it does not recognize, and MUST NOT treat the -payload as malformed on that basis.*# +consuming application MUST ignore a JSON object key that does not name a field it recognizes, and +MUST NOT treat the payload as malformed on that basis.*# + +[tck-not-testable tck-id-payloads-json-unknown-not-metric]#[yellow-background]*[tck-id-payloads-json-unknown-not-metric] The +requirement above applies only to the names of fields within a message. It does not apply to a +metric whose name or alias the consuming application does not recognize, which is a mismatch +against the metric structure the application holds and is resolved by the rules in the +link:#operational_behavior_modified_metadata_updates[Modified Metadata Updates] section rather than +by ignoring it.*# + +Non-normative comment: The two cases look similar in a payload and are not the same thing. An +unrecognized field name means the payload carries something this version of the application does +not know about, such as a field added by a later revision of this specification or a third party +extension; ignoring it is what allows an older application to keep working. An unrecognized metric +means the application's view of what the Edge Node reports is out of date, which is a +synchronization problem and a reason to issue a Rebirth Request, not something to pass over. Non-normative comment: The naming rule keeps a JSON payload visually identical to the field names in the Protocol Buffers schema, so the two encodings can be read against one another and against From c0781dce73bd4b92b5cb4a7928cddcc1d9b6e6d2 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Tue, 27 Oct 2026 20:43:14 +0000 Subject: [PATCH 12/15] [#231] Make DateTime a signed 64-bit value A DateTime value is a nanosecond count since epoch, the same quantity the payload and metric timestamps carry, so it follows them to int64 rather than staying unsigned. Leaving it as uint64 meant a DateTime value and a payload timestamp differed in signedness despite meaning the same thing. This moves a DateTime value from the uint64_value arm of TypedValue to int64_value, so unlike the timestamp field changes it is a wire change. A DateTime in a DataSet.Column is unaffected: there is no signed repeated field, so it stays in long_values alongside UInt64 and is carried as its two's complement representation, which the DataSet.Column section already requires for signed values. --- .../main/asciidoc/chapters/Sparkplug_6_Payloads.adoc | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 0c60cb74..06c21208 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -1722,9 +1722,15 @@ enum DataType { *** Google Protocol Buffer Type: string *** Sparkplug enum value: 12 * _DateTime_ -** Date time value as uint64 value representing nanoseconds since epoch (Jan 1, 1970) -** Google Protocol Buffer Type: uint64 +** Date time value as int64 value representing nanoseconds since epoch (Jan 1, 1970) +** Google Protocol Buffer Type: int64 ** Sparkplug enum value: 13 +** Non-normative comment: a DateTime value is a nanosecond count since epoch, the same quantity the +payload and metric timestamps carry, and is signed for the same reason. Every mainstream time +representation it interoperates with is signed, so the ceiling is 2262-04-11 as described in the +link:#operational_behavior_timestamps[Timestamps] section. A DateTime carried in a DataSet.Column +shares the long_values array with UInt64, as the +link:#payloads_c_dataset_column[DataSet.Column] section describes. * _Text_ ** String value (UTF-8) ** Google Protocol Buffer Type: string From bcf75babe2be5218c288fb59f50c937d6dae9533 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Tue, 27 Oct 2026 22:07:14 +0000 Subject: [PATCH 13/15] [#231] Bound the 64-bit JSON Schema type and define PropertySet TypedValues Review found two defects in the normative JSON Schema and its prose. nonNegative64 was "type": ["string","integer"] with "pattern": "^[0-9]+$". JSON Schema applies pattern only to strings, so a negative JSON number validated cleanly, as did a string far beyond the 64-bit cap. That governs seq, bd_seq, descriptor_seq, timestamp, alias, MetaData size and seq, and Status code - the fields whose range the rest of the specification relies on. Now bounded on both arms. The earlier negative tests missed this because they used the string form. The TypedValue flattening rule named only Metric and Template.Parameter as datatype carriers, but PropertySet.values is a repeated TypedValue, so a property value had no enclosing field to carry its datatype and could not be encoded by following the rule. The JSON Schema already resolved this the other way, nesting an object with its own datatype - which the rule forbade. The rule is now scoped to Metric and Template.Parameter, and the PropertySet case is stated to match the schema. Also corrects a reference to PropertyValues, which is not a v4 message. --- .../chapters/Sparkplug_6_Payloads.adoc | 21 ++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 06c21208..f15755bc 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -3740,9 +3740,14 @@ the datatype and selects one of several value fields. In JSON the datatype is al the enclosing object and the value field is not needed to disambiguate. [tck-testable tck-id-payloads-json-typedvalue-flat]#[yellow-background]*[tck-id-payloads-json-typedvalue-flat] A -TypedValue MUST be represented as the value itself, under the key 'value', rather than as a nested -object, and its datatype MUST be carried by the 'datatype' key of the enclosing Metric or the -'type' key of the enclosing Template.Parameter.*# +TypedValue that is a member of a Metric or of a Template.Parameter MUST be represented as the value +itself, under the key 'value', rather than as a nested object, and its datatype MUST be carried by +the 'datatype' key of the enclosing Metric or the 'type' key of the enclosing Template.Parameter.*# + +[tck-testable tck-id-payloads-json-typedvalue-propertyset]#[yellow-background]*[tck-id-payloads-json-typedvalue-propertyset] A +TypedValue that is a member of a PropertySet MUST instead be represented as an object carrying its +own 'datatype' key, its 'value', and 'is_null' where applicable, because a property value has no +enclosing Metric or Template.Parameter to carry its datatype.*# [tck-testable tck-id-payloads-json-typedvalue-parameter-keys]#[yellow-background]*[tck-id-payloads-json-typedvalue-parameter-keys] A Template.Parameter MUST therefore be represented with the keys 'name', 'type', and 'value', and @@ -3767,7 +3772,7 @@ application. [tck-testable tck-id-payloads-json-propertyset-object]#[yellow-background]*[tck-id-payloads-json-propertyset-object] A PropertySet MUST be represented as a JSON object whose keys are the property names and whose values -are the corresponding PropertyValues.*# +are the corresponding TypedValues.*# [tck-testable tck-id-payloads-json-propertyset-unique]#[yellow-background]*[tck-id-payloads-json-propertyset-unique] Property names within a single PropertySet MUST be unique.*# @@ -3848,7 +3853,13 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# }, "not": { "required": [ "content", "compressed" ] }, "$defs": { - "nonNegative64": { "type": [ "string", "integer" ], "pattern": "^[0-9]+$" }, + "nonNegative64": { + "comment": "0 to 9223372036854775807, as a JSON string or, on read, a JSON number", + "oneOf": [ + { "type": "integer", "minimum": 0, "maximum": 9223372036854775807 }, + { "type": "string", "pattern": "^(0|[1-9][0-9]{0,18})$" } + ] + }, "base64": { "type": "string", "contentEncoding": "base64" }, "number": { "oneOf": [ From 1fcab1ddcd8d09d5a15aabf2a42feb2a6f428767 Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Wed, 28 Oct 2026 21:31:02 +0000 Subject: [PATCH 14/15] [#231] Address review: STATE namespace, command addressing, and schema bounds Required: - The namespace substitution note made STATE ambiguous. STATE is pinned to spCv4.0/STATE/sparkplug_host_id, so a JSON Edge Node following the note would have waited on spJv4.0/STATE while the Host published on spCv4.0/STATE and never seen it come online. STATE is now stated as the one exception, with the reason. - Nothing said which namespace a Host Application uses to address a particular Edge Node. A Host supports both encodings, so it could send an NCMD or a Rebirth Request on a namespace the Edge Node does not subscribe to; the MQTT Server accepts it and it is never delivered, so Rebirth recovery fails silently. A Host now publishes to an Edge Node on the namespace of that Edge Node's NBIRTH, and an Edge Node subscribes to its command topics on the namespace it publishes on. Also: - nonNegative64 capped every 64-bit field at 2^63-1, but alias, MetaData.size and Status.code are uncapped uint64, so a legal spCv4.0 payload could not be expressed as a schema-valid spJv4.0 one - contradicting the lossless conversion this section claims. Split into uint64 and sequence64, with the string branch of sequence64 bounded by length and its limit stated. - The Sparkplug Aware MQTT Server encoding requirement is now conditional on replacing NDEATH timestamps, which is the only payload-touching behavior required of it and is itself a MAY. - Flattening drops the TypedValue's own datatype and is_null. The two copies must now agree in the Protocol Buffers encoding, the single JSON key represents both, and a null Template.Parameter has a JSON form: is_null is added to the parameter keys and to the schema. - The metric ordering MUST is scoped to payloads carrying more than one value for the same metric, with a SHOULD elsewhere so a BIRTH in browse order stays conformant, and it now says it does not reach Template or Command metrics. - Nits: INT64 rollover wording, a stale (max value of an INT64), DateTimeArray elements described as signed, and a MUST NOT inside a non-normative comment. --- .../chapters/Sparkplug_10_Conformance.adoc | 23 ++++-- .../asciidoc/chapters/Sparkplug_4_Topics.adoc | 12 +++ .../Sparkplug_5_Operational_Behavior.adoc | 2 +- .../chapters/Sparkplug_6_Payloads.adoc | 77 +++++++++++++------ 4 files changed, 85 insertions(+), 29 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc index e290e74c..fa9475c4 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_10_Conformance.adoc @@ -51,6 +51,9 @@ selected by configuration.*# * [tck-testable tck-id-conformance-edge-node-encoding-session]#[yellow-background]*[tck-id-conformance-edge-node-encoding-session] A Sparkplug Edge Node MUST publish every message of an MQTT session using a single payload encoding on the corresponding single namespace.*# +* [tck-testable tck-id-conformance-edge-node-encoding-subscribe]#[yellow-background]*[tck-id-conformance-edge-node-encoding-subscribe] A +Sparkplug Edge Node MUST subscribe to its command topics on the same namespace it publishes on, so +that a Host Application addressing it on that namespace reaches it.*# ** Non-normative comment: an Edge Node has one sequence number stream, so publishing part of a session on one namespace and part on the other would leave a consumer of either namespace with gaps in that stream. @@ -100,6 +103,14 @@ consume messages on both the spCv4.0 and spJv4.0 namespaces.*# * [tck-testable tck-id-conformance-host-encoding-publish]#[yellow-background]*[tck-id-conformance-host-encoding-publish] When a Sparkplug Host Application publishes a message, it MUST encode that message using the encoding that corresponds to the namespace it publishes on.*# +* [tck-testable tck-id-conformance-host-encoding-address]#[yellow-background]*[tck-id-conformance-host-encoding-address] When +a Sparkplug Host Application publishes a message addressed to a particular Edge Node or Device, +including an NCMD, a DCMD, and a Rebirth Request, it MUST publish on the namespace of that Edge +Node's current NBIRTH.*# +** Non-normative comment: a Host Application supports both encodings, so without this it could +address an Edge Node on a namespace that Edge Node does not subscribe to. The message would be +accepted by the MQTT Server and never delivered, and a Rebirth Request in particular would fail +silently, leaving the Host unable to recover its view of that Edge Node. ** Non-normative comment: which namespaces a Host Application subscribes to remains a deployment decision. The requirement is that it is capable of decoding either, so that an Edge Node's choice of encoding does not determine which Host Applications can consume it. This is the property that makes @@ -168,11 +179,13 @@ continue to be published with the MQTT retain flag set to false and are not reta * [tck-testable tck-id-conformance-mqtt-aware-basic]#[yellow-background]*[tck-id-conformance-mqtt-aware-basic] A Sparkplug Aware MQTT Server MUST support all aspects of a Sparkplug Compliant MQTT Server*# * [tck-testable tck-id-conformance-mqtt-aware-encoding-both]#[yellow-background]*[tck-id-conformance-mqtt-aware-encoding-both] A -Sparkplug Aware MQTT Server MUST support both Sparkplug payload encodings on both the spCv4.0 and -spJv4.0 namespaces.*# -** Non-normative comment: unlike a Sparkplug Compliant MQTT Server, a Sparkplug Aware MQTT Server -decodes and re-encodes payloads in order to perform the behavior required of it below, so it cannot -treat a payload as opaque and must therefore implement both encodings. +Sparkplug Aware MQTT Server that replaces NDEATH timestamps MUST do so for payloads in both +encodings, on both the spCv4.0 and spJv4.0 namespaces.*# +** Non-normative comment: replacing an NDEATH timestamp is the only behavior required of a +Sparkplug Aware MQTT Server that decodes a payload at all, and it is itself optional. A Server that +does not do it never inspects a payload and needs neither encoding. The requirement is therefore +conditional, so that offering the timestamp behavior for one encoding only cannot leave Edge Nodes +on the other namespace silently unserved. * [tck-testable tck-id-conformance-mqtt-aware-ndeath-timestamp]#[yellow-background]*[tck-id-conformance-mqtt-aware-ndeath-timestamp] A Sparkplug Aware MQTT Server MAY replace the timestmap of NDEATH messages. If it does, it MUST set the timestamp to the UTC time at which it attempts to deliver the NDEATH to subscribed clients*# diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc index 2d5d0986..6bad2e4a 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_4_Topics.adoc @@ -78,6 +78,18 @@ that element substituted, because the two namespaces share one topic structure. payload carried on a topic is determined by the namespace element of that topic, as required in the link:#payloads_json_encoding_selection[Encoding Selection] section. +[tck-testable tck-id-topics-state-single-namespace]#[yellow-background]*[tck-id-topics-state-single-namespace] The +STATE topic is the one exception. A STATE message MUST be published on +'spCv4.0/STATE/sparkplug_host_id' whatever payload encoding the Edge Nodes and Host Applications in +that infrastructure use, and an Edge Node that waits on a Primary Host Application MUST subscribe +to that topic rather than to a spJv4.0 equivalent.*# + +Non-normative comment: STATE carries a JSON document rather than a Sparkplug payload, so there is +nothing for a second namespace to encode differently. Publishing it on one topic also keeps a +single Will Message per Host Application. Without this rule a JSON Edge Node would wait on +spJv4.0/STATE while the Host published on spCv4.0/STATE, and would never see the Host come +online. + [[topics_topic_prefix_element]] ==== topic_prefix Element diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc index 6802639c..2ab161a2 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_5_Operational_Behavior.adoc @@ -1265,7 +1265,7 @@ values stop making sense. A Host Application also has to establish where in a descriptor sequence stream it is starting from. A BIRTH begins a stream and carries a descriptor_seq of 0. A REBIRTH does not begin a stream; it slip-streams the Host Application into a stream that is already running and carries the next -incrementing value, which is only 0 in the case of an INT64 rollover. +incrementing value, which is only 0 in the case of a sequence number rollover. * [tck-testable tck-id-operational-behavior-host-partial-descriptor-seq-anchor]#[yellow-background]*[tck-id-operational-behavior-host-partial-descriptor-seq-anchor] A Host Application MUST anchor its expected descriptor_seq for a Sparkplug Descriptor on the diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index f15755bc..8dcf1bc6 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -635,8 +635,15 @@ link:#payloads_c_compression[Compression] section. ** _metrics_ *** This is an array of metrics representing key/value/datatype values. Metrics are further defined link:#payloads_c_metric[here]. -*** [tck-testable tck-id-payloads-metrics-ordered]#[yellow-background]*[tck-id-payloads-metrics-ordered] The -metrics of a payload MUST be ordered by metric value change time, oldest first.*# +*** [tck-testable tck-id-payloads-metrics-ordered]#[yellow-background]*[tck-id-payloads-metrics-ordered] A +payload that carries more than one value for the same metric MUST order its metrics by value change +time, oldest first.*# +*** [tck-testable tck-id-payloads-metrics-ordered-should]#[yellow-background]*[tck-id-payloads-metrics-ordered-should] Every +other payload SHOULD order its metrics the same way.*# +*** [tck-not-testable tck-id-payloads-metrics-ordered-scope]#[yellow-background]*[tck-id-payloads-metrics-ordered-scope] This +ordering applies to the metrics array of the payload's content. It does not apply to the metrics of +a Template, or to the arguments or results of a Command, whose order is defined by the Template +Definition or the Command definition.*# *** [tck-testable tck-id-payloads-metrics-ordered-no-timestamp]#[yellow-background]*[tck-id-payloads-metrics-ordered-no-timestamp] A metric that omits its timestamp MUST be ordered as though its value change time were the payload timestamp, which is the time that metric is taken to have changed as described in the @@ -645,9 +652,11 @@ link:#payloads_c_metric[Metric] section.*# that share a value change time MAY appear in any order relative to one another, and a consuming application MUST NOT reorder them.*# *** Non-normative comment: Ordering by value change time means a consuming application that applies -the metrics in array order arrives at the correct current value without comparing timestamps, which -matters when one payload carries several values for the same metric, as a payload of historical -values does. Metrics remain identified by name or alias, so position still carries no identity. +the metrics in array order arrives at the correct current value without comparing timestamps. That +is why it is required only where a payload carries several values for the same metric, as a payload +of historical values does; elsewhere it is a recommendation, so that an Edge Node emitting a BIRTH +in browse order remains conformant. Metrics remain identified by name or alias, so position still +carries no identity. *** Non-normative comment: The order among metrics that share a value change time is the publisher's and is preserved by both encodings, because a JSON array and a Google Protocol Buffers repeated field are both ordered. Multi-part file publishing depends on this: every metric of a multi-part @@ -1846,7 +1855,8 @@ ends on a byte boundary. *** Sparkplug enum value: 33 *** Example (string array to Metric bytes_value): [ABC, hello] -> [0x41, 0x42, 0x43, 0x00, 0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x00] ** _DateTimeArray_ -*** DateTimeArray as an array of packed little endian bytes where each Datetime value is an 8-byte +*** DateTimeArray as an array of packed little endian bytes where each DateTime value is a signed +8-byte value representing the number of nanoseconds since epoch in UTC *** Google Protocol Buffer Type: bytes *** Sparkplug enum value: 34 @@ -3103,8 +3113,8 @@ same bd_seq number, and that value MUST be the one registered in the Will Messag session's MQTT CONNECT packet.*# * [tck-testable tck-id-payloads-state-bdseq-increment]#[yellow-background]*[tck-id-payloads-state-bdseq-increment] The bd_seq number MUST start at zero and MUST increment by one on every new MQTT CONNECT packet the -Sparkplug Host Application sends to a given MQTT Server, until a value of 9223372036854775807 (max -value of an INT64) is reached. At that point the bd_seq number of the following MQTT CONNECT packet +Sparkplug Host Application sends to a given MQTT Server, until a value of 9223372036854775807 is +reached. At that point the bd_seq number of the following MQTT CONNECT packet MUST be zero.*# ** Non-normative comment: these are the same rules that apply to an Edge Node's bd_seq number. As with an Edge Node, the bd_seq number is not incremented from message to message within a session. @@ -3313,7 +3323,7 @@ conflict with the requirement that metrics be ordered by value change time, stat link:#payloads_c_payload_content[Content] section, because every metric of a multi-part file publish carries the same timestamp. They therefore tie, the ordering requirement does not constrain their relative order, and the order the publishing Edge Node or Device chooses is carried unchanged -by both payload encodings. A consuming application MUST NOT reorder them. +by both payload encodings. * [tck-testable tck-id-payloads-file-multi-metadata-size-first]#[yellow-background]*[tck-id-payloads-file-multi-metadata-size-first] The first metric regarding the file publish's metric MetaData size MUST be included and set to the uint64 @@ -3749,9 +3759,22 @@ TypedValue that is a member of a PropertySet MUST instead be represented as an o own 'datatype' key, its 'value', and 'is_null' where applicable, because a property value has no enclosing Metric or Template.Parameter to carry its datatype.*# +[tck-testable tck-id-payloads-typedvalue-datatype-agree]#[yellow-background]*[tck-id-payloads-typedvalue-datatype-agree] Where +a Metric carries both its own datatype and a TypedValue that also carries one, as it does in the +Google Protocol Buffers encoding, the two MUST be equal, and the same applies to is_null.*# + +[tck-testable tck-id-payloads-json-typedvalue-carries-both]#[yellow-background]*[tck-id-payloads-json-typedvalue-carries-both] The +single 'datatype' and 'is_null' key a flattened JSON value carries represents both, so an +application converting a JSON payload to the Google Protocol Buffers encoding MUST set the Metric +field and the TypedValue field from it.*# + +[tck-testable tck-id-payloads-json-parameter-is-null]#[yellow-background]*[tck-id-payloads-json-parameter-is-null] A +Template.Parameter whose value is null MUST carry 'is_null' set to true and omit 'value', as any +other null value does.*# + [tck-testable tck-id-payloads-json-typedvalue-parameter-keys]#[yellow-background]*[tck-id-payloads-json-typedvalue-parameter-keys] A -Template.Parameter MUST therefore be represented with the keys 'name', 'type', and 'value', and -MUST NOT use the key 'typed_value'.*# +Template.Parameter MUST therefore be represented with the keys 'name', 'type', 'value', and +'is_null' where applicable, and MUST NOT use the key 'typed_value'.*# Non-normative comment: These are the two places where a JSON key is not a Protocol Buffers field name. 'value' is the name of the oneof within TypedValue rather than of any field in it, because @@ -3853,9 +3876,16 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# }, "not": { "required": [ "content", "compressed" ] }, "$defs": { - "nonNegative64": { - "comment": "0 to 9223372036854775807, as a JSON string or, on read, a JSON number", - "oneOf": [ + "uint64": { + "comment": "an unsigned 64-bit value: 0 to 18446744073709551615", + "anyOf": [ + { "type": "integer", "minimum": 0, "maximum": 18446744073709551615 }, + { "type": "string", "pattern": "^(0|[1-9][0-9]{0,19})$" } + ] + }, + "sequence64": { + "comment": "a Sparkplug sequence number or timestamp: 0 to 9223372036854775807. The string form is bounded by length only; the exact upper bound is normative prose, not expressible as a readable pattern.", + "anyOf": [ { "type": "integer", "minimum": 0, "maximum": 9223372036854775807 }, { "type": "string", "pattern": "^(0|[1-9][0-9]{0,18})$" } ] @@ -3876,10 +3906,10 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "header": { "type": "object", "properties": { - "timestamp": { "$ref": "#/$defs/nonNegative64" }, - "seq": { "$ref": "#/$defs/nonNegative64" }, - "bd_seq": { "$ref": "#/$defs/nonNegative64" }, - "descriptor_seq": { "$ref": "#/$defs/nonNegative64" }, + "timestamp": { "$ref": "#/$defs/sequence64" }, + "seq": { "$ref": "#/$defs/sequence64" }, + "bd_seq": { "$ref": "#/$defs/sequence64" }, + "descriptor_seq": { "$ref": "#/$defs/sequence64" }, "definition_id": { "type": "string" }, "previous_definition_id": { "type": "string" }, "source": { "type": "string" }, @@ -3909,8 +3939,8 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "type": "object", "properties": { "name": { "type": "string" }, - "alias": { "$ref": "#/$defs/nonNegative64" }, - "timestamp": { "$ref": "#/$defs/nonNegative64" }, + "alias": { "$ref": "#/$defs/uint64" }, + "timestamp": { "$ref": "#/$defs/sequence64" }, "datatype": { "$ref": "#/$defs/datatype" }, "quality": { "type": "integer", "minimum": 0 }, "quality_context": { "type": "string" }, @@ -3942,8 +3972,8 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "properties": { "is_multi_part": { "type": "boolean" }, "content_type": { "type": "string" }, - "size": { "$ref": "#/$defs/nonNegative64" }, - "seq": { "$ref": "#/$defs/nonNegative64" }, + "size": { "$ref": "#/$defs/uint64" }, + "seq": { "$ref": "#/$defs/sequence64" }, "file_name": { "type": "string" }, "file_type": { "type": "string" }, "md5": { "type": "string" }, @@ -3981,6 +4011,7 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "properties": { "name": { "type": "string" }, "type": { "$ref": "#/$defs/datatype" }, + "is_null": { "type": "boolean" }, "value": { "$ref": "#/$defs/value" } } } @@ -3996,7 +4027,7 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "status": { "type": "object", "properties": { - "code": { "$ref": "#/$defs/nonNegative64" }, + "code": { "$ref": "#/$defs/uint64" }, "message": { "type": "string" } } } From 40728fa51e8d009d6ff0696adcd109e3e1d14e1a Mon Sep 17 00:00:00 2001 From: wes-johnson Date: Thu, 29 Oct 2026 02:04:29 +0000 Subject: [PATCH 15/15] [#231] Point MetaData.seq at uint64 and note the length-bounded string form MetaData.seq sequences the parts of a multi-part file transfer and is an uncapped uint64 in both the schema and the prose. The earlier split pointed it at sequence64 because the bulk rename matched on the field name, which the two fields share, so it was capped at 2^63-1 by accident - the same class of mistake as the regex that produced the recursion. Also gives the uint64 definition the same note sequence64 carries, since its string branch is bounded by length rather than by value, and re-wraps the DateTimeArray entry left split mid-phrase by the earlier edit. --- .../src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc index 8dcf1bc6..8a67787d 100644 --- a/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc +++ b/specification/src/main/asciidoc/chapters/Sparkplug_6_Payloads.adoc @@ -1856,8 +1856,7 @@ ends on a byte boundary. *** Example (string array to Metric bytes_value): [ABC, hello] -> [0x41, 0x42, 0x43, 0x00, 0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x00] ** _DateTimeArray_ *** DateTimeArray as an array of packed little endian bytes where each DateTime value is a signed -8-byte -value representing the number of nanoseconds since epoch in UTC +8-byte value representing the number of nanoseconds since epoch in UTC *** Google Protocol Buffer Type: bytes *** Sparkplug enum value: 34 *** Example (DateTime array -> nanoseconds since epoch -> Metric bytes_value): ['Wednesday, October 21, 2009 5:27:55.335 AM', 'Friday, June 24, 2022 9:57:55 PM'] -> [1256102875335000000, 1656107875000000000] -> [0xC0, 0x7F, 0xB0, 0xF5, 0xE8, 0x92, 0x6E, 0x11, 0x00, 0x1E, 0x1A, 0x7F, 0x56, 0xAD, 0xFB, 0x16] @@ -3877,7 +3876,7 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "not": { "required": [ "content", "compressed" ] }, "$defs": { "uint64": { - "comment": "an unsigned 64-bit value: 0 to 18446744073709551615", + "comment": "an unsigned 64-bit value: 0 to 18446744073709551615. The string form is bounded by length only; the exact upper bound is normative prose, not expressible as a readable pattern.", "anyOf": [ { "type": "integer", "minimum": 0, "maximum": 18446744073709551615 }, { "type": "string", "pattern": "^(0|[1-9][0-9]{0,19})$" } @@ -3973,7 +3972,7 @@ Sparkplug JSON payload MUST validate against the JSON Schema below.*# "is_multi_part": { "type": "boolean" }, "content_type": { "type": "string" }, "size": { "$ref": "#/$defs/uint64" }, - "seq": { "$ref": "#/$defs/sequence64" }, + "seq": { "$ref": "#/$defs/uint64" }, "file_name": { "type": "string" }, "file_type": { "type": "string" }, "md5": { "type": "string" },