Repository navigation
[#231] Define the spJv4.0 JSON payload encoding #645
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
4198e90
1685509
999e5ea
1b39c72
ea2cc00
8b35e0e
de91196
86d291a
18d02e6
4fad709
a10f5c0
c0781dc
bcf75ba
1fcab1d
40728fa
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,8 +35,28 @@ typically be both an Edge Node and a Host Application. | |
| ==== Sparkplug Edge Node | ||
|
|
||
| A Sparkplug Edge Node is typically an 'Edge Gateway'. It sends and receives data to an MQTT Server | ||
| using the spCv4.0/# namespace. Edge Nodes typically interact with physical devices to gather data | ||
| and also write to device outputs. | ||
| using a Sparkplug namespace, either spCv4.0/# or spJv4.0/#. Edge Nodes typically interact with | ||
| physical devices to gather data and also write to device outputs. | ||
|
|
||
| Sparkplug defines two payload encodings, as described in the | ||
| link:#payloads_json_encoding_selection[Encoding Selection] section. An Edge Node is not required to | ||
| implement both. | ||
|
|
||
| * [tck-testable tck-id-conformance-edge-node-encoding-one]#[yellow-background]*[tck-id-conformance-edge-node-encoding-one] A | ||
| Sparkplug Edge Node MUST support at least one of the two Sparkplug payload encodings, and MUST | ||
| publish on the namespace that corresponds to the encoding it uses.*# | ||
| * [tck-not-testable tck-id-conformance-edge-node-encoding-both]#[yellow-background]*[tck-id-conformance-edge-node-encoding-both] A | ||
| Sparkplug Edge Node MAY support both payload encodings, in which case the encoding it uses is | ||
| selected by configuration.*# | ||
| * [tck-testable tck-id-conformance-edge-node-encoding-session]#[yellow-background]*[tck-id-conformance-edge-node-encoding-session] A | ||
| Sparkplug Edge Node MUST publish every message of an MQTT session using a single payload encoding on | ||
| the corresponding single namespace.*# | ||
| * [tck-testable tck-id-conformance-edge-node-encoding-subscribe]#[yellow-background]*[tck-id-conformance-edge-node-encoding-subscribe] A | ||
| Sparkplug Edge Node MUST subscribe to its command topics on the same namespace it publishes on, so | ||
| that a Host Application addressing it on that namespace reaches it.*# | ||
| ** Non-normative comment: an Edge Node has one sequence number stream, so publishing part of a | ||
| session on one namespace and part on the other would leave a consumer of either namespace with gaps | ||
| in that stream. | ||
|
|
||
| Support for strongly typed commands is optional. An Edge Node that exposes commands must do so in a | ||
| conformant manner. | ||
|
|
@@ -74,6 +94,28 @@ NREBIRTH, and NDEATH of every Edge Node reporting a Device of interest, which ca | |
| state that encapsulates the state of its Devices) and its use of the descriptor sequence number in | ||
| place of the Edge-Node-wide sequence number. | ||
|
|
||
| A Sparkplug Host Application consumes data produced by Edge Nodes that it does not control, and so | ||
| cannot assume which payload encoding those Edge Nodes use. | ||
|
|
||
| * [tck-testable tck-id-conformance-host-encoding-both]#[yellow-background]*[tck-id-conformance-host-encoding-both] A | ||
| Sparkplug Host Application MUST support both Sparkplug payload encodings, and MUST be able to | ||
| consume messages on both the spCv4.0 and spJv4.0 namespaces.*# | ||
| * [tck-testable tck-id-conformance-host-encoding-publish]#[yellow-background]*[tck-id-conformance-host-encoding-publish] When | ||
| a Sparkplug Host Application publishes a message, it MUST encode that message using the encoding | ||
| that corresponds to the namespace it publishes on.*# | ||
| * [tck-testable tck-id-conformance-host-encoding-address]#[yellow-background]*[tck-id-conformance-host-encoding-address] When | ||
| a Sparkplug Host Application publishes a message addressed to a particular Edge Node or Device, | ||
| including an NCMD, a DCMD, and a Rebirth Request, it MUST publish on the namespace of that Edge | ||
| Node's current NBIRTH.*# | ||
| ** Non-normative comment: a Host Application supports both encodings, so without this it could | ||
| address an Edge Node on a namespace that Edge Node does not subscribe to. The message would be | ||
| accepted by the MQTT Server and never delivered, and a Rebirth Request in particular would fail | ||
| silently, leaving the Host unable to recover its view of that Edge Node. | ||
| ** Non-normative comment: which namespaces a Host Application subscribes to remains a deployment | ||
| decision. The requirement is that it is capable of decoding either, so that an Edge Node's choice of | ||
| encoding does not determine which Host Applications can consume it. This is the property that makes | ||
| the two encodings interoperable rather than two separate ecosystems. | ||
|
|
||
| [tck-testable tck-id-conformance-primary-host]#[yellow-background]*[tck-id-conformance-primary-host] Sparkplug | ||
| Host Applications MUST publish 'STATE' messages that represent its Birth and Death Certificates.*# | ||
|
|
||
|
|
@@ -116,6 +158,10 @@ Sparkplug conformant MQTT Server MUST support all aspects of Will Messages inclu | |
| * [tck-testable tck-id-conformance-mqtt-retained]#[yellow-background]*[tck-id-conformance-mqtt-retained] A | ||
| Sparkplug conformant MQTT Server MUST support all aspects of the 'retain flag'*# | ||
|
|
||
| Non-normative comment: a Sparkplug Compliant MQTT Server has no payload requirements of any kind. It | ||
| treats a Sparkplug payload as an opaque sequence of bytes, so both payload encodings are carried by | ||
| the same server with no additional support. | ||
|
|
||
| [[conformance_sparkplug_aware_mqtt_server]] | ||
| ==== Sparkplug Aware MQTT Server | ||
|
|
||
|
|
@@ -132,6 +178,14 @@ continue to be published with the MQTT retain flag set to false and are not reta | |
|
|
||
| * [tck-testable tck-id-conformance-mqtt-aware-basic]#[yellow-background]*[tck-id-conformance-mqtt-aware-basic] A | ||
| Sparkplug Aware MQTT Server MUST support all aspects of a Sparkplug Compliant MQTT Server*# | ||
| * [tck-testable tck-id-conformance-mqtt-aware-encoding-both]#[yellow-background]*[tck-id-conformance-mqtt-aware-encoding-both] A | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Optional (but it affects every broker vendor): the premise here doesn't hold. The only payload-touching behaviour required of an Aware server below is As written, every Aware server has to implement JSON decode/encode even if it never rewrites a timestamp. Suggest making it conditional: "If a Sparkplug Aware MQTT Server replaces NDEATH timestamps, it MUST do so for both encodings." and fixing the comment.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 1fcab1d. You're right that the premise didn't hold — I had asserted the Aware server decodes payloads, when the only payload-touching behaviour required of it is the NDEATH timestamp replacement, which is itself a MAY. Now conditional: a Sparkplug Aware MQTT Server that replaces NDEATH timestamps MUST do so for payloads in both encodings. A server that does not never inspects a payload and needs neither encoding. The comment is rewritten to say that, and to note the requirement exists so that offering the timestamp behaviour for one encoding only cannot leave Edge Nodes on the other namespace silently unserved. |
||
| Sparkplug Aware MQTT Server that replaces NDEATH timestamps MUST do so for payloads in both | ||
| encodings, on both the spCv4.0 and spJv4.0 namespaces.*# | ||
| ** Non-normative comment: replacing an NDEATH timestamp is the only behavior required of a | ||
| Sparkplug Aware MQTT Server that decodes a payload at all, and it is itself optional. A Server that | ||
| does not do it never inspects a payload and needs neither encoding. The requirement is therefore | ||
| conditional, so that offering the timestamp behavior for one encoding only cannot leave Edge Nodes | ||
| on the other namespace silently unserved. | ||
| * [tck-testable tck-id-conformance-mqtt-aware-ndeath-timestamp]#[yellow-background]*[tck-id-conformance-mqtt-aware-ndeath-timestamp] A | ||
| Sparkplug Aware MQTT Server MAY replace the timestmap of NDEATH messages. If it does, it MUST set | ||
| the timestamp to the UTC time at which it attempts to deliver the NDEATH to subscribed clients*# | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -72,6 +72,24 @@ and spJv4.0 namespaces, which differ only in payload encoding. For earlier 'Spar | |
| 'Sparkplug B' topic namespace definitions and concepts, see the earlier versions of the Sparkplug | ||
| Specification. | ||
|
|
||
| Non-normative comment: Topics are written with the spCv4.0 namespace element throughout the | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Required: this note contradicts the STATE rule at A JSON-only Edge Node configured to wait for a Primary Host follows this note and watches Suggest excluding STATE explicitly, e.g. "STATE is not a Sparkplug payload and is published only at spCv4.0/STATE/sparkplug_host_id regardless of the encoding an Edge Node uses.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 1fcab1d. You're right, and the failure mode is exactly as you describe — a JSON Edge Node would have waited on a Added |
||
| remainder of this specification. Every such topic applies equally to the spJv4.0 namespace with | ||
| that element substituted, because the two namespaces share one topic structure. The encoding of the | ||
| payload carried on a topic is determined by the namespace element of that topic, as required in the | ||
| link:#payloads_json_encoding_selection[Encoding Selection] section. | ||
|
|
||
| [tck-testable tck-id-topics-state-single-namespace]#[yellow-background]*[tck-id-topics-state-single-namespace] The | ||
| STATE topic is the one exception. A STATE message MUST be published on | ||
| 'spCv4.0/STATE/sparkplug_host_id' whatever payload encoding the Edge Nodes and Host Applications in | ||
| that infrastructure use, and an Edge Node that waits on a Primary Host Application MUST subscribe | ||
| to that topic rather than to a spJv4.0 equivalent.*# | ||
|
|
||
| Non-normative comment: STATE carries a JSON document rather than a Sparkplug payload, so there is | ||
| nothing for a second namespace to encode differently. Publishing it on one topic also keeps a | ||
| single Will Message per Host Application. Without this rule a JSON Edge Node would wait on | ||
| spJv4.0/STATE while the Host published on spCv4.0/STATE, and would never see the Host come | ||
| online. | ||
|
|
||
| [[topics_topic_prefix_element]] | ||
| ==== topic_prefix Element | ||
|
|
||
|
|
@@ -324,7 +342,8 @@ definitions MUST be published in the NMETA.*# | |
| include a sequence number. When the NMETA is published at the start of a BIRTH sequence it is the | ||
| first message and its sequence number MUST be 0. When the NMETA is published as an update | ||
| (accompanying an NMMETA), its sequence number MUST be one greater than the sequence number of the | ||
| previous message from this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the | ||
| previous message from this Edge Node, unless that message contained a value of 9223372036854775807, | ||
| in which case the | ||
| sequence number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-nmeta-bdseq]#[yellow-background]*[tck-id-topics-nmeta-bdseq] The NMETA | ||
| MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# | ||
|
|
@@ -368,7 +387,8 @@ NMMETA MUST include only the metrics affected by the change, and each metric MUS | |
| of ADD, UPDATE, or DELETE.*# | ||
| * [tck-testable tck-id-topics-nmmeta-seq]#[yellow-background]*[tck-id-topics-nmmeta-seq] The NMMETA MUST | ||
| include a sequence number that is one greater than the sequence number of the previous message | ||
| published by this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the sequence | ||
| published by this Edge Node, unless that message contained a value of 9223372036854775807, in which | ||
| case the sequence | ||
| number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-nmmeta-bdseq]#[yellow-background]*[tck-id-topics-nmmeta-bdseq] The NMMETA | ||
| MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# | ||
|
|
@@ -433,7 +453,7 @@ immediately prior MQTT CONNECT packet's Will Message.*# | |
| matching bd_seq number allows Host Applications to correlate NBIRTHs to NDEATHs.*# | ||
| * [tck-testable tck-id-topics-nbirth-bdseq-increment]#[yellow-background]*[tck-id-topics-nbirth-bdseq-increment] The | ||
| bd_seq number MUST start at zero and increment by one on every new MQTT CONNECT packet until it | ||
| reaches a maximum of 18446744073709551615 (max value of a UINT64). At this point, the following | ||
| reaches a maximum of 9223372036854775807. At this point, the following | ||
| bd_seq number MUST be zero.*# | ||
|
|
||
| The NBIRTH message can also include additional Node Control payload components. These are used by a | ||
|
|
@@ -528,7 +548,7 @@ payload.*# | |
| ** This allows Host Applications to correlate NREBIRTHs to NDEATHs. | ||
| * [tck-testable tck-id-topics-nrebirth-bdseq-increment]#[yellow-background]*[tck-id-topics-nrebirth-bdseq-increment] The | ||
| bd_seq number MUST start at zero and increment by one on every new MQTT CONNECT packet until it | ||
| reaches a maximum of 18446744073709551615 (max value of a UINT64). At this point, the following | ||
| reaches a maximum of 9223372036854775807. At this point, the following | ||
| bd_seq number MUST be zero.*# | ||
|
|
||
| The NREBIRTH message can also include additional Node Control payload components. These are used by | ||
|
|
@@ -588,7 +608,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# | |
| * [tck-testable tck-id-topics-ndata-seq-num]#[yellow-background]*[tck-id-topics-ndata-seq-num] The | ||
| NDATA MUST include a sequence number in the payload and it MUST have a value of one greater than the | ||
| previous MQTT message from the Edge Node contained unless the previous MQTT message contained a | ||
| value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# | ||
| value of 9223372036854775807. In this case the sequence number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-ndata-timestamp-1]#[yellow-background]*[tck-id-topics-ndata-timestamp-1] The | ||
| NDATA MUST include the overall payload timestamp denoting the date and time the message was sent from | ||
| the Edge Node.*# | ||
|
|
@@ -750,7 +770,8 @@ metadata MAY also be included. Aliases are optional.*# | |
| DMETA MUST NOT be used to convey current metric values.*# | ||
| * [tck-testable tck-id-topics-dmeta-seq]#[yellow-background]*[tck-id-topics-dmeta-seq] The DMETA MUST | ||
| include a sequence number that is one greater than the sequence number of the previous message from | ||
| this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the sequence number MUST | ||
| this Edge Node, unless that message contained a value of 9223372036854775807, in which case the | ||
| sequence number MUST | ||
| be 0. A DMETA is never the first message of a BIRTH sequence, as it follows the NMETA and NBIRTH.*# | ||
| * [tck-testable tck-id-topics-dmeta-bdseq]#[yellow-background]*[tck-id-topics-dmeta-bdseq] The DMETA | ||
| MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# | ||
|
|
@@ -792,7 +813,8 @@ DMMETA MUST include only the metrics affected by the change, and each metric MUS | |
| of ADD, UPDATE, or DELETE.*# | ||
| * [tck-testable tck-id-topics-dmmeta-seq]#[yellow-background]*[tck-id-topics-dmmeta-seq] The DMMETA MUST | ||
| include a sequence number that is one greater than the sequence number of the previous message | ||
| published by this Edge Node, unless that message contained a value of 18446744073709551615 (max value of a UINT64), in which case the sequence | ||
| published by this Edge Node, unless that message contained a value of 9223372036854775807, in which | ||
| case the sequence | ||
| number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-dmmeta-bdseq]#[yellow-background]*[tck-id-topics-dmmeta-bdseq] The DMMETA | ||
| MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# | ||
|
|
@@ -834,7 +856,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# | |
| * [tck-testable tck-id-topics-dbirth-seq]#[yellow-background]*[tck-id-topics-dbirth-seq] The DBIRTH | ||
| MUST include a sequence number in the payload and it MUST have a value of one greater than the | ||
| previous MQTT message from the Edge Node contained unless the previous MQTT message contained a | ||
| value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# | ||
| value of 9223372036854775807. In this case the sequence number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-dbirth-timestamp-1]#[yellow-background]*[tck-id-topics-dbirth-timestamp-1] The | ||
| DBIRTH MUST include the overall payload timestamp denoting the date and time the message was sent | ||
| from the Edge Node.*# | ||
|
|
@@ -922,7 +944,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# | |
| * [tck-testable tck-id-topics-drebirth-seq]#[yellow-background]*[tck-id-topics-drebirth-seq] The DREBIRTH | ||
| MUST include a sequence number in the payload and it MUST have a value of one greater than the | ||
| previous MQTT message from the Edge Node contained unless the previous MQTT message contained a | ||
| value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# | ||
| value of 9223372036854775807. In this case the sequence number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-drebirth-timestamp-1]#[yellow-background]*[tck-id-topics-drebirth-timestamp-1] The | ||
| DREBIRTH MUST include the overall payload timestamp denoting the date and time the message was sent | ||
| from the Edge Node.*# | ||
|
|
@@ -1000,7 +1022,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# | |
| * [tck-testable tck-id-topics-ddata-seq-num]#[yellow-background]*[tck-id-topics-ddata-seq-num] The | ||
| DDATA MUST include a sequence number in the payload and it MUST have a value of one greater than the | ||
| previous MQTT message from the Edge Node contained unless the previous MQTT message contained a | ||
| value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# | ||
| value of 9223372036854775807. In this case the sequence number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-ddata-timestamp-1]#[yellow-background]*[tck-id-topics-ddata-timestamp-1] The | ||
| DDATA MUST include the overall payload timestamp denoting the date and time the message was sent from | ||
| the Edge Node.*# | ||
|
|
@@ -1046,7 +1068,7 @@ messages MUST be published with MQTT QoS equal to 0 and retain equal to false.*# | |
| * [tck-testable tck-id-topics-ddeath-seq-num]#[yellow-background]*[tck-id-topics-ddeath-seq-num] The | ||
| DDEATH MUST include a sequence number in the payload and it MUST have a value of one greater than | ||
| the previous MQTT message from the Edge Node contained unless the previous MQTT message contained a | ||
| value of 18446744073709551615 (max value of a UINT64). In this case the sequence number MUST be 0.*# | ||
| value of 9223372036854775807. In this case the sequence number MUST be 0.*# | ||
| * [tck-testable tck-id-topics-ddeath-bdseq]#[yellow-background]*[tck-id-topics-ddeath-bdseq] The | ||
| DDEATH MUST include a bd_seq number matching the bd_seq of the current MQTT session.*# | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Required: nothing says which namespace a Host uses to command an Edge Node. This only ties the encoding to the namespace the Host publishes on; the Edge Node subscribes to
<ns>/NCMD|DCMD/...and REBIRTH on its own namespace.A Host that supports both encodings (as it now must) and defaults to spCv4.0 sends a rebirth request or NCMD to an Edge Node that births on spJv4.0, and it's never delivered. Rebirth recovery silently fails for every JSON Edge Node.
Suggest: a Host MUST publish NCMD/DCMD/rebirth to an Edge Node on the namespace of that Edge Node's current NBIRTH, and an Edge Node MUST subscribe to its command topics on the namespace it publishes on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 1fcab1d. Agreed this was a real gap — the rule only covered which encoding a Host uses, not which namespace it addresses.
Two statements, one for each direction, close to your suggested wording:
conformance-host-encoding-address: when a Host Application publishes a message addressed to a particular Edge Node or Device, including an NCMD, a DCMD, and a Rebirth Request, it MUST publish on the namespace of that Edge Node's current NBIRTH.conformance-edge-node-encoding-subscribe: an Edge Node MUST subscribe to its command topics on the same namespace it publishes on.The non-normative comment records why it matters: the MQTT Server accepts the misaddressed message and it is simply never delivered, so a Rebirth Request fails silently and the Host cannot recover its view of that Edge Node.