Skip to content

[5.3] Fix DB2Decoder NPE when chained DSS response is split across TCP segments - #1716

Merged
tsegismont merged 1 commit into
eclipse-vertx:masterfrom
DavideD:899-main
Oct 2, 2026
Merged

tsegismont merged 1 commit into
eclipse-vertx:masterfrom
DavideD:899-main

Conversation

@DavideD

@DavideD DavideD commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #899

A DRDA response from DB2 can consist of multiple chained DSS (Data Stream Structure) segments, linked via a chain bit (0x40) in byte 3 of each segment's header. The computeLength() method is supposed to calculate the total length of all chained segments before the decoder processes them as a single logical response.

Bug: When TCP delivers only the first segment(s) of a chained response (with the chain bit set) but the next chained segment has not arrived yet, computeLength() returns the partial length (equal to readableBytes). Since decode() only waits when payloadLength > readableBytes, it proceeds to call decodePayload() with incomplete data. The command handler processes the partial response and either:

  1. Calls ensureALayerDataInBuffer() to read the next DSS header but the buffer is empty: "Needed to have 6 in buffer but only had 0"

  2. Calls fireCommandSuccess()/fireCommandFailure() which polls the command from the inflight queue. When the remaining chained segments arrive, inflight.peek() returns the WRONG command: "Invalid correlator ID. Got 2 expected 1"

  3. Or inflight.peek() returns null: NullPointerException in decodePayload()

  4. Leftover DSS data from command A is fed to command B's handler, which encounters unexpected DRDA codepoints: "Found unknown codepoint: 0x2411"

The crash sequence:

  1. TCP delivers DSS_1 (chain bit set), DSS_2 not yet received
  2. computeLength() returns len(DSS_1) == readableBytes
  3. decode() proceeds (payloadLength <= readableBytes)
  4. Command processes partial data, polls itself from inflight
  5. DSS_2 arrives -> dispatched to wrong command or null -> error

This is intermittent because TCP segmentation is non-deterministic - whether the OS delivers all DSS segments in one read() depends on timing, buffer sizes, and network conditions.

Fix: After the while loop, check if dssContinues is still true (meaning the chain bit was set on the last DSS we read but the next segment hasn't arrived). If so, return readableBytes + 1 to force decode() to wait for more data.

Also fix a secondary bug: use getUnsignedShort() instead of getShort() for reading DSS lengths. The DRDA spec defines DSS lengths as unsigned 16-bit values (0-65535), but getShort() returns a signed Java short (-32768 to 32767), which would produce negative values for segments larger than 32767 bytes.

Motivation:

Explain here the context, and why you're making that change, what is the problem you're trying to solve.

Conformance:

You should have signed the Eclipse Contributor Agreement as explained in https://github.com/eclipse/vert.x/blob/master/CONTRIBUTING.md
Please also make sure you adhere to the code style guidelines: https://github.com/vert-x3/wiki/wiki/Vert.x-code-style-guidelines

…ents

Fixes eclipse-vertx#899

A DRDA response from DB2 can consist of multiple chained DSS (Data
Stream Structure) segments, linked via a chain bit (0x40) in byte 3 of
each segment's header. The computeLength() method is supposed to
calculate the total length of all chained segments before the decoder
processes them as a single logical response.

Bug: When TCP delivers only the first segment(s) of a chained response
(with the chain bit set) but the next chained segment has not arrived
yet, computeLength() returns the partial length (equal to
readableBytes). Since decode() only waits when payloadLength >
readableBytes, it proceeds to call decodePayload() with incomplete
data. The command handler processes the partial response and either:

  1. Calls ensureALayerDataInBuffer() to read the next DSS header but
     the buffer is empty:
     "Needed to have 6 in buffer but only had 0"

  2. Calls fireCommandSuccess()/fireCommandFailure() which polls the
     command from the inflight queue. When the remaining chained
     segments arrive, inflight.peek() returns the WRONG command:
     "Invalid correlator ID. Got 2 expected 1"

  3. Or inflight.peek() returns null:
     NullPointerException in decodePayload()

  4. Leftover DSS data from command A is fed to command B's handler,
     which encounters unexpected DRDA codepoints:
     "Found unknown codepoint: 0x2411"

The crash sequence:
1. TCP delivers DSS_1 (chain bit set), DSS_2 not yet received
2. computeLength() returns len(DSS_1) == readableBytes
3. decode() proceeds (payloadLength <= readableBytes)
4. Command processes partial data, polls itself from inflight
5. DSS_2 arrives -> dispatched to wrong command or null -> error

This is intermittent because TCP segmentation is non-deterministic -
whether the OS delivers all DSS segments in one read() depends on
timing, buffer sizes, and network conditions.

Fix: After the while loop, check if dssContinues is still true (meaning
the chain bit was set on the last DSS we read but the next segment
hasn't arrived). If so, return readableBytes + 1 to force decode() to
wait for more data.

Also fix a secondary bug: use getUnsignedShort() instead of getShort()
for reading DSS lengths. The DRDA spec defines DSS lengths as unsigned
16-bit values (0-65535), but getShort() returns a signed Java short
(-32768 to 32767), which would produce negative values for segments
larger than 32767 bytes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@DavideD DavideD changed the title Fix DB2Decoder NPE when chained DSS response is split across TCP segments [5.3] Fix DB2Decoder NPE when chained DSS response is split across TCP segments Oct 2, 2026
@tsegismont tsegismont added this to the 5.3.0 milestone Oct 2, 2026

@tsegismont tsegismont left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @DavideD !

@tsegismont
tsegismont merged commit cf5acd8 into eclipse-vertx:master Oct 2, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Needed to have 6 in buffer but only had 0"

2 participants