Skip to content

Map points, ids and vectors onto Qdrant's protobuf messages - #60

Merged
tonytonycoder11 merged 1 commit into
mainfrom
feat/m31-grpc-point-mapping
Jul 31, 2026
Merged

Map points, ids and vectors onto Qdrant's protobuf messages#60
tonytonycoder11 merged 1 commit into
mainfrom
feat/m31-grpc-point-mapping

Conversation

@tonytonycoder11

Copy link
Copy Markdown
Contributor

Third slice of #52, after #54 and #58. Points, ids and vectors — the mapping where the two models stop
lining up.

The three places they disagree

  • A point id is a uint64 and Java has no unsigned long. ULong.MAX_VALUE goes out as -1 and
    comes back as ULong.MAX_VALUE. That is correct and looks wrong, so the test asserts both halves
    rather than only that the round trip closes — otherwise a symmetric bug would pass.
  • VectorData.Named is recursive; the wire form is not. map<string, Vector> holds one dense,
    sparse or multi variant per entry, with no way to nest another map. Flattening it would store a
    different collection than the caller described, so it is refused and the message names the offending
    key.
  • VectorData.Raw has no equivalent. It exists so the REST engine can carry a JSON shape this
    client does not model yet, which is an escape hatch protobuf does not have.

A trap worth naming

Writes send Vector; reads come back as VectorOutput, a different message with the inference
variants (Document, Image, InferenceObject) dropped, because a server never echoes back a request
to embed something. The round-trip test deliberately goes out as one and back as the other, since
assuming they are interchangeable is exactly the mistake to catch at this layer rather than in the
transport.

Smaller decisions

  • An empty payload map reads back as no payload, not an empty object. Absence and emptiness are
    different answers to "what is stored on this point".
  • A vector missing from a VectorOutput maps to null rather than an empty dense vector — normal for a
    named-vector collection where only some vectors were written.
  • An id Qdrant sends with no variant set throws. That is a broken response, and defaulting it to zero
    would attach the payload to the wrong point.

Thirteen tests. Still no QdrantTransport implementation: filters are the last mapping before the
operations become mechanical.

The second mapping the gRPC transport needs, and the one where the two models
stop lining up.

A point id is a uint64 on the wire, and Java has no unsigned long. ULong.MAX_VALUE
therefore goes out as -1 and comes back as ULong.MAX_VALUE, which is correct and
looks wrong, so there is a test that asserts both halves rather than only that the
round trip closes.

A named vector map is map<string, Vector> and each entry carries its own dense,
sparse or multi variant. Kdrant's VectorData.Named is recursive and can hold
another Named, which has no wire form at all. Flattening it would store a
different collection than the caller described, so it is refused and the message
names the offending key.

VectorData.Raw is refused too. It exists so the REST engine can carry a shape this
client does not model yet, which is a JSON escape hatch and not something protobuf
has an equivalent for.

Reads come back as VectorOutput, a separate message from Vector with the inference
variants dropped, since a server never echoes back a request to embed a document.
The round-trip test goes out as one and back as the other on purpose, because
assuming they are interchangeable is exactly the mistake to catch here.

An empty payload map reads back as no payload rather than as an empty object.
Absence and emptiness are different answers to "what is stored on this point".
@tonytonycoder11
tonytonycoder11 requested a review from a team as a code owner July 31, 2026 11:58
@tonytonycoder11
tonytonycoder11 merged commit 91b4bd5 into main Jul 31, 2026
6 checks passed
@tonytonycoder11
tonytonycoder11 deleted the feat/m31-grpc-point-mapping branch July 31, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant