Skip to content

RUBY-3903 Add QE text queries (prefix/suffix/substring) - #3088

Merged
comandeo-mongo merged 15 commits into
mongodb:masterfrom
comandeo-mongo:3903-qe-text-queries
Jul 29, 2026
Merged

RUBY-3903 Add QE text queries (prefix/suffix/substring)#3088
comandeo-mongo merged 15 commits into
mongodb:masterfrom
comandeo-mongo:3903-qe-text-queries

Conversation

@comandeo-mongo

@comandeo-mongo comandeo-mongo commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

RUBY-3903

Adds Queryable Encryption text query support (GA prefix, suffix, and
substring) to the Ruby driver via the new String algorithm, and fixes
pre-existing QE explicit-encryption tests that break on server 9.0.

This PR covers three tickets:

  • RUBY-3903 — QE text queries (prefix/suffix/substring GA).
  • RUBY-3875 — QE case/diacritic sensitivity for explicit encryption
    (unblocked by the libmongocrypt bump; covered by the new prose Cases 8–11).
  • RUBY-3900 — QE range/equality explicit-encryption tests failing on the
    latest (9.0) server.

QE text queries (RUBY-3903)

  • Bump libmongocrypt-helper to ~> 1.20.1 (substring GA needs MONGOCRYPT-936).
  • Add FFI binding mongocrypt_ctx_setopt_algorithm_text and the
    ctx_setopt_algorithm_text wrapper, mirroring the existing range binding.
  • Support the String algorithm in ExplicitEncryptionContext: it accepts
    contention_factor, query_type (prefix/suffix/substring), and a new
    :string_opts option, converted from snake_case to the spec's camelCase BSON
    the same way range_opts is handled.
  • Copy the GA QE-Text unified spec fixtures (prefix, suffix, substring, plus
    compact/cleanupStructuredEncryptionData).
  • Add the "String Explicit Encryption" prose test (spec README section 27), all
    11 cases.

Notes:

  • The public option is :string_opts, matching the spec's GA stringOpts field.
  • Preview query types (prefixPreview/suffixPreview/substringPreview) are
    intentionally not implemented; this targets server 9.0+.

Case/diacritic sensitivity (RUBY-3875)

DRIVERS-3470 was a libmongocrypt bug, blocked on a libmongocrypt release. The
1.20.1 bump lifts that block, the driver already passes case_sensitive/
diacritic_sensitive through to stringOpts, and prose Cases 8–11 (regression
tests for DRIVERS-3470) verify case- and diacritic-insensitive queries return
the expected documents.

Range/equality on server 9.0 (RUBY-3900)

Server 9.0 enforces that explicit-encryption payloads match the collection's
configured contention and index bounds. Previously untested on 9.0, these broke
once the FLE suite started running against the latest server (below):

  • Refresh the range encryptedFields fixtures to include contention and
    trimFactor (they were stale and omitted both).
  • Fix the Date range test to use epoch-millisecond bounds matching the fixture
    (indexMin/indexMax), instead of Time.new(year).
  • Use a contention-10 collection for the non-zero contention case and align it
    with the spec prose test (insert 10 at cf 10, find once at cf 10).

CI

Enable an FLE test variant against the latest server (currently 9.0) on PRs, so
QE text query and other QE tests get real coverage instead of being skipped.
Adds a latest value to the mongodb-version axis and activates the previously
commented-out fle-latest matrix. The generated .evergreen/config.yml is
regenerated from the ERB templates.

Testing

Verified against a local 9.0 cluster with libmongocrypt 1.20.1 and the 9.0
mongocryptd:

  • QE-Text unified fixtures and the String prose test (11 cases) pass.
  • explicit_encryption_context_spec.rb (local KMS, no server) passes, exercising
    the real libmongocrypt binding for prefix/suffix/substring option conversion.
  • Range prose, explicit queryable encryption, and String prose specs together:
    74 examples, 0 failures (8 pending are topology-gated Decimal cases).

Server 9.0 enforces that explicit-encryption payloads match the collection's
configured contention and index bounds. Refresh the range encryptedFields
fixtures to include contention and trimFactor, fix the Date test to use
epoch-millisecond bounds matching the fixture, and use a contention-10
collection for the non-zero contention case (aligned with the spec prose test).
The unified test runner must drop enxcol_.<coll>.esc/.ecoc alongside the data
collection so QE __safeContent__ tag counts are reproducible for exact matching,
as required by the unified test format spec. Without this the QE-Text matching
query fixtures fail on __safeContent__ comparison.
Import the fle2v2-Rangev2-* unified fixtures (QE range auto encryption; ~45
files) which had no coverage in the driver. Fixing them required three unified
runner corrections mandated by the unified test format but previously missing:

- Parse fixtures in BSON mode so $numberLong/$date keep their types
  (server rejects a range field whose min/max type differs from the field type).
- Case-insensitive errorContains matching (spec requires it).
- Compare numbers by value across BSON integer widths (Int64 vs Integer).

Full unified-suite regression: these fixes reduce CSE unified failures from 72
to 40 (remaining are cloud-KMS-credential env failures) with no regressions;
non-CSE unified suites unchanged.
The resynced range encryptedFields fixtures configure trimFactor 1, so
the explicit RangeOpts must match; otherwise libmongocrypt uses its
default of 6 and the server rejects the payload.
@comandeo-mongo
comandeo-mongo marked this pull request as ready for review July 28, 2026 09:23
@comandeo-mongo
comandeo-mongo requested a review from a team as a code owner July 28, 2026 09:23
@comandeo-mongo
comandeo-mongo requested review from Copilot and jamis and removed request for Copilot July 28, 2026 09:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Queryable Encryption (QE) text query support to the Ruby driver by introducing explicit-encryption support for the GA String algorithm (prefix/suffix/substring), refreshing QE range fixtures/tests for MongoDB 9.0 enforcement, and enabling CI coverage against the “latest” server in the FLE test matrix.

Changes:

  • Add libmongocrypt FFI + driver plumbing for explicit encryption with the String algorithm (prefix/suffix/substring + string_opts).
  • Add/refresh QE fixtures and prose/unified tests for text queries and updated range/equality requirements on server 9.0.
  • Enable an Evergreen “fle-latest” PR buildvariant and add a latest MongoDB version axis value.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
spec/support/crypt/encrypted_fields/range-encryptedFields-Long.json Refreshes range encryptedFields fixture to include contention/trimFactor for server 9.0 compatibility.
spec/support/crypt/encrypted_fields/range-encryptedFields-Int.json Same as above for int range fixture.
spec/support/crypt/encrypted_fields/range-encryptedFields-DoublePrecision.json Same as above for double range fixture (with precision).
spec/support/crypt/encrypted_fields/range-encryptedFields-DoubleNoPrecision.json Same as above for double range fixture (no precision).
spec/support/crypt/encrypted_fields/range-encryptedFields-DecimalPrecision.json Adds contention/trimFactor for decimal range fixture (with precision).
spec/support/crypt/encrypted_fields/range-encryptedFields-DecimalNoPrecision.json Adds contention/trimFactor for decimal range fixture (no precision).
spec/support/crypt/encrypted_fields/range-encryptedFields-Date.json Adds contention/trimFactor and fixes date range bounds representation.
spec/support/crypt/encrypted_fields/encryptedFields-substring.json New encryptedFields fixture for substring QE text queries.
spec/support/crypt/encrypted_fields/encryptedFields-substring-ci-di.json New encryptedFields fixture for case/diacritic-insensitive substring indexing.
spec/support/crypt/encrypted_fields/encryptedFields-prefix-suffix.json New encryptedFields fixture for prefix/suffix QE text queries.
spec/support/crypt/encrypted_fields/encryptedFields-prefix-suffix-ci-di.json New encryptedFields fixture for case/diacritic-insensitive prefix/suffix indexing.
spec/support/crypt/encrypted_fields/encryptedFields-c10.json New encryptedFields fixture for contention=10 collection used by explicit QE tests.
spec/spec_tests/data/client_side_encryption/unified/QE-Text-suffixPreview.yml Adds unified QE text suffix preview fixture (server 8.2–8.x).
spec/spec_tests/data/client_side_encryption/unified/QE-Text-suffix.yml Adds unified QE text suffix GA fixture (server 9.0+).
spec/spec_tests/data/client_side_encryption/unified/QE-Text-substringPreview.yml Adds unified QE text substring preview fixture (server 8.2–8.x).
spec/spec_tests/data/client_side_encryption/unified/QE-Text-substring.yml Adds unified QE text substring GA fixture (server 9.0+).
spec/spec_tests/data/client_side_encryption/unified/QE-Text-prefixPreview.yml Adds unified QE text prefix preview fixture (server 8.2–8.x).
spec/spec_tests/data/client_side_encryption/unified/QE-Text-prefix.yml Adds unified QE text prefix GA fixture (server 9.0+).
spec/spec_tests/data/client_side_encryption/unified/QE-Text-compactStructuredEncryptionData.yml Adds unified fixture for compactStructuredEncryptionData with QE text.
spec/spec_tests/data/client_side_encryption/unified/QE-Text-cleanupStructuredEncryptionData.yml Adds unified fixture for cleanupStructuredEncryptionData with QE text.
spec/runners/unified/test.rb Ensures QE metadata collections (esc/ecoc) are dropped during initialData setup for reproducible safeContent.
spec/mongo/crypt/explicit_encryption_context_spec.rb Extends unit coverage for explicit encryption context initialization with String algorithm and string_opts.
spec/integration/client_side_encryption/string_explicit_encryption_prose_spec.rb Adds prose integration tests for GA String explicit encryption (11 cases).
spec/integration/client_side_encryption/range_explicit_encryption_prose_spec.rb Aligns range prose tests with updated fixtures (trimFactor + date bounds in epoch millis).
spec/integration/client_side_encryption/explicit_queryable_encryption_spec.rb Updates explicit QE tests for server 9.0 contention enforcement (adds contention-10 collection).
lib/mongo/crypt/explicit_encryption_context.rb Adds String algorithm handling, string_opts conversion, and hooks to set text algorithm options.
lib/mongo/crypt/binding.rb Adds FFI binding and wrapper for mongocrypt_ctx_setopt_algorithm_text.
lib/mongo/client_encryption.rb Updates ClientEncryption#encrypt documentation to include String text query options.
gemfiles/standard.rb Bumps libmongocrypt-helper to ~> 1.20.1 for substring GA support.
.evergreen/config/standard.yml.erb Enables a PR-tagged “fle-latest” buildvariant for latest-server QE coverage.
.evergreen/config/axes.yml.erb Adds a latest value to the MongoDB version axis.
.evergreen/config.yml Regenerates Evergreen config with the new latest axis value and enabled fle-latest matrix.
Comments suppressed due to low confidence (1)

spec/support/crypt/encrypted_fields/encryptedFields-prefix-suffix-ci-di.json:35

  • Same as above: encode contention using canonical Extended JSON ($numberLong) rather than a bare JSON number to avoid BSON int32/int64 ambiguity.
            "$numberInt": "10"
          },
          "contention": 0,
          "caseSensitive": false,
          "diacriticSensitive": false

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/mongo/crypt/explicit_encryption_context.rb Outdated
Comment on lines 104 to +107
# @option options [ Integer | nil ] :contention_factor Contention factor
# to be applied if encryption algorithm is set to "Indexed". If not
# provided, it defaults to a value of 0. Contention factor should be set
# only if encryption algorithm is set to "Indexed".
# to be applied if encryption algorithm is set to "Indexed" or "String".
# If not provided, it defaults to a value of 0. Contention factor should be
# set only if encryption algorithm is set to "Indexed" or "String".
Comment on lines 108 to +111
# @option options [ String | nil ] query_type Query type to be applied
# if encryption algorithm is set to "Indexed". Query type should be set
# only if encryption algorithm is set to "Indexed". The only allowed
# value is "equality".
# if encryption algorithm is set to "Indexed" or "String". Allowed values
# are "equality" (for "Indexed") and "prefix", "suffix", "substring"
# (for "String").
Comment thread spec/support/crypt/encrypted_fields/encryptedFields-substring.json
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@comandeo-mongo
comandeo-mongo merged commit e21cb9f into mongodb:master Jul 29, 2026
220 checks passed
@comandeo-mongo
comandeo-mongo deleted the 3903-qe-text-queries branch July 29, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants