Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ All significant changes to this project will be documented in this file.

## Unreleased

### New features

* Add immutable xor filters behind the `xor` feature, with 8- and 16-bit fingerprints, pre-hashed input APIs, and compatible serialization.

## v0.4.0 (2026-08-18)

### Breaking changes
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ Enable multiple algorithms by listing their features together, such as `features
| `tdigest` | `TDigestMut`, `TDigest` | Quantile and rank estimation, with high accuracy near distribution tails. |
| `theta` | `ThetaSketch` and set operations | Distinct counts, set expressions, and Jaccard similarity. |
| `tuple` | `TupleSketch` and set operations | Theta-style keys with user-defined summaries attached to retained entries. |
| `xor` | `XorFilter`, `XorFilterBuilder` | Compact immutable probabilistic set membership with 8- or 16-bit fingerprints. |

See the [API documentation](https://docs.rs/datasketches) for configuration, accuracy guarantees, serialization, and examples for each algorithm.

Expand Down
6 changes: 6 additions & 0 deletions datasketches/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ hll = []
tdigest = []
theta = []
tuple = []
xor = []

[[test]]
name = "bloom_test"
Expand Down Expand Up @@ -87,6 +88,11 @@ name = "tuple_test"
path = "tests/tuple_test/main.rs"
required-features = ["tuple"]

[[test]]
name = "xor_test"
path = "tests/xor_test/main.rs"
required-features = ["xor"]

[dev-dependencies]
googletest = { workspace = true }
insta = { workspace = true }
Expand Down
9 changes: 9 additions & 0 deletions datasketches/src/codec/family.rs
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,15 @@ impl Family {
min_pre_longs: 3,
max_pre_longs: 4,
};

/// Xor filter for probabilistic set membership.
#[cfg(feature = "xor")]
pub const XORFILTER: Family = Family {
id: 22,
name: "XORFILTER",
min_pre_longs: 3,
max_pre_longs: 3,
};
}

impl Family {
Expand Down
2 changes: 2 additions & 0 deletions datasketches/src/codec/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ pub use self::encode::SketchBytes;
feature = "tdigest",
feature = "theta",
feature = "tuple",
feature = "xor",
))]
#[allow(dead_code)] // some utilities are only used for certain sketches
pub(crate) mod assert;
Expand All @@ -44,5 +45,6 @@ pub(crate) mod assert;
feature = "tdigest",
feature = "theta",
feature = "tuple",
feature = "xor",
))]
pub(crate) mod family;
5 changes: 3 additions & 2 deletions datasketches/src/hash/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@ mod murmurhash;
))]
pub(crate) use self::murmurhash::*;

#[cfg(feature = "bloom")]
#[cfg(any(feature = "bloom", feature = "xor"))]
mod xxhash;
#[cfg(feature = "bloom")]
#[cfg(any(feature = "bloom", feature = "xor"))]
pub(crate) use self::xxhash::*;

#[cfg(any(
Expand Down Expand Up @@ -95,6 +95,7 @@ pub(crate) const DEFAULT_UPDATE_SEED: u64 = 9001;
feature = "hll",
feature = "theta",
feature = "tuple",
feature = "xor",
))]
fn read_u64_le(bytes: &[u8]) -> u64 {
let mut buf = [0u8; 8];
Expand Down
2 changes: 2 additions & 0 deletions datasketches/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ pub use self::thetafamily::common as thetacommon;
pub use self::thetafamily::theta;
#[cfg(feature = "tuple")]
pub use self::thetafamily::tuple;
#[cfg(feature = "xor")]
pub mod xor;

// common modules
pub mod codec;
Expand Down
Loading