apache/datasketches-rust

Description REQ deserialization currently accepts compactor states that cannot be produced by a valid sketch. Those states can later panic in safe public APIs or violate documented return-value contracts. Two concrete examples on current main: A single-level image with num_sections = 0 deserializes successfully. A subsequent ReqSketch::merge panics at 1u64 << (self.num_sections - 1) because the subtraction underflows. A single-level image whose level-0 compactor declares lg_weight = 63 and contains one item deserializes successfully. Its inclusive rank for that item is approximately 9.22e18, even though ReqSketch::rank promises a result in [0.0, 1.0]. Relevant code: Compactor::deserialize validates only broad section_size_raw and lg_weight bounds ensure_enough_sections assumes num_sections is nonzero ReqSketch::deserialize accepts the reconstructed compactors without validating cross-field invariants Expected fix Validate wire-controlled compactor state before constructing a ReqSke

Good point. I wonder if we'd have a tracking issue to impl fmt::Display for all sketches when applicible. This is what datasketches-java does IMO, but the format can be considered once more. cc @notfilippo @ariesdevil @ZENOTME Originally posted by @tisonkun in #204 (comment)

ThetaIntersection::to_sketch and TupleIntersection::to_sketch currently panic when called before the first update. Both operators expose has_result, and tests explicitly cover the panic, but absence of an intersection state is part of the public state machine and deserves a deliberate API contract. We should decide whether result extraction should: return Option when no result is a normal state; return Result when calling before update is considered recoverable misuse; or retain the panic, with a clear invariant and local Panics documentation. Points to settle: Apply the same contract to Theta and Tuple intersections. Decide whether has_result remains useful after result extraction becomes typed. Define the mathematical meaning of zero updates; the identity for intersection cannot be represented as a finite sketch. Consider reset if it is added to the operators later. Check Java and C++ precedent, while keeping the Rust API misuse-resistant. Document migration if the return type cha

This issue supersedes #10 to squash the historical discussion. Currently, the C++/Go/Java implementations of DataSketches provide snapshot generators for sketches' serialization data. You can check the following scripts for how their snapshots get generated: https://github.com/apache/datasketches-tck/blob/d363b12d/serialization/cpp/generator.py https://github.com/apache/datasketches-tck/blob/d363b12d/serialization/go/generator.py https://github.com/apache/datasketches-tck/blob/d363b12d/serialization/java/generator.py We may provide a binary in datasketches-rust as well to generate snapshots for supported sketches' serialization data. I can imagine it as a local workspace member depending on datasketches and we will have a cargo x task to invoke the generator for automation. Or else, being part of the serde_tests but write the Rust sketch's serialization data to a specific location.