vortex-data/vortex

Duckdb has nested struct accessors like x.y.z. This generally produces a get_item expression which we handle in filters, so WHERE x.y.z = 1 will not read x into memory, but will read only x.y.z. However, projection pushdown doesn't support this, so SELECT x.y.z reads x. Duckdb has another mechanism to optimize this which is a TableFunction callback supports_pushdown_extract. If you define it, you get special ColumnIndex with type STRUCT_EXTRACT. In order to support this, we need to: Teach Projection::new and Filter::new how to handle extract column indices. We need to convert these to a chain of get_item expressions. We also need to generate unique names when packing the output struct. Handle casts inside struct extract columns if extracted column type differs from a LogicalType for this column. Return no stats for extracted columns since Vortex doesn't support nested stats (yet). Forbid struct extract columns in aggregation (preferred) or change aggregation optimizer pass to support

DATE_TRUNC is evaluated entirely by the query engine today. It appears in ClickBench Q42 and SpatialBench Q3/Q5. Vortex should support it so engines can push it into the scan and prune zones with it. Approach: function plus comparison rewrite date_trunc(u, ts) is a step function that never decreases, so comparing it with a constant is exactly a range check on ts. The bounds are computed once, at planning time: date_trunc(u, ts) = X -> ts >= X AND ts < next(X) (X aligned to u) -> false, null for null ts (X not aligned) date_trunc(u, ts) < X -> ts < ceil(X) date_trunc(u, ts) <= X -> ts < next(floor(X)) date_trunc(u, ts) >= X -> ts >= ceil(X) date_trunc(u, ts) > X -> ts >= next(floor(X)) IN / <> -> OR of ranges / negated range Filters against a constant use the rewrite. Pruning comes for free from the existing min/max rules. Comparisons are cheaper than truncation and can run on compressed data. Zoned timestamps a

scalar_at receives only an ArrayView, so it doesn't cache anything — validity, child wrappers and any decode are rebuilt on every row. probe_scalar receives a ProbeState, which carries the encoding's own retained state plus a child probe per slot, so a repeated read keeps its preparation. Example migration: #9905 Writing a probe_scalar Cache in type ProbeState anything the encoding owns itself — a decoded page, a rebuilt array, a cursor. Use () when there's nothing. Don't cache anything owned by a child. Read children through state.slot(..), which hands back that child's own probe and so uses the child's cache. Leave scalar_at as Self::probe_scalar(&mut ProbeState::once(array), index, ctx) until the vtable method is removed. Encodings to migrate Bool Constant Decimal Filter FixedSizeList Interleave List ListView Map Null Patched PiecewiseSequence ScalarFn Union VarBin VarBinView Variant ALPRD BitPacked ByteBool DecimalByteParts Delta OnPair ParquetVariant

We have a different bloom filter type so we need to carefully extract duckdb's parameters.

We support function pushdown from #7727. In our benchmarks, we also have prefix/starts_with and suffix/ends_with. We should push down these functions as well. For that we need to implement Vortex's prefix/suffix/substring expressions. Our contains() function is pushed down as %pattern% (LIKE) which has bad performance. We should have a separate expression for that as well.