hardwood-hq/hardwood

Summary The four Filter-category JFR events report how much a filter skipped, but not which filter did it. In a process that runs many different reads, the counts cannot be attributed: Event Reports Missing dev.hardwood.RowGroupFilter row groups dropped by statistics/bloom push-down which predicate dev.hardwood.PageFilter pages dropped by the Column Index which predicate dev.hardwood.RecordFilter records dropped by per-record evaluation which predicate dev.hardwood.RowGroupByteRangeFilter row groups excluded by split selection which range Thread and timestamp are the only correlation available today, and nothing else in the recording names the predicate, so there is nothing to join them to. Proposal: add a predicate field to each, carrying a structural rendering with the literals elided: and(gt(id, ?), eq(label, ?)) or(isNull(city), lt(zip, ?)) Why literals are elided Two reasons, and they point the same way. Sensitivity. Every field these events carry today is schema

ParquetFileWriter.columnOrders() returns one TYPE_DEFINED_ORDER per leaf column and FileMetaDataWriter writes the list when it is non-empty, so produced files carry it. Nothing checks that they do. It is worth pinning because of what depends on it rather than because it looks fragile. A reader only trusts min_value / max_value under the order the footer declares; with column_orders absent, parquet-java falls back to the deprecated min / max semantics, under which a BYTE_ARRAY bound is compared signed. Hardwood writes STRING bounds unsigned and DECIMAL bounds signed, both correct under TYPE_DEFINED_ORDER — so a file that lost the list would have its string statistics silently misread by every other implementation, and every existing statistics test would still pass, since they all read the file back through Hardwood's own reader, which does not need the list to know its own conventions. That is the shape of bug the interop gate exists to catch and currently cannot. What to do Two asse

On the hardwood dive column chunk detail screen, the Storage group shows Size statistics chunk + 2 pages while the Drill into menu on the same frame already shows ▶ Pages 2 pages Column index present · levels Offset index present · unencoded The three menu entries say everything the row does: how many pages there are, that the column index carries per-page level histograms, and that the offset index carries per-page unencoded sizes. That the chunk itself has a SizeStatistics is in turn implied by the populated Unencoded, Present and Records rows a few lines above. _designs/DIVE_SIZE_STATISTICS.md states the rule this violates — dive drops rows the screen already carries, which is why Present and Records are hidden for a column that cannot be null or repeat. Worth keeping is the degraded form, — (not written): that one says something the menu does not, since a chunk with no SizeStatistics still has a page index and vice versa. So the change is to

Problem A parity pass over dive's screens against the inspect commands turned up several facts that dive shows and the commands do not. Each is a small additive change; they are grouped here rather than filed individually. Missing from Fact Where dive shows it inspect pages Page byte offset in the file OffsetIndexScreen.java:110 — #, Offset, Size, First row inspect pages Uncompressed size per page PagesScreen.java:213 has both Comp and Uncomp; the command has Compressed only inspect pages The column index null_pages flag ColumnIndexScreen.java:203 — Null page column inspect rowgroups Column-index / offset-index presence per row group RowGroupsScreen.java:113 — CI, OI columns inspect rowgroups Logical type alongside the physical type ColumnChunksScreen.java:103 — Logical column inspect rowgroups One column across all row groups, with stats ColumnAcrossRowGroupsScreen.java:135 — RG, Rows, Pages, Comp, Ratio, Dict, CI, Nulls, Min, Max The last one is the only

Problem hardwood footer and dive's Footer screen share a name and almost no content. The command prints five lines (FooterCommand.java:72-76): file size, footer offset, footer length, and the two magic markers. Dive's FooterScreen renders considerably more from the same footer (FooterScreen.java:289-344): File size, format version, created-by Footer trailer offset and trailer byte count The data region vs footer+indexes byte split Encoding histogram across the file Codec histogram across the file Index coverage: how many chunks carry a column index, an offset index, a bloom filter Dictionary coverage: how many chunks have a dictionary page Aggregates: compressed, uncompressed, compression ratio None of that is reachable non-interactively. Bloom-filter presence in particular is not surfaced by any command at all — a grep of cli/command/ for loom returns nothing — even though it decides whether eq/in pushdown can use one. The aggregates are cheap: everything above is derived from f