hardwood-hq/hardwood

The Data preview title reports the absolute row range and the visible column window: Data preview (rows 1–38 of 1,000,000 · cols 1–7 of 19) What it does not report is which row group the visible rows come from. That is the one piece of structural context the screen is missing: everywhere else in dive (Row groups, Column chunks, Pages, Column index) the row group is the organising unit, and when you page through the data you lose track of where you are relative to it. It also hides the boundaries — a page of rows can straddle two row groups and nothing on screen says so, even though that boundary is exactly what explains a jump in decode latency, a change in encoding, or a discontinuity in the values of a clustered column. Proposal Extend the title with the row group the cursor is in, e.g. Data preview (rows 1–38 of 1,000,000 · RG 1 of 24 · cols 1–7 of 19) Details to settle during implementation: The row group should follow the cursor (firstRow + selectedRow), not the

Summary The four Filter-category JFR events report how much a filter skipped, but not which filter did it. In a process that runs many different reads, the counts cannot be attributed: Event Reports Missing dev.hardwood.RowGroupFilter row groups dropped by statistics/bloom push-down which predicate dev.hardwood.PageFilter pages dropped by the Column Index which predicate dev.hardwood.RecordFilter records dropped by per-record evaluation which predicate dev.hardwood.RowGroupByteRangeFilter row groups excluded by split selection which range Thread and timestamp are the only correlation available today, and nothing else in the recording names the predicate, so there is nothing to join them to. Proposal: add a predicate field to each, carrying a structural rendering with the literals elided: and(gt(id, ?), eq(label, ?)) or(isNull(city), lt(zip, ?)) Why literals are elided Two reasons, and they point the same way. Sensitivity. Every field these events carry today is schema

ParquetFileWriter.columnOrders() returns one TYPE_DEFINED_ORDER per leaf column and FileMetaDataWriter writes the list when it is non-empty, so produced files carry it. Nothing checks that they do. It is worth pinning because of what depends on it rather than because it looks fragile. A reader only trusts min_value / max_value under the order the footer declares; with column_orders absent, parquet-java falls back to the deprecated min / max semantics, under which a BYTE_ARRAY bound is compared signed. Hardwood writes STRING bounds unsigned and DECIMAL bounds signed, both correct under TYPE_DEFINED_ORDER — so a file that lost the list would have its string statistics silently misread by every other implementation, and every existing statistics test would still pass, since they all read the file back through Hardwood's own reader, which does not need the list to know its own conventions. That is the shape of bug the interop gate exists to catch and currently cannot. What to do Two asse

On the hardwood dive column chunk detail screen, the Storage group shows Size statistics chunk + 2 pages while the Drill into menu on the same frame already shows ▶ Pages 2 pages Column index present · levels Offset index present · unencoded The three menu entries say everything the row does: how many pages there are, that the column index carries per-page level histograms, and that the offset index carries per-page unencoded sizes. That the chunk itself has a SizeStatistics is in turn implied by the populated Unencoded, Present and Records rows a few lines above. _designs/DIVE_SIZE_STATISTICS.md states the rule this violates — dive drops rows the screen already carries, which is why Present and Records are hidden for a column that cannot be null or repeat. Worth keeping is the degraded form, — (not written): that one says something the menu does not, since a chunk with no SizeStatistics still has a page index and vice versa. So the change is to

Problem A parity pass over dive's screens against the inspect commands turned up several facts that dive shows and the commands do not. Each is a small additive change; they are grouped here rather than filed individually. Missing from Fact Where dive shows it inspect pages Page byte offset in the file OffsetIndexScreen.java:110 — #, Offset, Size, First row inspect pages Uncompressed size per page PagesScreen.java:213 has both Comp and Uncomp; the command has Compressed only inspect pages The column index null_pages flag ColumnIndexScreen.java:203 — Null page column inspect rowgroups Column-index / offset-index presence per row group RowGroupsScreen.java:113 — CI, OI columns inspect rowgroups Logical type alongside the physical type ColumnChunksScreen.java:103 — Logical column inspect rowgroups One column across all row groups, with stats ColumnAcrossRowGroupsScreen.java:135 — RG, Rows, Pages, Comp, Ratio, Dict, CI, Nulls, Min, Max The last one is the only