apache/iceberg

Problem The map comparison helpers decode both keys and values with the map key type: Object actualKey = actualKeys.get(i, convert(keyType)); Object actualValue = actualValues.get(i, convert(keyType)); The value must be decoded with convert(valueType). When the key and value types differ, the current assertions may decode the value incorrectly and do not validate what they appear to validate. PR #14984 fixes GenericsHelpers for Spark 4.2 and will also fix the duplicate in Spark 4.2 TestHelpers. The same issue remains in both helpers for: spark/v3.5 spark/v4.0 spark/v4.1 Proposed fix Update the remaining six occurrences to use: Object actualValue = actualValues.get(i, convert(valueType)); Add or confirm coverage using a map whose key and value have different Spark types. Context Identified while reviewing Spark 4.2 support in #14984: #14984 (comment)

This issue was reported to the private Apache Iceberg security mailing list. The submitter is being kept anonymous because the report was sent to a private list. After review, the issue is not considered a serious vulnerability that needs to be kept private, so it is being filed publicly here for tracking and resolution. Note: this submission was generated by AI. Please review its claims and source references carefully before acting on them. Summary Spark scopes file_list_view with raw string prefix matching, letting sibling paths fall inside orphan cleanup. Affected Maven coordinates versioned integration artifacts: org.apache.iceberg:iceberg-spark-3.4_*, org.apache.iceberg:iceberg-spark-3.5_*, org.apache.iceberg:iceberg-spark-4.0_2.13, org.apache.iceberg:iceberg-spark-4.1_2.13 Attacker prerequisites control over a path, prefix, batch, or file list that sits adjacent to a legitimately scoped prefix ability to trigger the credential-selection or cleanup path with that crafted input

Methods marked with @deprecated ... will be removed in 1.12.0 needs to be removed now since 1.11.0 is released.

Feature Request / Improvement Description RewriteTablePathUtil.rewriteDVFile currently rewrites deletion vector (DV) Puffin files by reading all rewritten blobs into an in-memory list before writing them out, which can create unnecessary memory pressure when rewriting larger DV files or files with many blobs. Suggested improvement Refactor RewriteTablePathUtil.rewriteDVFile to stream rewritten blobs directly to the output PuffinWriter while iterating through the source blobs, instead of collecting all rewritten blobs into a temporary list first. Query engine None Willingness to contribute I can contribute this improvement/feature independently I would be willing to contribute this improvement/feature with guidance from the Iceberg community I cannot contribute this improvement/feature at this time

Feature Request / Improvement After #15288, Spark Writing to Branches doc (docs/docs/spark-writes.md) no longer matches that in codes. * <p>The target branch can be specified via table identifier, write option, or in SQL: * * <ul> * <li>The identifier and option branches can't conflict. If both are set, they must match. * <li>Identifier and option branches take priority over the session WAP branch. * <li>If neither the option nor the identifier branch is set and WAP is enabled for this table, * use the WAP branch from the session SQL config. * </ul> * * <p>Note: WAP ID and WAP branch cannot be set at the same time. * * <p>Note: The target branch may be created during the write operation if it does not exist. Query engine Spark Willingness to contribute I can contribute this improvement/feature independently I would be willing to contribute this improvement/feature with guidance from the Iceberg community I cannot contribute this impr