Basekick-Labs/arc

Found while reviewing #906. Problem internal/storage/local.go, in LocalBackend.ListUnusable (line 1129 on main @ 5b52a65), the long comment above the .part carve-out ends with: // (though see #762: no caller enumerates staged partials under the data // root yet, and ListStaged reports them under the base key). Both halves are stale: reclaimStagedPartials in internal/api/databases.go enumerates staged partials under database/ on every DROP DATABASE, and the edge-sync sweep in internal/edgesync/receive.go does the same under .sync-staging/. Callers exist. Since #906, ListStaged no longer reports every .part file under its base key: a partial whose stripped key DeleteStaged would refuse is omitted there and is reported by this very function instead. That is the relationship the comment should state. #762 is closed. What to change Rewrite the parenthetical to describe the current contract: ListStaged returns only keys DeleteStaged accepts (#772), the two reclaim loops consume it, and a

Found while reviewing #830. Problem internal/audit/middleware.go builds an AuditEvent from c.Path(), c.IP(), c.Get("User-Agent"), c.Get("x-arc-database"), c.Params(...) and c.Query("db"), then hands the event to Logger.LogEvent, which only enqueues the pointer on a channel. The batch writer (writerLoop) serializes the fields up to a second later. Arc runs Fiber with Immutable=false (see the note in internal/api/msgpack.go), so every one of those accessors returns a string that aliases a pooled buffer: c.Query is app.getString(QueryArgs().Peek(key)), c.Path() is app.getString(c.pathBuffer) on the pooled Ctx, and c.Get reads the request header buffer. Once the handler returns, the connection's RequestCtx and the Ctx go back to their pools and the next request overwrites those bytes. An audit row can therefore record a later request's path, database or user agent, or garbage. #830 adds path and reason from c.Query to the same event through audit.DetailLocalsKey, which the review asks to c

Found while running the #633 end-to-end check. Problem validateWhereClause (internal/api/delete.go) applies dangerousKeywordPattern.FindString(where) to the raw WHERE text, before any string-literal masking. A literal that happens to contain a forbidden word is rejected as if it were SQL: POST /api/v1/delete {"database":"icedb","measurement":"m","where":"host = 'drop'","confirm":true} {"success":false,"error":"WHERE clause contains forbidden keyword: DROP"} Any tag or field value such as status = 'delete-pending', action = 'update', kind = 'alter' is undeletable through the API. The value is data, not SQL. Fix shape Mask string literals with the shared sqlutil masker (the same normalization the query path uses before its own gating) and run the keyword and prefix scans on the masked form. Keep the unmatched-quote check on the raw text. Add a regression test with a literal containing each forbidden keyword, plus one proving '; DROP TABLE x -- outside a literal is still refused. Not a

Summary Ingest appends "time" to a measurement's sort keys; compaction does not. With any custom sort key configured, compaction re-sorts the data by the configured columns only, dropping the time ordering that ingest established within each group. The comment directly above the compaction call site claims the opposite: // Build ORDER BY clause from sort keys // This ensures compacted files maintain the same sort order as ingested files orderByClause := buildOrderByClause(j.SortKeys) internal/compaction/job.go:744-746 Root cause config.ParseSortKeys deliberately returns only the additional columns — time is documented as being appended "at the usage site": // Users only configure ADDITIONAL sort columns - the "time" column is always // appended automatically at the usage site (getSortKeys in arrow_writer.go). internal/config/sort_keys.go:9-13 There are two usage sites, and only one does the append. Ingest appends it — internal/ingest/arrow_writer.go:1003-1013: // Always ensure "tim

Split out of #762 (item 3). The first half is sized for a first contribution; the second half is a design call and is explicitly out of scope here. Problem internal/storage/local.go ListStaged (line 1163) walks for *.part, strips PartSuffix, and reports the result as the key the partial belongs to. DeleteStaged (line 1044) resolves that key through stagedPath (line 985), which validates the key body. Two consequences: A partial whose key fails validateKeyBody (a legacy partial of a key that is illegal today) is reported by ListStaged and then refused by DeleteStaged: the enumerator's output is not accepted by its own deleter. reclaimStagedPartials (internal/api/databases.go:790-809) pipes one into the other, so DROP DATABASE logs a warning for that file on every run and never reclaims it. A committed object whose real name ends in .part (legal before #744) is reported as a partial of a different key, and DeleteStaged on that key removes the committed object: ListStaged("") -> "db/cpu