Good First Issues in Scala

Labels: documentation, bug, good first issue Difficulty: easy — a five-minute fix and a good first contribution. Problem The pin multiplexer table in docs/source/nonmetals/elemrv_h.rst (around line 184) disagrees with the RTL from pin 3 onwards. Every entry from there down is shifted by one. The RTL is authoritative — addPinmuxOption in hardware/scala/elemrv_h/ElemRV.scala:95-106: Pin Function 0 Function 1 Docs currently say 0-2 gpio0_0/pwm0_out_0, gpio0_1/pio0_0, gpio0_2/pio0_1 correct 3 uart0_tx gpio0_3 gpio0_3 / pio0_2 4 uart0_rx gpio0_4 uart0_tx / gpio0_4 5 uart0_cts gpio0_5 uart0_rx / gpio0_5 6 uart0_rts gpio0_6 uart0_cts / gpio0_6 7 gpio0_7 pwm0_comp_0 uart0_rts / gpio0_7 8 gpio0_8 i2c0_scl gpio0_8 / pwm0_comp_0 9 gpio0_9 i2c0_sda gpio0_9 / i2c0_scl 10 gpio0_10 i2c0_interrupt_0 gpio0_10 / i2c0_sda 11 gpio0_11 pio0_2 gpio0_11 / i2c0_interrupt_0 The board's own pinctrl definitions agree with the RTL, so it is only the table that is wrong — but it

Labels: documentation, good first issue Difficulty: easy Problem docs/source/nonmetals/ contains elemrv_h.rst and elemrv_c.rst, and the toctree in index.rst lists exactly those two. There is no elemrv_n.rst — ElemRV-N, the most capable nonmetal SoC and the one with HyperRAM, a data cache and branch prediction, is undocumented. What to do Add docs/source/nonmetals/elemrv_n.rst following elemrv_h.rst, and add it to the toctree. Take every value from the RTL, not from the other pages: CPU configuration — VexiiRiscvCoreParameter.performance(...) in zibal/hardware/scala/zibal/platform/nonmetals/Nitrogen.scala: ISA string, cache sizes, BTB sets, PMP regions. Clock domains and frequencies from hardware/scala/elemrv_n/SG13CMOS5L/SG13CMOS5L.scala — note that Nitrogen has five (system, debug, hyperbus, xip, peripheral), unlike H's two. Memory map from Nitrogen.scala: OCRAM, HyperRAM cached and uncached windows, SPI flash, peripherals. Peripheral list, interrupt numbers and error-source nu

Labels: documentation, good first issue Difficulty: easy — but only useful if done on a genuinely clean machine. Problem Nobody has confirmed that a newcomer can get from git clone to a running SoC without prior knowledge of the project. The flow involves Google's repo for the manifest, a podman container, task as the entry point, and a Renode emulation — each fine on its own, unusual in combination, and none of it obvious. The most valuable thing this project can offer a curious visitor is a running RISC-V SoC with a blinking LED in fifteen minutes, no FPGA required. That path exists — task baremetal:compile then task sim:emulate — but it is not written down as a single verified sequence, and it is not known whether it works from scratch. What to do On a machine (or fresh VM/container) with nothing installed: Follow the current README.rst and docs/source/installation.rst literally — no fixing things from memory, no steps from experience. Write down every point where you had

The SRAM macro for the OnChipRAM is currently placed right in the center of the north edge DRT is struggling to route at the bottom of the macro (especially at both corners) with some violations. Place the macro closer to the north edge Try to find a better spot. Maybe at the east or west edge?

Description Migrate important tests to Spark-34+ Remove Spark-33 unit tests Remove Spark-33 shim layer Remove building scripts Gluten version None

jackson-module-scala supports simple Scala 3 enums like these: enum ColorEnum { case Red, Green, Blue } enum JavaCompatibleColorEnum extends java.lang.Enum[JavaCompatibleColorEnum] { case Red, Green, Blue } enum JavaCompatibleColorEnum extends java.lang.Enum[JavaCompatibleColorEnum] { case Red, Green, Blue } enum Color(val rgb: Int): case Red extends Color(0xFF0000) case Green extends Color(0x00FF00) case Blue extends Color(0x0000FF) case Mix(mix: Int) extends Color(mix) We don't yet support Parameterized Cases in Scala 3 Enums Scala 3 enums support cases with parameters, similar to how you'd define a sealed trait hierarchy with case classes, but with much less boilerplate. Basic Syntax enum Shape: case Circle(radius: Double) case Rectangle(width: Double, height: Double) case Triangle(base: Double, height: Double) Basically, the current code uses the toString on the case instance and and we look for the derived valueOf function that Scala creates on the derived compani

This comes from the investigation of SPARK-56444 (apache/spark@74530b911b5) in Apache Spark 4.2. It was noticed that the equals() implementation and hashCode() were not congruous in BatchScanExec. Specifically, BatchScanExec did not consider the keyGroupedPartitioning in its hashCode() calculation, but did take it into account for equals(). I see now that we have a similar problem in GpuBatchScanExec: override def equals(other: Any): Boolean = other match { case other: GpuBatchScanExec => this.batch != null && this.batch == other.batch && this.runtimeFilters == other.runtimeFilters && this.spjParams == other.spjParams case _ => false } override def hashCode(): Int = Objects.hashCode(batch, runtimeFilters) We should consider including the spjParams in hashCode(). Applies to all shims.

See PR #1419 which disabled it, figure out a fix, then re-enable.

Motivation Recently Scaladex has been receiving unusually many requests, which has caused unresponsiveness. These requests may come from legitimate users or AI assistants, but it may also be malicious scraper bots. Either way, there is a need to distribute server's resources fairly and evenly. Identified Obstacles The public API doesn't seem to have any authentication or rate limiting. Therefore, a carelessly written script can flood the server with requests completely by accident. Implementation Guideline I propose to implement rate limiting for the public API. An optional authentication by Github token may be worth considering. Expectations The rate limit would make accidental flooding very unlikely, and purposeful DoS attacks harder to orchestrate. Authenticated users could enjoy an increased rate limit; potentially it would be possible to identify the users which abuse the server.

Describe the bug In ANSI mode, Comet's wide-decimal arithmetic overflow reports a different value parameter from Spark. PR #5169 fixes the exception type and propagates the error class, SQLSTATE, and query context, but Comet still formats the rescaled i256 result while Spark formats its pre-toPrecision Decimal. This is a follow-up to #5072. Steps to reproduce Using Spark 4.1.2, run the query once with Comet disabled and once with Comet enabled: SET spark.sql.ansi.enabled = true; CREATE TABLE tbl (_1 DECIMAL(20, 0)) USING PARQUET; INSERT INTO tbl VALUES (11000000000000000000); SELECT _1 * _1 FROM tbl; Spark reports: [NUMERIC_VALUE_OUT_OF_RANGE.WITH_SUGGESTION] 121000000000000000000000000000000000000 cannot be represented as Decimal(38, 6). If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error, and return NULL instead. SQLSTATE: 22003 Comet reports: [NUMERIC_VALUE_OUT_OF_RANGE.WITH_SUGGESTION] 121000000000000000000000000000000000000.000000 cannot be represented as

Describe the bug Nested floating-point elements use total-order comparison in Comet, where -0.0 and 0.0 are distinct. Spark uses ordering.equiv for nested elements, which checks numeric equality first and therefore treats both signed zeros as equal. This affects: arrays_overlap when the array elements are themselves arrays or structs containing floats. The nested fallback in array_position. The flat arrays_overlap path is intentionally different: Spark uses Double.equals there and distinguishes signed zeros. Steps to reproduce CREATE TABLE t( a ARRAY<ARRAY<DOUBLE>>, b ARRAY<ARRAY<DOUBLE>>, v ARRAY<DOUBLE> ) USING parquet; INSERT INTO t VALUES ( array(array(0.0D)), array(array(-0.0D)), array(-0.0D) ); SELECT arrays_overlap(a, b), array_position(a, v) FROM t; Spark returns true and 1; Comet returns false and 0. Expected behavior Nested arrays_overlap and array_position should treat -0.0 as equal to 0.0, matching Spark, while continuing to treat NaN as equal to NaN. Additi

What is the problem the feature request solves? This epic tracks the follow-ups from an audit of the native Rust code (spark-expr, core, shuffle, common) looking for hand-rolled logic that replicates existing arrow-rs compute kernels and could be replaced by calling the kernel. The workspace is on arrow 58.4.0; some of the duplicated code predates public availability of the arrow API it copies. Every candidate was judged for Spark-semantics equivalence before being filed: much of Comet's native code intentionally diverges from arrow for Spark compatibility (ANSI errors, HALF_UP rounding, Java string formats, timezone DST rules, byte-format compatibility), and those cases are listed at the bottom as verified-intentional rather than filed as issues. Kernel equivalences below were verified against the vendored arrow 58.4.0 sources where noted in the individual issues. Delete duplicated arrow code (zero risk) #5088 Remove timezone.rs copy of arrow-array Tz (now public in arrow 58; the cr

What is the problem the feature request solves? native/spark-expr/src/timezone.rs is a full copy of arrow-array's Tz/TzOffset (the file header says "This is basically from arrow-array::timezone (private)"). In arrow 58, arrow::array::timezone::Tz is public when the chrono-tz feature is enabled, which our workspace already does. In fact native/spark-expr/src/kernels/temporal.rs already imports arrow's Tz, so the crate currently carries two parallel, incompatible Tz types. Describe the potential solution Delete timezone.rs and switch its consumers (spark-expr/src/utils.rs, spark-expr/src/conversion_funcs/temporal.rs, spark-expr/src/conversion_funcs/string.rs) to arrow::array::timezone::Tz. Consumers only use FromStr plus the chrono::TimeZone trait, which arrow's type provides identically. Additional context Fixed-offset parsing grammars match for Spark-legal offsets: Comet parses via chrono %:z/%#z, arrow 58 accepts [+-]XX:XX, [+-]XXXX, [+-]XX. Error message text differs slightly but onl

Describe the bug prepare_object_store_with_configs in native/core/src/parquet/parquet_support.rs caches object store instances under (url_key, config_hash), where url_key is built from: let url_key = format!( "{}://{}", scheme, &url[url::Position::BeforeHost..url::Position::AfterPort], ); Position::BeforeHost starts after the URL userinfo. For ABFS, the container is the userinfo, not the host: abfss://[email protected]/path/to/file.parquet So abfss://[email protected]/... and abfss://[email protected]/... both produce the key abfss://acct.dfs.core.windows.net. config_hash is computed over the per-session Hadoop config map, which does not vary by container, so it does not disambiguate them either. Meanwhile MicrosoftAzureBuilder::parse_url bakes the container into the store instance from the URL userinfo, and the Path handed to that store is container-relative (the bucket/container is stripped by Path::from_url_path(url.path()) in

Context Dict and OrderedDict are designed to avoid (K, V) tuple allocations: iteration, lookup, and transformation pass separate key and value arguments rather than tuples, to eliminate boxing in hot paths. Two APIs still break that premise by forcing tuple allocation: API Dict OrderedDict apply[K, V](entries: (K, V)*) kyo-data/shared/src/main/scala/kyo/Dict.scala:83 kyo-data/shared/src/main/scala/kyo/OrderedDict.scala:88 map[K2, V2](fn: (K, V) => (K2, V2)) kyo-data/shared/src/main/scala/kyo/Dict.scala:416 kyo-data/shared/src/main/scala/kyo/OrderedDict.scala:423 The vararg constructor allocates a tuple per entry at the call site; map allocates a tuple per element for the result of fn. Origin Raised by @fwbrasil while reviewing #1749 (OrderedDict). OrderedDict deliberately mirrors Dict's current design, so the tuple-allocating shape is pre-existing on Dict and inherited by OrderedDict. Split out as a separate follow-up per that review: #1749 (comment) (constructor) #1749 (c

I only tested this on a Mac. import io/error import io/filesystem def main() = { with on[IOError].panic println("before readFile") val s = readFile("data.txt") // any non-empty file println("after readFile") // LLVM never gets here println(s.substring(0, 12)) } $ effekt --backend=js repro.effekt before readFile after readFile <here are the contents of data.txt> $ effekt --backend=llvm repro.effekt before readFile [error] Process exited with non-zero exit code 138. I tried minimising a bit: the only difference I can observe is a local vs global definition of the max chunk size: import io import io/error import io/filesystem import bytearray // on LLVM, global works, local does not // val chunkSizeGlobal = 1048576 // lines marked with a `// !` point to lines using the local chunk size def readFile(path: String): String / Exception[IOError] = { val file = openForReading(path); with on[IOError].finalize { close(file) } val chunkSize = 1048576 // ! var buffer = by

Problem When using regex patterns with the alternation operator | in the OrganizeImports groups configuration, the regex fails to match imports that individual patterns (without alternation) match correctly. Example Given this .scalafix.conf: OrganizeImports { groups = [ "re:^(org\\.apache\\.pekko\\.?|pekko\\.)", "org.slf4j.", "*" ] removeUnused = true } The regex ^(org\.apache\.pekko\.?|pekko\.) should match both: import org.apache.pekko (syntax string: org.apache.pekko) import pekko.actor.Actor (syntax string: pekko.actor.Actor) Expected: Both imports go to group 0. Actual: Neither import goes to group 0. Both fall through to the * catch-all group. What works Each pattern works correctly when used standalone (without alternation): # This matches `import org.apache.pekko` correctly: groups = ["re:org\\.apache\\.pekko", "*"] # This matches `import pekko.*` correctly: groups = ["pekko.", "*"] What doesn't work Combining them with | fails: # Neither pattern matches w

Describe the bug Three cache tests carry @allow_non_gpu_conditional(is_spark_350_or_351(), "InMemoryTableScanExec"), but that allowance never does anything. The test-mode plan checker has a dedicated branch for InMemoryTableScanExec that accepts a CPU cache scan without ever consulting the allowed-non-GPU list, so the marker grants an allowance that is never read. Impact None functional. On Spark 3.5.0/3.5.1 the cache scan is expected to run on CPU (InMemoryTableScan is disabled there — see PR #13434), and the checker already tolerates that regardless of the marker. Removing the markers changes no test outcome on any Spark version. This is dead test config, not a correctness bug. What to do Delete the three allow_non_gpu_conditional(...) lines in cache_test.py (test_aqe_cache_version_specific_behavior, test_persist_with_groupby_join_version_specific, test_cached_groupby_sum_version_specific). Nothing else is required. Environment details N/A (test harness; all environments). Addition

And probably just remove support for BFT sequencer connections. Reasoning: It matches what we do in prod. It doesn't seem like we actually want to enable BFT sequencer connections for SVs as the fact that you lose rewards when your node doesn't work sets the right incentives. Need to see if this gets annoying in tests with cantonbft blacklisting. Worst case I don't mind disabling blacklisting in integration tests.

We have the sequencer tps and topology cap configurations which are applied by every SV through the sequencer configuration. That configuration ideally should be exposed through the info helm chart that every sv deploys at info.*

Description see discussion #12312 setup Almalinux8 CI image for Gluten, example on centos7 https://github.com/apache/gluten/blob/main/dev/docker/Dockerfile.centos7-gcc13-static-build Migrate to use the new image for x86/arm native lib build: https://github.com/apache/gluten/blob/main/.github/workflows/velox_backend_x86.yml#L71 Gluten version None

What is the problem the feature request solves? The community standards page indicates that there is no top level CONTRIBUTING.md document. It is good practice to have this file since new contributors may look for this. This can be a very minimal document that simply links to the existing contributors guide. Describe the potential solution No response Additional context No response

We can generate events similarly to how we generate attributes. See: https://github.com/open-telemetry/semantic-conventions-java/pull/489/changes

Summary The workspace/ module (workspaceClient, workspaceRunner, workspaceShared) enables containerized code execution. ContainerisedWorkspaceTest in modules/it/ tests the workspace directly, but there are no integration tests that verify the full flow: an agent receives a coding task → calls a workspace tool → workspace executes code in Docker → result returned to agent → agent produces final response. What needs to be done Add modules/it/src/test/scala/org/llm4s/workspace/WorkspaceAgentIntegrationSpec.scala: Requires: Docker running (DOCKER_AVAILABLE=true); skip if unavailable Basic code execution via agent tool: Register a workspace execution tool in ToolRegistry; run Agent(MockLLMClient) where the mock returns a tool call run_code(language="python", code="print(1+1)"); verify workspace executes and returns "2" to the agent Timeout enforcement: Submit code that sleeps 60s; verify timeout error returned within configured deadline Sandbox isolation: Attempt to write to filesy

Summary The mcp/ subsystem (7 sources, 12 unit tests) handles the Model Context Protocol — connecting to MCP servers, listing tools, and invoking them from agents. There are no integration tests that spin up a real (or local test) MCP server, connect to it, and verify that tools discovered via MCP can be called by an agent end-to-end. What needs to be done Add modules/it/src/test/scala/org/llm4s/mcp/MCPServerIntegrationSpec.scala: Option A — Embedded test MCP server (preferred, CI-safe): Start a minimal in-process MCP server (using the MCP SDK or a tiny HTTP stub) that exposes 2–3 test tools (e.g. echo, add, reverse) Connect MCPClient to the local server Verify: listTools() returns the 3 expected tools with correct schemas Invoke echo(message="hello") via MCP; verify response "hello" returned Wire the MCP tools into a ToolRegistry and run Agent(MockLLMClient) that calls the MCP tool; verify full call-return cycle Option B — Real MCP server (tagged, optional): Tag McpRequired

Summary The agent/memory/ subsystem supports SimpleMemoryManager (in-memory) and VectorMemoryStore (embedding-backed). There is a PostgresMemoryStoreSpec in modules/it/ but it tests the store in isolation. Missing are integration tests that wire VectorMemoryStore into a full Agent run and verify that: memories recorded in conversation turn N are retrieved and injected into the context of turn N+1. What needs to be done Add modules/it/src/test/scala/org/llm4s/agent/memory/VectorMemoryAgentIntegrationSpec.scala that: Requires: pgvector (PGVECTOR_TEST_URL) + Ollama (OLLAMA_AVAILABLE) for embeddings Uses assume() to skip when services unavailable Record and retrieve: Use SimpleMemoryManager wired to VectorMemoryStore; record 5 user facts; run getRelevantContext(query) and assert relevant facts are returned Memory in agent loop: Run Agent for 3 turns with VectorMemoryStore; after turn 2, assert turn-1 memory is present in turn-3 context (injected into system prompt) Memory isolation by use

Summary The modules/it/ module has separate tests for PgVectorStore and for the RAG search index bug, but no test that runs the full RAG pipeline end-to-end with real components: Ollama embeddings + pgvector storage + hybrid keyword search + agent answering from retrieved context. This is the most representative real-world usage and it is currently untested. What needs to be done Add modules/it/src/test/scala/org/llm4s/rag/RAGPipelineOllamaIntegrationSpec.scala that: Requires: local Ollama (OLLAMA_AVAILABLE=true, model nomic-embed-text) + PostgreSQL with pgvector (PGVECTOR_TEST_URL set) Uses assume() to skip gracefully when either service is unavailable Index phase: Load 10 sample text documents → chunk → embed with Ollama → store in pgvector Search phase: Run semantic search queries → verify top-K results are semantically relevant (not exact match) Hybrid search phase: Enable keyword index alongside vector index; run hybrid queries; verify combined results Agent RAG phase: Us

Cast from maptype to string needs to be improved. Check #4630, how many tests currently falls back to spark when trying to cast Map to String Originally posted by @comphead in #4630 (comment)

Overview There is no Gradle example in the repository. Most Java backend developers use Gradle. A standalone, runnable Gradle project that shows how to call llm4s from Java in 5 minutes would dramatically lower the barrier for Java shop adoption — and it's a great first contribution because it requires no changes to the Scala codebase. What to create Create a standalone Gradle project at modules/samples/gradle-java/ (this does NOT need to be part of the sbt build): modules/samples/gradle-java/ ├── README.md ├── settings.gradle.kts ├── build.gradle.kts └── src/main/java/org/llm4s/samples/HelloLLM4S.java build.gradle.kts plugins { java application } repositories { mavenCentral() } dependencies { implementation("org.llm4s:core_3:0.1.16") { // Exclude if you manage logging yourself: // exclude(group = "ch.qos.logback", module = "logback-classic") } // Scala 3 runtime required: implementation("org.scala-lang:scala3-lib

Description IBM now provides instances to Apache so Gluten should be able to run with Power Currently the Power team is working with Apache/arrow project to add those instances at apache org level. Once that is done Gluten should be able to use those instances as well. IBM/actionspz#102 Gluten version None