#data-processing (18 Repositories)
Ranked open-source repositories tagged with #data-processing, scored by pull request acceptance likelihood and maintainer engagement velocity.
39.0%
39.1h
18 repositories tagged #data-processing
johnkerl/miller
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
numaproj/numaflow
Kubernetes-native platform to run massively parallel data/streaming jobs
kubeflow/mcp-apache-spark-history-server
MCP Server and CLI for Apache Spark History Server. Debug Spark applications from AI agents, scripts, or the terminal.
rocketride-org/rocketride-server
High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.
NVIDIA-NeMo/Curator
Scalable data pre processing and curation toolkit for LLMs
datachain-ai/datachain
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
mech-lang/mech
🦾 Mech is a programming language for building data-driven systems like robots, games, and interfaces. Start here!
apache/wayang
Apache Wayang is the first cross-platform data processing system.
elixir-broadway/broadway
Concurrent and multi-stage data ingestion and data processing with Elixir
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
streamnative/pulsar-spark
Spark Connector to read and write with Pulsar
utdemir/distributed-dataset
A distributed data processing framework in Haskell.
lithops-cloud/lithops
A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
senbox-org/snap-engine
ESA Earth Observation Toolbox and Java Development Platform
TangleML/tangle
Tangle is a web app that allows the users to build and run Machine Learning pipelines without having to set up development environment.
Puchaczov/Musoq
SQL Runtime without any database
constellation-rs/amadeus
Harmonious distributed data analysis in Rust.
keithorange/PatternPy
📈 PatternPy: A Python package revolutionizing trading analysis with high-speed pattern recognition, leveraging Pandas & Numpy. Effortlessly spot Head & Shoulders, Tops & Bottoms, Supports & Resistances. For experts & beginners. #TradingMadeEasy 🔥