#data-processing (22 Repositories)
Ranked open-source repositories tagged with #data-processing, scored by pull request acceptance likelihood and maintainer engagement velocity.
46.8%
33.4h
22 repositories tagged #data-processing
rocketride-org/rocketride-server
High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.
numaproj/numaflow
Kubernetes-native platform to run massively parallel data/streaming jobs
NVIDIA/DALI
A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
NVIDIA-NeMo/Curator
Scalable data pre processing and curation toolkit for LLMs
datachain-ai/datachain
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
apache/wayang
Apache Wayang is the first cross-platform data processing system.
docwire/docwire
DocWire SDK: local-first C++20 data-processing infrastructure for deterministic, auditable, on-premise workflows. Supports 100+ formats, OCR, local AI, and opt-in cloud AI pipelines
senbox-org/snap-engine
ESA Earth Observation Toolbox and Java Development Platform
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
johnkerl/miller
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
mech-lang/mech
🦾 Mech is a programming language for building data-driven systems like robots, games, and interfaces. Start here!
Puchaczov/Musoq
SQL Runtime without any database
senbox-org/snap-desktop
Desktop GUI for SNAP based on NetBeans Platform
kubeflow/mcp-apache-spark-history-server
MCP Server and CLI for Apache Spark History Server. Debug Spark applications from AI agents, scripts, or the terminal.
elixir-broadway/broadway
Concurrent and multi-stage data ingestion and data processing with Elixir
streamnative/pulsar-spark
Spark Connector to read and write with Pulsar
keithorange/PatternPy
📈 PatternPy: A Python package revolutionizing trading analysis with high-speed pattern recognition, leveraging Pandas & Numpy. Effortlessly spot Head & Shoulders, Tops & Bottoms, Supports & Resistances. For experts & beginners. #TradingMadeEasy 🔥
constellation-rs/amadeus
Harmonious distributed data analysis in Rust.
utdemir/distributed-dataset
A distributed data processing framework in Haskell.
TangleML/tangle
Tangle is a web app that allows the users to build and run Machine Learning pipelines without having to set up development environment.
lithops-cloud/lithops
A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
eddyzzl/marvis-risk-agent
MARVIS-Agent: all-purpose credit risk agent for model development, validation, data processing, feature engineering, and strategy workflows.