#bigdata (30 Repositories)
Ranked open-source repositories tagged with #bigdata, scored by pull request acceptance likelihood and maintainer engagement velocity.
46.3%
39.1h
30 repositories tagged #bigdata
volcano-sh/volcano
A Cloud Native Batch System (Project under CNCF)
apache/shardingsphere
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
rustfs/rustfs
RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.
apache/hudi
Upserts, Deletes And Incremental Processing on Big Data.
databendlabs/databend
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.
apache/avro-rs
Rust SDK for Apache Avro - a data serialization system.
NationalSecurityAgency/datawave
DataWave is an ingest/query framework that leverages Apache Accumulo to provide fast, secure data access.
reductstore/reductstore
High-performance, time-indexed object storage for robotics and industrial IoT
apache/livy
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
marmotdata/marmot
The open-source context layer for your AI. Catalog your tables, topics, queues and APIs then expose real metadata to your AI agents.
arvados/arvados
An open source platform for managing and analyzing biomedical big data
apache/avro
Apache Avro is a data serialization system.
apconw/Aix-DB
Aix-DB 基于 LangChain/LangGraph 框架，结合 MCP Skills 多智能体协作架构，实现自然语言到数据洞察的端到端转换。
pbreheny/biglasso
biglasso: Extending Lasso Model Fitting to Big Data in R
jamesmudd/jhdf
A pure Java HDF5 library
dotnet/spark
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
AbsaOSS/spline
Data Lineage Tracking And Visualization Solution
NewLifeX/NewLife.XCode
20 年演进的 .NET 高性能数据中间件，聚焦极致性能、海量数据、自动建模/迁移、多级缓存、自动分表分库，支持 MySQL/SQLite/SqlServer/Oracle/Postgresql/达梦 等
datafusion-contrib/StreamFusion
World's First OSS Flink Accelerator built on Apache DataFusion
apache/celeborn
Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
apache/airavata
A general purpose Distributed Systems Framework
scikit-hep/uproot5
ROOT I/O in pure Python and NumPy.
HariSekhon/Knowledge-Base
Large Tech Knowledge Base from 20 years in DevOps, Linux, Cloud, Big Data, AWS, GCP and international Consulting including extensive Travel Tips around the world
aliyun/aliyun-maxcompute-data-collectors
nebulastream/nebulastream
Data management for the sensor-edge-cloud continuum
aws-samples/aws-etl-orchestrator
A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.
achtungsoftware/alarik
High performance distributed S3 compatible object storage focused on speed and designed to be an open alternative to MinIO and RustFS.
gearpump/gearpump
Lightweight real-time big data streaming engine over Apache Pekko
spotify/big-data-rosetta-code
Code snippets for solving common big data problems in various platforms. Inspired by Rosetta Code
transferia/transferia
Open Source Cloud Native Ingestion engine