Most agent-friendly big data repos

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Apache Flink

Apache Spark - A unified analytics engine for large-scale data processing

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

ClickHouse® is a real-time analytics database management system

Apache Beam is a unified programming model for Batch and Streaming data processing.

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)