Best Data Lakehouse Software

BigLake provides enterprise storage, governance and performance for building analytical, operational and real-time AI workloads on an open lakehouse. Apache Iceberg tables managed through its Iceberg REST catalog are readable and writable from BigQuery, Google Cloud Managed Service for Apache Spark and Iceberg-compatible engines such as Spark, Trino and Flink, and Google has extended this to a borderless lakehouse reaching AWS, Databricks and Snowflake data.

Adopting Iceberg rather than a proprietary format is the decision that makes this interesting. A managed lakehouse holding data in an open table format means the data remains readable by engines the vendor does not control, which is the opposite of how warehouse lock-in has historically worked.

The cross-cloud reach is the more aggressive move. Making another platform's data queryable from here targets exactly the organisations that already committed elsewhere, and it is a bet that governance and AI access matter more to them than where the bytes sit.

Governance is what makes any of that safe to enable. Data readable from several engines across several clouds needs one place where access is decided, and a shared catalog with enforced permissions is the difference between interoperability and an uncontrolled copy sprawl.

Starburst is an enterprise intelligence platform built on Trino and Iceberg, offering an open federated lakehouse that queries data across hybrid environments, available as a managed service on major clouds or as self-managed Starburst Enterprise for private cloud, hybrid and on-premises deployment.

Federation is the core idea and it inverts the usual approach. The conventional answer to scattered data is to copy everything into a central warehouse, which means building and maintaining pipelines, accepting that the copy is always somewhat stale, and paying to store the same data twice. Querying it where it already lives removes the pipeline and the duplication, at the cost of query performance depending on systems the platform does not control.

No lock-in and no replatforming is stated explicitly and it is credible here rather than rhetorical, because the foundations are open source: Trino as the query engine and Iceberg as the table format are both projects a customer could run without Starburst. That is a materially different position from a proprietary warehouse where the data format itself is the lock-in.

Pricing is credit-based and published, which is unusual at this end of the market. The free tier costs nothing forever with up to three clusters and standard execution for ad hoc queries, then tiers start at 0.50 dollars per credit adding flexible execution modes, streaming ingest and advanced cluster management, 0.75 dollars per credit with early access features, and 1.00 dollars per credit with compliance tooling and the highest uptime guarantees. AIDA, the conversational analytics layer, bills token usage separately.

Dremio is a lakehouse platform published in three editions: Dremio Cloud as a fully managed platform for agentic AI, Dremio Enterprise as self-managed software running on Kubernetes on-premise or in cloud, and Dremio Community Edition as a free query engine for local machines or servers, with an AI agent, a semantic layer, an intelligent query engine and an open catalogue.

The semantic layer is the capability that distinguishes this from a query engine alone, and it addresses the gap that opens once a lakehouse actually works.

That gap is a familiar one. Once data is queryable directly in object storage, analysts can reach it, but what they find is raw: physical table names, technical column names, and no indication of which of four similar tables is the one people actually use. Being able to query everything is not the same as being able to understand it, and the usual result is that a small number of people who know the layout become a bottleneck for everyone else.

A semantic layer presents business-meaningful views over that physical data, so an analyst works with concepts rather than storage layout, and definitions live in one governed place rather than being re-derived in every query.

Publishing a free community edition for local machines is genuinely useful rather than a marketing gesture, because it lets an engineer evaluate the engine against their own data on their own laptop before any commercial conversation, which in this category tells you far more than a demonstration.

The three-edition structure covers the real deployment constraint. Cloud suits organisations without infrastructure preference, Enterprise on Kubernetes suits those who must keep data in their own environment, and both run the same engine.

Pricing is published as a consumption model with a $0.20 rate for the managed agentic lakehouse on AWS, with Azure noted as coming soon, and a $400 figure published within the range.

AWS Lake Formation builds and secures data lakes on AWS, providing centralised permissions that are enforced consistently across the analytics services reading the data, including table, column and row-level access control over data held in S3 and catalogued in the Glue Data Catalog.

Centralised permissions are the actual product and they solve a problem that grows quietly. A data lake accessed by several engines otherwise needs its access rules expressed once per engine, and those definitions drift, so a column restricted in one query tool remains visible through another.

Column and row level control is what makes a shared lake usable for sensitive data. Without it the only options are giving a team the whole table or copying a filtered version for them, and copies are where governance quietly fails because nobody updates the restriction on the copy.

The trade is depth of integration against portability. These permissions are enforced by AWS services reading the data, so the model works well inside that estate and does not travel to engines outside it, which is worth establishing before treating it as the governance layer for everything. An organisation querying the same lake from a tool outside AWS still needs its rules expressed there too.

Delta Lake is an open source storage framework enabling a format-agnostic lakehouse, working with compute engines including Spark, PrestoDB, Flink, Trino, Hive, Snowflake, Google BigQuery, Athena, Redshift, Databricks and Azure Fabric. Through UniForm, Delta tables can be read by Iceberg and Hudi clients, and the project's aim is one format unifying ETL, warehousing and machine learning in a lakehouse.

What it adds to a data lake is transactional guarantees, which is the distinction the whole lakehouse idea rests on. Files in object storage have no notion of a transaction, so a reader can see a half-written update and a failed job can leave a table corrupt, and a transaction log over those files restores the behaviour people expect from a database.

Time travel follows from the same log and is more useful than it first appears. Querying a table as it stood last Tuesday answers questions about when a number changed that are otherwise unanswerable, and it makes reproducing a model training run possible.

Interoperability with Iceberg and Hudi clients is the strategically significant piece. The lakehouse format contest has been costly for users, and a table readable by all three removes the need to pick a winner before committing data to a format.

Apache Iceberg is an open table format for analytic datasets, described by the project as a high-performance format for huge analytic tables that brings the reliability and simplicity of SQL tables to big data, while making it possible for engines including Spark, Trino, Flink, Presto, Hive and Impala to work on the same tables at the same time safely.

That last capability is the reason Iceberg has been adopted so widely, and it solves a problem that quietly constrained data platforms for a decade.

Before table formats like this, data in a lake was a collection of files in object storage with a directory convention layered over it. Every engine had to agree on that convention, and none of them could safely write while another was reading, because there was no notion of a transaction. The practical consequences were familiar to anyone who ran such a platform: a query returning half-written results because a job was still writing, no way to change a schema without rewriting everything, and no way to know what the table looked like an hour ago.

Iceberg introduces a proper table abstraction with atomic commits, so a reader always sees a consistent snapshot and writers do not corrupt readers. Schema evolution becomes a metadata operation rather than a full rewrite, and time travel to a previous snapshot becomes possible because the snapshots are tracked rather than overwritten.

Multi-engine access is the strategic point. An organisation can run Spark for its pipelines, Trino for interactive queries and Flink for streaming against the same tables, without copying data between systems or maintaining three versions of the same dataset. That was previously the main source of both cost and disagreement between teams.

The format is also what makes storage genuinely separable from compute, which is the architecture that has displaced traditional Hadoop clusters for most new work.

Iceberg is free and open source under the Apache licence, with managed implementations offered by cloud and platform vendors.