Skip to content

Streamhouse

Streamhouse is an open, vendor-neutral category for data architectures that keep the current state of a business continuously available to production applications, analytics, and AI agents. This is a key initiative, started by Ververica and embrassed by Confluent and IBM in 2026 with the need to serve fresh data to human and AI agents.

High level View

It groups change data capture, event streams, stream processing, open table formats, catalogs, and low-latency serving.

High-level capabilities

Data remains continuously current as business events occur. Streamhouse is in the operational critical path for data production to human and AI agents.

This is the architecture to support the end-to-end vision of data as a product, starting from the methodology.

Concrete Confluent Based Architecture

The Confluent distribution of a streamhouse is the Data streaming platform (DSP). The DSP is a complete, cloud-native foundation. It is built on open standards such as Apache Kafka, Apache Flink and Apache Iceberg and offers proprietary, differentiated features designed to abstract complexity of building and operating a streamhouse.

High-level Confluent Cloud Streamhouse
  • A snapshot query is a one-time Flink SQL query that reads a consistent, point-in-time view of a table, returns the results, and then terminates

Shifting to real-time processing

Preparing medallion layers is possible by using data already in Kafka. The classical batch processing can be shifted to the left of the architecture, by doing transformation, filtering, deduplication to build the silver layer in Kafka Topic using Flink processing.

The business logic that applies to data in Kafka topics so in Flink tables can be built in Flink to create gold records. Those gold records, representing dimensions and facts, can be persisted for long term in object storages, in Iceberg Tables. An external query engines, and federated catalog can be used on top of those Iceberg Tables. Most of the logic done in batch should be easily ported to real-time processing. Still some consideration like state size, running all time streaming processing, time and watermark add to the semantic complexity.

Adopting an event-driven architecture, silver and gold topics/ business events are reusable and not just usable for analytic query engines.