We indexed Kafka so analysts could finally query it. Here's what changed.
Leo Delmouly
More Stories
Kafka Is Fast — Getting Answers From It Isn’t
Why querying Kafka directly changes the real-time analytics game
Leo Delmouly
Streambased at Berlin Buzzwords 2025
Mid June saw the Berlin Buzzwords conference attracting the latest and greatest in AI, search, streaming and Iceberg to come together and share ideas and views of the future.
Tom Scott
My Internship Experience at Streambased: From Confusion to Confidence
Interning at Streambased has been an incredible journey full of challenges, learning, and growth. While my first day was tough, it was also filled with excitement and new ideas that set the tone for what became a transformative experience. Looking back, I can honestly say I haven’t regretted a single day.
Timothy Tse
Streambased Indexing
Streambased is an analytics company but really it’s an indexing company! Indexing is the core technology that enables Streambased to create fast logical projections over Kafka data.
Tom Scott
Current London 2025
Current 2025 has officially wrapped, and after 48 hours packed with learning, networking, and insightful discussions, there's much to reflect on.
Tom Scott
DSF Game On 2025
This Saturday, May 17th, we had the pleasure of sponsoring DSF Game On 2025, the 11th annual Data Science Festival held at CodeNode in London. With over 600 data professionals, analysts, engineers, and AI enthusiasts gathering under one roof, the event lived up to its reputation as one of the UK’s premier community-driven festivals in the data science space.
Tom Scott
Streambased at Kafka Meetup Dublin
Streambased is going on a world tour this summer! On top of Berlin and London, you can also catch up with us in Dublin at the end of this month.
Tom Scott
Streambased at Data Streaming Summit 2025
We're really looking forward to presenting our talk: Melting Icebergs: Enabling analytical access to Kafka data through Iceberg projections at Data Streaming Summit Virtual 2025.
Tom Scott
I.S.K. architecture
I.S.K. (Iceberg Service for Kafka) bridges the gap between real-time event streaming and modern analytical workflows by providing a logical, Iceberg-compliant interface over operational data systems. By materializing metadata and data views on the fly, I.S.K. enables zero-latency access to streaming data using familiar analytical tools and paradigms. Through ephemeral metadata, runtime partitioning, and indexing techniques, it delivers powerful performance optimizations without requiring changes to the underlying streaming infrastructure.
Tom Scott
A.S.K. architecture
For this post we're going to take a more technical dive into Streambased most mature product: A.S.K. (Analytics Service for Kafka). A.S.K. offers a high-performance, SQL-oriented query engine specifically designed for Kafka
Tom Scott
Real-time data isn't real-time if you're still waiting for reports
In insurance, real-time data should mean real-time decisions. But today, underwriting and risk analytics are stuck in a multi-hop pipeline: data flows through batch jobs, ETL processes, and static BI tools before anyone can act on it. By then, the risk is already on the books. Kafka powers real-time applications, yet analytics remain an afterthought. Insurers need to break free from the lag, enabling underwriters and finance teams to explore data as it happens, without waiting for reports. Because when a storm is forming, you need a live radar, not yesterday's forecast.
Leo Delmouly
Shifting Left
In the modern data stack every data job has an offline and online component. Typically these are served by vastly different systems and employ a multi-hop architecture to make data sets available for each. This can be enormously inefficient as each hop typically involves physical data movement, transformation and processing. These hops are commonly developed in isolation across many technologies, languages and approaches resulting in a Data MESS rather than a Data Mesh.
Tom Scott