Mastering Kafka Streams and ksqlDB Building real-time data systems by example (Seymour, Mitch)（Z-Library）

【One-Line Pitch】 A hands-on tour of real-time stream processing that teaches you to build, operate, and choose between Kafka Streams and ksqlDB using concrete business scenarios. Best for data engineers, analysts, and data scientists who want to move from batch thinking to continuous event processing. 【Book Arc】 - **Opening (~0%–15%)**: Frames the problem of unbounded, fast-moving data and lays the Kafka foundation — topics, partitions, consumer groups, and the commit log as the storage abstraction behind streams. - **Early (~15%–35%)**: Introduces Kafka Streams itself: its operational goals (scalability, reliability, maintainability), the processor topology/DAG model, tasks and threads, and the high-level DSL versus the lower-level Processor API. - **Middle (~35%–55%)**: Moves from stateless to stateful processing — stateless operators like map/mapValues/flatMapValues, custom serialization (including Avro and Schema Registry trade-offs), then stateful operations such as joins and aggregations. - **Late (~55%–80%)**: Deepens state management — persistent store layout, fault tolerance via changelog topics and standby replicas, rebalancing pitfalls, and techniques to control state size and deduplicate writes. - **Ending (~80%–100%)**: Shifts to ksqlDB as the SQL-based alternative, covering querying key-value stores (local vs. remote), windows and time semantics, and deployment/operational patterns for both technologies. 【Key Takeaways】 - **Streams are stored as append-only commit logs** (Early): Kafka models continuous data as an ordered, append-only log — the same abstraction found in databases and version control — which underpins how topics persist and replay events. - **Kafka Streams favors a simple deployment model** (Early): Unlike cluster-based engines such as Flink or Spark Streaming, it runs as a library with a friendlier learning curve and supports true event-at-a-time processing. - **Topologies are DAGs of source, stream, and sink processors** (Early): Designing the topology first, then implementing it in Java, is the recommended mental model for building any Kafka Streams app. - **Tasks and threads are the unit of parallelism** (Early): A task maps to a topic-partition; adding threads redistributes tasks across CPUs without changing task count, which matters for scaling. - **Prefer mapValues/flatMapValues over map/flatMap** (Middle): Value-only operators let Kafka Streams execute more efficiently and avoid unnecessary rekeying unless stateful operations downstream require it. - **Stateful processing trades simplicity for power** (Middle): Capturing state enables joins and aggregations but introduces concerns around fault tolerance, rebalancing, and state size that stateless pipelines avoid. - **Schema handling is a real design decision** (Middle): Embedding Avro schemas in each record avoids extra infrastructure, while Confluent Schema Registry yields smaller payloads and safer schema evolution at the cost of another service. - **ksqlDB offers a gentler on-ramp** (Late): Non-Java developers can express stream processing in SQL, making it a practical entry point before committing to the Kafka Streams API. 【Reading Tips】 - Deep-read the Kafka fundamentals chapter if you're new to topics, partitions, and consumer groups — later chapters assume this fluency. - Skim the code-heavy setup and serialization sections on a first pass; return to them when you actually implement a pipeline. - Treat the state management and rebalancing material as the hardest part — slow down there, since operational mistakes here cause real production pain. - If you're not a Java developer, jump to the ksqlDB chapters first to build intuition, then circle back to Kafka Streams. - Keep the GitHub repo open alongside the book; the tutorials are meant to be run, not just read. 【Coverage Limits】 This guide is synthesized from stratified excerpts covering roughly the first half of the book plus table-of-contents fragments; later ksqlDB chapters and detailed operational configurations are only partially represented, so specifics there are inferred from headings rather than full text.