Kafka for Architects Event-driven architecture, logs, microservices, real-time event processing (Katya Gorshkova)（Z-Library）

# Kafka for Architects: Reading Guide ## 【One-Line Pitch】 A practical, code-free guide for enterprise and solutions architects who need to make smart architectural decisions about Apache Kafka—covering event-driven design, data contracts, streaming patterns, and operational strategy. If you're responsible for deciding *whether* and *how* Kafka fits into your systems, this book is your playbook. ## 【Book Arc】 - **Opening (~0%–14%)**: Introduces Kafka from an architect's perspective—what it is, why event-driven architecture matters, and the core building blocks (topics, partitions, brokers, producers, consumers). Establishes the mental model of Kafka as a distributed commit log rather than just a message queue. - **Early (~14%–29%)**: Dives into cluster data architecture—partitioning, replication, message structure, compaction, and topic configuration. Then moves to the producer side: serialization, partitioning strategies, acknowledgments, batching, and common production challenges. - **Early–Middle (~29%–43%)**: Covers consumer applications—group coordination, offset management, rebalancing, and scalability. Transitions into real-world use cases (microservices, data integration, log collection) and compares Kafka against alternatives like RabbitMQ and Pulsar. - **Middle (~43%–57%)**: Focuses on data contracts and event design—fact vs. delta events, schema evolution, Schema Registry, and governance. Introduces interaction patterns (request-response, CQRS, event sourcing), Kafka Connect, and delivery guarantees including transactions and the outbox pattern. - **Middle–Late (~57%–79%)**: Explores streaming applications with Kafka Streams and alternatives (ksqlDB, Flink), then shifts to enterprise delivery—KRaft controllers, deployment choices (on-prem, cloud, hybrid), security, project organization, testing, and operations. - **Ending (~79%–100%)**: Covers cluster evolution, monitoring, performance tuning, disaster recovery with RTO/RPO engineering, and forward-looking topics like Kafka as orchestration, WebAssembly integration, diskless Kafka, and AI/ML use cases. ## 【Key Takeaways】 - **Kafka is a distributed commit log, not just a message broker** (Early): Understanding this distinction shapes every architectural decision—from retention policies to replayability. It's why Kafka excels at both messaging and data storage. - **Partitioning is the key to scalability and ordering** (Early): Partitions enable parallel processing but constrain ordering guarantees. Choosing the right partition count and key strategy is a fundamental design decision that's hard to change later. - **Data contracts are a first-class architectural concern** (Middle): Without server-side validation, Kafka shifts responsibility to producers and consumers. Schema Registry with compatibility rules is essential for safe evolution—and you need a governance strategy, not just tooling. - **Event design matters more than message format** (Middle): Distinguishing fact events from delta events, and choosing between composite, atomic, or aggregate structures, determines whether your system can handle state changes cleanly and evolve over time. - **Kafka's "dumb pipe" philosophy is a feature** (Middle): The broker does no processing—all intelligence lives in clients and stream processors. This keeps Kafka simple and scalable but pushes complexity to your applications. - **Delivery guarantees require deliberate design** (Middle): Idempotent producers, transactions, and the outbox pattern solve real-world consistency problems. These patterns are essential for microservices that need exactly-once semantics. - **Operations are architecture** (Late): KRaft controllers, deployment choices, security configuration, monitoring, and disaster recovery planning are architectural decisions—not afterthoughts. RTO/RPO targets should drive your cluster design. - **Kafka's ecosystem is evolving rapidly** (Late): From diskless brokers to WebAssembly and AI integration, the platform is expanding beyond traditional streaming. Architects should track these trends for future-proofing decisions. ## 【Reading Tips】 - **Skim the Customer 360 ODS example** if you want the concepts without the running case study—it's woven throughout but the core principles stand alone. - **Deep-read chapters 2 and 6** (cluster architecture and data contracts)—these are the most architecturally dense and where the book's unique value lies. - **Use chapter 5 as a decision aid** when evaluating whether Kafka is right for your use case—it honestly covers limitations and alternatives. - **Pay attention to "Field notes" sections**—they translate theory into practical decision-making, which is exactly what architects need. - **Skip the code-level details** if you're not implementing directly—the book stays above code, but some sections (Kafka Streams, testing) still get implementation-heavy. ## 【Coverage Limits】 This guide synthesizes the table of contents, preface, and early chapter content. Detailed technical discussions of specific APIs, configuration parameters, and code examples are not covered here—the excerpts provide structure and key concepts rather than full implementation details. ##