Tag Archives: Distributed systems

Kafka producer and partitions

There are only a few possible ways to specify partitions while using the Kafka Producer API Just specify it in the ProducerRecord itself If key is not null, (by default) Kafka will hash your key and calculate the partition If key … Continue reading

Posted in Distributed systems, Kafka | Tagged , , | Leave a comment

Kafka Streams state stores…

This blog explores some common aspects of state stores in Kafka Streams… Default state store By default, Kafka Streams uses the RocksDB as it’s default state store In-memory or persistent ? This parameter of the state store is configurable. RocksDB can … Continue reading

Posted in Distributed systems, Kafka | Tagged , , , , | 1 Comment

Docker-ized Kafka Streams applications

Here is another example of a Kafka Streams based application.. this time, it’s about running it in Docker containers – spawn more containers to distribute the processing load. More details in the README Cheers!

Posted in Distributed systems, Kafka | Tagged , , , , | Leave a comment

Kafka Streams based application

A Kafka Streams sample application is available on Github… This is a microservice (packaged in form of an Uber JAR) which uses the Kafka Streams Processor (low level) API to calculate the Cumulative Moving Average of the CPU metrics of each machine … Continue reading

Posted in Distributed systems, Kafka | Tagged , , , | Leave a comment

Tip: ClassCastException in Kafka Streams reducer

If you use the groupByKey function on a KStream without specifying a Serdes, the (one configured in the StreamsConfig will be used by default e.g. in the below snippet, it’s Serdes.String(). As a result, you will face a ClassCastExcpetion in case you execute … Continue reading

Posted in Distributed systems, Kafka | Tagged , , , , , | Leave a comment

Kafka Partitioning…

Partitions are the key to scalability attributes of Kafka. Developers can also implement custom partitioning algorithm to override the default partition assignment behavior. This post will briefly cover Partitions in general Data distribution, default partitioning, and Example of custom partitioning … Continue reading

Posted in Distributed systems, Kafka | Tagged , , , , | 1 Comment

Hello (distributed) world !

Hi there … ! I am the guy from ‘Thinking in Java EE‘ have been interested in distributed systems and its equivalent software solutions for some time now.. So I thought of starting off yet another blog (the one you’re … Continue reading

Posted in Distributed systems | Tagged , , , | 1 Comment