Tag Archives: Distributed systems

Kafka producer and partitions

Posted on May 30, 2017 by Abhishek

There are only a few possible ways to specify partitions while using the Kafka Producer API Just specify it in the ProducerRecord itself If key is not null, (by default) Kafka will hash your key and calculate the partition If key … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , | Leave a comment

Kafka Streams state stores…

Posted on March 21, 2017 by Abhishek

This blog explores some common aspects of state stores in Kafka Streams… Default state store By default, Kafka Streams uses the RocksDB as it’s default state store In-memory or persistent ? This parameter of the state store is configurable. RocksDB can … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , , , | 3 Comments

Docker-ized Kafka Streams applications

Posted on March 12, 2017 by Abhishek

Here is another example of a Kafka Streams based application.. this time, it’s about running it in Docker containers – spawn more containers to distribute the processing load. More details in the README Cheers!

Posted in Distributed systems, Kafka | Tagged , , , , | Leave a comment

Tip: ClassCastException in Kafka Streams reducer

Posted on February 21, 2017 by Abhishek

If you use the groupByKey function on a KStream without specifying a Serdes, the (one configured in the StreamsConfig will be used by default e.g. in the below snippet, it’s Serdes.String(). As a result, you will face a ClassCastExcpetion in case you execute … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , , , , | Leave a comment

Kafka Partitioning…

Posted on December 13, 2016 by Abhishek

Partitions are the key to scalability attributes of Kafka. Developers can also implement custom partitioning algorithm to override the default partition assignment behavior. This post will briefly cover Partitions in general Data distribution, default partitioning, and Example of custom partitioning … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , , , | 4 Comments

Hello (distributed) world !

Posted on November 22, 2016 by Abhishek

Hi there … ! I am the guy from ‘Thinking in Java EE‘ have been interested in distributed systems and its equivalent software solutions for some time now.. So I thought of starting off yet another blog (the one you’re … Continue reading →

Posted in Distributed systems | Tagged , , , | 1 Comment