Kafka producer and partitions

There are only a few possible ways to specify partitions while using the Kafka Producer API

  • Just specify it in the ProducerRecord itself
  • If key is not null, (by default) Kafka will hash your key and calculate the partition
  • If key is null, (by default) Kafka will round-robin b/w all the partitions (to load balance the data)
  • If not, just use a custom Partitioner

If interested, you can also check out the Kafka Partitioning blog

Cheers!

Advertisements

About Abhishek

Java EE & distributed systems junkie who frequently blogs at abhirockzz.wordpress.com as well as simplydistributed.wordpress.com. Oh, I have also authored a few (mini) books, articles, Refcards etc. :-)
This entry was posted in Distributed systems, Kafka and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s