Kafka producer and partitions

There are only a few possible ways to specify partitions while using the Kafka Producer API

  • Just specify it in the ProducerRecord itself
  • If key is not null, (by default) Kafka will hash your key and calculate the partition
  • If key is null, (by default) Kafka will round-robin b/w all the partitions (to load balance the data)
  • If not, just use a custom Partitioner

If interested, you can also check out the Kafka Partitioning blog

Cheers!

Advertisements

About Abhishek

Loves Java EE, distributed KV stores and messaging systems. Frequently blogs at abhirockzz.wordpress.com as well as simplydistributed.wordpress.com. Oh, I have also authored a few (mini) books, articles, Refcards etc. :-)
This entry was posted in Distributed systems, Kafka and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s