Category Archives: Kafka

Accessing Kafka on host machine from minikube pods

Posted on December 14, 2018 by Abhishek

There are times when you want to access processes running on your host machine e.g. databases etc. from your minikube Kubernetes cluster in Virtual Box (or any other supported provider) For details, continue reading this blog on Medium Cheers!

Posted in Distributed systems, Kafka, Kubernetes | Tagged , , | Leave a comment

Kafka Go client: No Producer error during connection ?

Posted on November 30, 2018 by Abhishek

Although its rare, but there are times when you actually want to see errors in your code – more importantly, at the right time ! Kafka Go Producer behaviour You need to be mindful of this while using the Kafka … Continue reading →

Posted in Distributed systems, go, Kafka, messaging | Tagged , , | Leave a comment

Kafka Go client quick start

Posted on September 14, 2018 by Abhishek

Here is a Docker based example for Kafka using Go client Overview Simple producer and consumer apps Confluent Go client for Kafka its based on librdkafka (Kafka C client) client setup part is taken care of with a pre-built Docker image – … Continue reading →

Posted in Distributed systems, Kafka, messaging | Leave a comment

NATS & Kafka: random notes

Posted on March 30, 2018 by Abhishek

This is not NATS vs Kafka by any means – just jotting down some info. NATS recently joined CNCF (which host projects like Kubernetes, Prometheus etc. – look at the dominance of Golang here!) and that’s when it caught my attention … Continue reading →

Posted in Distributed systems, Kafka, messaging | Tagged , , , | Leave a comment

Session @ Oracle Code San Francisco 2017

Posted on October 8, 2017 by Abhishek

Here are the details for my talk @ Oracle Code (a track at JavaOne 2017) – Streaming Solutions for Real time problems (Stream Processing solutions using Apache Kafka, Kafka Streams and Redis) Code (Github) – https://github.com/abhirockzz/accs-ehcs-stream-processing Video Slides Streaming Solutions for … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , , , , , , , , | Leave a comment

Debezium test drive

Posted on June 9, 2017 by Abhishek

Debezium is an open source, distributed change data capture system built on top of Apache Kafka. I tried it out and the project is available on Github Setup Details are in the README. It uses the Debezium tutorial as a … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , , , , , | 1 Comment

Kafka & Websocket

Posted on June 1, 2017 by Abhishek

For those who are interested in an example of Kafka working with the (Java EE) Websocket API, please check out this blog . There is an associated Github project as well Cheers!

Posted in Distributed systems, Kafka | Tagged , , | Leave a comment

Kafka producer and partitions

Posted on May 30, 2017 by Abhishek

There are only a few possible ways to specify partitions while using the Kafka Producer API Just specify it in the ProducerRecord itself If key is not null, (by default) Kafka will hash your key and calculate the partition If key … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , | Leave a comment

Kafka Streams state stores…

Posted on March 21, 2017 by Abhishek

This blog explores some common aspects of state stores in Kafka Streams… Default state store By default, Kafka Streams uses the RocksDB as it’s default state store In-memory or persistent ? This parameter of the state store is configurable. RocksDB can … Continue reading →

Posted in Distributed systems, Kafka | Tagged , , , , | 3 Comments

Docker-ized Kafka Streams applications

Posted on March 12, 2017 by Abhishek

Here is another example of a Kafka Streams based application.. this time, it’s about running it in Docker containers – spawn more containers to distribute the processing load. More details in the README Cheers!

Posted in Distributed systems, Kafka | Tagged , , , , | Leave a comment