Kafka Go client: No Producer error during connection ?

Although its rare, but there are times when you actually want to see errors in your code – more importantly, at the right time !

Kafka Go Producer behaviour

You need to be mindful of this while using the Kafka Go client producer. For e.g. if you were to supply an incorrect value for the Kafka broker …

	func main() {
	kafkaBroker := "foo:9092"
	p, err := kafka.NewProducer(&kafka.ConfigMap{"bootstrap.servers": kafkaBroker})
	if err != nil {
	fmt.Println("producer creation failed ", err.Error())
	return
	}
	topic := "bar"
	partition := kafka.TopicPartition{Topic: &topic, Partition: kafka.PartitionAny}
	msg := &kafka.Message{TopicPartition: partition, Key: []byte("hello"), Value: []byte("world")}
	err = p.Produce(msg, nil)
	fmt.Println("done…")
	}

view raw snippet1.go hosted with ❤ by GitHub

no error

… assuming you don’t have a Kafka broker at foo:9092, you would expect that the above code will respond with producer creation failed along with the specific error details. Instead, the flow carries on and ends by printing done. This is because the error returned by Produce is only in case message does not get enqueued to the internal librdkafka queue

You should…

Hook up to the delivery producer reports channel (using Producer.Events()) in order to catch this error – better late than never right !

	package main
	import (
	"fmt"
	"github.com/confluentinc/confluent-kafka-go/kafka"
	)
	func main() {
	kafkaBroker := "foo:9092"
	p, err := kafka.NewProducer(&kafka.ConfigMap{"bootstrap.servers": kafkaBroker})
	if err != nil {
	fmt.Println("producer creation failed ", err.Error())
	return
	}
	topic := "bar"
	partition := kafka.TopicPartition{Topic: &topic, Partition: kafka.PartitionAny}
	msg := &kafka.Message{TopicPartition: partition, Key: []byte("hello"), Value: []byte("world")}
	err = p.Produce(msg, nil)
	fmt.Println("done…")
	event := <-p.Events()
	switch e := event.(type) {
	case kafka.Error:
	pErr := e
	fmt.Println("producer error", pErr.String())
	default:
	fmt.Println("Kafka producer event", e)
	}
	}

view raw snippet2.go hosted with ❤ by GitHub

Now, the error is rightfully returned – producer error foo:9092/bootstrap: Failed to resolve 'foo:9092': nodename nor servname provided, or not known (after 1543569582778ms in state INIT)

You can also provide your own channel to the Produce method to receive delivery events. Only the error is handled and the other possible event i.e. *kafka.Message is ignored under the default case umbrella

Cheers!

sdfsdfsdfsdfsdf