Implementing a Kafka Producer and Consumer In Golang (With Full Examples) For Production
Like what I write? Let me know your email, and I'll send you more posts like this. No spam, I promise! In this post we will learn how to create a Kafka producer and consumer in Go. We will also look at how to tune some configuration options to make our application production-ready. Kafka is an open-source event streaming platform, used for publishing and processing events at high-throughput. There are a lot of popular libraries for Go in order to interface with Kafka. For this post, we will be using the kafka-go library (but the same concepts will apply for any other library as well). If you want to skip the explanation and see the code, you can view it on Github First, make sure you have a running Kafka cluster on your machine. Before diving into the code, we should know about brokers and topics, which will be needed by both the producer and consumer. A “topic” can be thought of as a distinct queue or channel where messages are sent. For most production applications, there isn’t a sin
Implementing a Kafka Producer and Consumer In Golang (With Full Examples) For Production Sep 20, 2020 By Soham Kamani Table of Contents Getting Started Creating the Kafka Producer Creating the Kafka Consumer Tuning Kafka Client Configuration Further Reading In this post we will learn how to create a Kafka producer and consumer in Go . We will also look at how to tune some configuration options to make our application production-ready . Kafka is an open-source event streaming platform, used for publishing and processing events at high-throughput. There are a lot of popular libraries for Go in o
related reading
- Apache Kafka and Go - Getting Started Tutorialdeveloper.confluent.io
- How to Build a Real-Time Notification System with Go and Kafkafreecodecamp.org
- Why is Kafka so fast? How does it work?blog.bytebytego.com
- Designkafka.apache.org
- Mediumzicodeng.medium.com
- Schema Registry in Kafka: Avro, JSON and Protobuf | by Ismael Sánchez Chaves | .Net Programming | Mediummedium.com
- Kafka Producer transactions incrementing offsets twice for a messagegroups.google.com
- Multi-Threaded Messaging with the Apache Kafka Consumerconfluent.io
- Exactly-once Semantics is Possible: Here's How Apache Kafka Does itconfluent.io
- Best practices for right-sizing your Apache Kafka clusters to optimize performance and cost | AWS Big Data Blogaws.amazon.com
- When to use RabbitMQ over Kafka? - Stack Overflowstackoverflow.com
- Notes on structured concurrency, or: Go statement considered harmful - njs blogvorpus.org