Kafka Producer transactions incrementing offsets twice for a message
Groups Cuộc trò chuyện Tất cả nhóm và bài viết Cuộc trò chuyện mới Nhóm của tôi Các nhóm gần đây Nhóm yêu thích Cuộc trò chuyện yêu thích Confluent Platform Cuộc trò chuyện 99+ Giới thiệu Bảo mật • Điều khoản Kafka Producer transactions incrementing offsets twice for a message 3.865 lượt xem Chuyển tới thư đầu tiên chưa đọc Đăng ký Sampath Kumar chưa đọc, 04:05:43 12 thg 1, 2018 đến Confluent Platform We are trying to implement exactly once semantics using transaction. When a message is pushed to topic the offset gets incremented by two instead of one. ex : my current offset is 10, i am pushing message to topic. my next offset needs to be 11. however next offset is updated as 12. In the consumer side received only 1 message and my consumer group says lag of 1 even though my consumer is still running. This is happening only when we use transactions. When next message is pushed offset moved to 14 and consumer received 1 message only. We are using Kafka 1.0 f
Kafka Producer transactions incrementing offsets twice for a message Groups Groups     Kafka Producer transactions incrementing offsets twice for a message 4,192 views Skip to first unread message  Sampath Kumar unread, Jan 11, 2018, 1:05:43 PM 1/11/18    Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses
Explore this link on the map →related reading
- Building Systems Using Transactions in Apache Kafka®developer.confluent.io
- Exactly-once Semantics is Possible: Here's How Apache Kafka Does itconfluent.io
- Multi-Threaded Messaging with the Apache Kafka Consumerconfluent.io
- On Idempotency Keys - Gunnar Morlingmorling.dev
- DBMS Musings: It’s Time to Move on from Two Phase Commitdbmsmusings.blogspot.com
- Students' Guide to Raft :: Jon Gjengsetthesquareplanet.com
- Implementing a Kafka Producer and Consumer In Golang (With Full Examples) For Productionsohamkamani.com
- Schema Registry in Kafka: Avro, JSON and Protobuf | by Ismael Sánchez Chaves | .Net Programming | Mediummedium.com
- Track every PostgreSQL data change using Debezium - DEV Communitydev.to
- Best practices for right-sizing your Apache Kafka clusters to optimize performance and cost | AWS Big Data Blogaws.amazon.com
- debezium-examples/postgres-toast at main · debezium/debezium-examples · GitHubgithub.com
- Flink CDC for Postgres: Lessons Learned | sap1ens.comsap1ens.com