- 9月 26, 2022
- 9月 22, 2022
-
-
由 Jeff Kim 创作于
KAFKA-13778: Fetch from follower should never run the preferred read replica selection (#11965) (#801) The current preferred read replica selection logic relies on `partition.leaderReplicaIdOpt` to determine if the selection must be run. The issue is that `partition.leaderReplicaIdOpt` is defined for both the leader and the followers thus the logic is ran all the time. The impact is not too bad as the leader is selected most of the time when the logic is ran by the follower and the leader is filtered out. However there are cases where the selection on a follower could redirect the consumer to another follower under certain rare conditions. For instance with the `RackAwareReplicaSelector `, the follower must have stale replica states from a previous leadership and must have other followers in the same rack for instance. Other implementation of the selection logic could be more impacted. This patch ensures that the preferred read replica selection is only ran by the leader. Reviewers: David Jacot <djacot@confluent.io> Co-authored-by: bozhao12 <102274736+bozhao12@users.noreply.github.com>
-
- 9月 15, 2022
-
-
由 Feng Min 创作于
-
由 Feng Min 创作于
* apache-kafka/3.2: (45 commits) MINOR: Bump version in upgrade guide to 3.2.3 KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits (#12626) KAFKA-14196; Do not continue fetching partitions awaiting auto-commit prior to revocation (#12603) MINOR: 3.2 branch version to 3.2.3-SNAPSHOT Bump version to 3.2.2 Upgrade Netty and Jackson versions for CVE fixes [KAFKA-14044] (#12376) KAFKA-14194: Fix NPE in Cluster.nodeIfOnline (#12584) MINOR: Update LICENSE-binary MINOR: Align Scala version to 2.13.8 MINOR: Bump version in upgrade guide to 3.2.2 ...
-
- 9月 13, 2022
-
-
由 Tom Bentley 创作于
-
由 Jason Gustafson 创作于
Asynchronous offset commits may throw an unexpected WakeupException following #11631 and #12244. This patch fixes the problem by passing through a flag to ensureCoordinatorReady to indicate whether wakeups should be disabled. This is used to disable wakeups in the context of asynchronous offset commits. All other uses leave wakeups enabled. Note: this patch builds on top of #12611. Co-Authored-By: Guozhang Wang <wangguoz@gmail.com> Reviewers: Luke Chen <showuon@gmail.com>
-
由 Philip Nee 创作于
When auto-commit is enabled with the "eager" rebalance strategy, the consumer will commit all offsets prior to revocation. Following recent changes, this offset commit is done asynchronously, which means there is an opportunity for fetches to continue returning data to the application. When this happens, the progress is lost following revocation, which results in duplicate consumption. This patch fixes the problem by adding a flag in `SubscriptionState` to ensure that partitions which are awaiting revocation will not continue being fetched. Reviewers: Luke Chen <showuon@gmail.com>, Jason Gustafson <jason@confluent.io>
-
- 9月 09, 2022
-
-
由 Mickael Maison 创作于
-
由 Mickael Maison 创作于
-
由 Thomas Cooper 创作于
Reviewers: Luke Chen <showuon@gmail.com>
-
由 Andrew Dean 创作于
When utilizing the rack-aware consumer configuration and rolling updates are being applied to the Kafka brokers the metadata updates can be in a transient state and a given topic-partition can be missing from the metadata. This seems to resolve itself after a bit of time but before it can resolve the `Cluster.nodeIfOnline` method throws an NPE. This patch checks to make sure that a given topic-partition has partition info available before using that partition info. Reviewers: David Jacot <djacot@confluent.io>
-
- 9月 02, 2022
-
-
由 Mickael Maison 创作于
-
由 Mickael Maison 创作于
-
由 Mickael Maison 创作于
-
由 Manikumar Reddy 创作于
This adds a new configuration `sasl.server.max.receive.size` that sets the maximum receive size for requests before and during authentication. Reviewers: Tom Bentley <tbentley@redhat.com>, Mickael Maison <mickael.maison@gmail.com> Co-authored-by: Manikumar Reddy <manikumar.reddy@gmail.com> Co-authored-by: Mickael Maison <mickael.maison@gmail.com>
-
由 Colin Patrick McCabe 创作于
When deserializing KRPC (which is used for RPCs sent to Kafka, Kafka Metadata records, and some other things), check that we have at least N bytes remaining before allocating an array of size N. Remove DataInputStreamReadable since it was hard to make this class aware of how many bytes were remaining. Instead, when reading an individual record in the Raft layer, simply create a ByteBufferAccessor with a ByteBuffer containing just the bytes we're interested in. Add SimpleArraysMessageTest and ByteBufferAccessorTest. Also add some additional tests in RequestResponseTest. Reviewers: Tom Bentley <tbentley@redhat.com>, Mickael Maison <mickael.maison@gmail.com>, Colin McCabe <colin@cmccabe.xyz> Co-authored-by: Colin McCabe <colin@cmccabe.xyz> Co-authored-by: Manikumar Reddy <manikumar.reddy@gmail.com> Co-authored-by: Mickael Maison <mickael.maison@gmail.com>
-
- 8月 26, 2022
-
-
由 Divij Vaidya 创作于
Reviewers: Mickael Maison <mickael.maison@gmail.com>
-
由 Colin Patrick McCabe 创作于
Fix two bugs related to dynamic broker configs in KRaft. The first bug is that we are calling reloadUpdatedFilesWithoutConfigChange when a topic configuration is changed, but not when a broker configuration is changed. This is backwards. This function must be called only for broker configs, and never for topic configs or cluster configs. The second bug is that there were several configurations such as max.connections which are related to broker listeners, but which do not involve changing the registered listeners. We should support these configurations in KRaft. This PR fixes the configuration change validation to support this case. Reviewers: Jason Gustafson <jason@confluent.io>, Matthew de Detrich <mdedetrich@gmail.com>
-
- 8月 25, 2022
-
-
由 xjin-Confluent 创作于
-
- 8月 13, 2022
-
-
由 Derek Troy-West 创作于
Update notable changes documentation to mention requiring IDEMPOTENT_WRITE permission when producing messages with default/idempotent configuration and broker version lower than 2.8.0. Reviewers: Ismael Juma <ismael@juma.me.uk>, Luke Chen <showuon@gmail.com>
-
- 8月 09, 2022
-
-
由 Aneesh Garg 创作于
Jira: https://confluentinc.atlassian.net/browse/KSECURITY-483 Migrating log4j12 to reload4j Migrating slf4j-log4j12 to slf4j-reload4j.jar
-
- 8月 06, 2022
-
-
由 Andrew Borley 创作于
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Aaron Freeland <afreeland@gmail.com>
-
- 8月 05, 2022
-
-
由 Aneesh Garg 创作于
Jira: https://confluentinc.atlassian.net/browse/KSECURITY-509 Upgrading netty to v4.1.79.Final
-
由 David Arthur 创作于
-
由 David Arthur 创作于
Enable some of the dynamic broker reconfiguration tests in KRaft mode
-
- 7月 30, 2022
-
-
由 Confluent Jenkins Bot 创作于
-
- 7月 29, 2022
-
-
由 David Arthur 创作于
-
- 7月 22, 2022
-
-
由 David Arthur 创作于
-
由 Viktor Somogyi-Vass 创作于
Reviewers: Luke Chen <showuon@gmail.com>
-
- 7月 21, 2022
-
-
由 David Arthur 创作于
Reviewers: Randall Hauch <rhauch@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
-
- 7月 20, 2022
-
-
由 Shawn 创作于
In KAFKA-13310, we tried to fix a issue that consumer#poll(duration) will be returned after the provided duration. It's because if rebalance needed, we'll try to commit current offset first before rebalance synchronously. And if the offset committing takes too long, the consumer#poll will spend more time than provided duration. To fix that, we change commit sync with commit async before rebalance (i.e. onPrepareJoin). However, in this ticket, we found the async commit will keep sending a new commit request during each Consumer#poll, because the offset commit never completes in time. The impact is that the existing consumer will be kicked out of the group after rebalance timeout without joining the group. That is, suppose we have consumer A in group G, and now consumer B joined the group, after the rebalance, only consumer B in the group. Besides, there's also another bug found during fixing this bug. Before KAFKA-13310, we commitOffset sync with rebalanceTimeout, which will retry when retriable error until timeout. After KAFKA-13310, we thought we have retry, but we'll retry after partitions revoking. That is, even though the retried offset commit successfully, it still causes some partitions offsets un-committed, and after rebalance, other consumers will consume overlapping records. Reviewers: RivenSun <riven.sun@zoom.us>, Luke Chen <showuon@gmail.com>
-
由 Walker Carlson 创作于
This reverts commit 4835c64f Reviewers: Matthias J. Sax <matthias@confluent.io>
-
- 7月 19, 2022
-
-
由 Christopher L. Shannon 创作于
Make sure to ack all records where produce failed, when a connector's `errors.tolerance` config property is set to `all`. Acking is essential so that the task will continue to commit future record offsets properly and remove the records from internal tracking, preventing a memory leak. Reviewers: Chris Egerton <fearthecellos@gmail.com>, Randall Hauch <rhauch@gmail.com>
-
- 7月 18, 2022
-
-
由 Okada Haruki 创作于
Currently, preferredReplicaImbalanceCount calculation has a race that becomes negative when topic deletion is initiated simultaneously. This PR addresses the problem by fixing cleanPreferredReplicaImbalanceMetric to be called only once per topic-deletion procedure Reviewers: Luke Chen <showuon@gmail.com>
-
- 7月 17, 2022
-
-
由 Lucas Bradstreet 创作于
With 6f81bb18 merged, we can now use c4.xlarge instances with our system tests, which are much more reliable for spot than m3.xlarge.
-
- 7月 12, 2022
-
-
由 Kirk True 创作于
- Different objects should be considered unique even with same content to support logout - Added comments for SaslExtension re: removal of equals and hashCode - Also swapped out the use of mocks in exchange for *real* SaslExtensions so that we exercise the use of default equals() and hashCode() methods. - Updates to implement equals and hashCode and add tests in SaslExtensionsTest to confirm Co-authored-by: Purshotam Chauhan <pchauhan@confluent.io> Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
-
由 Eugene Tolbakov 创作于
KIP-800 added the `reason` field to the JoinGroupRequest and the LeaveGroupRequest as I mean to provide more information to the group coordinator. In https://issues.apache.org/jira/browse/KAFKA-13998, we discovered that the size of the field is limited to 32767 chars by our serialisation mechanism. At the moment, the field either provided directly by the user or constructed internally is directly set regardless of its length. This patch sends only the first 255 chars of the used provided or internally generated reason on the wire. Given the purpose of this field, that seems acceptable and that should still provide enough information to operators to understand the cause of a rebalance. Reviewers: David Jacot <djacot@confluent.io>
-
- 7月 11, 2022
-
-
由 Jason Gustafson 创作于
When cleaning a topic with transactional data, if the keys used in the user data happen to conflict with the keys in the transaction markers, it is possible for the markers to get removed before the corresponding data from the transaction is removed. This results in a hanging transaction or the loss of the transaction's atomicity since it would effectively get bundled into the next transaction in the log. Currently control records are excluded when building the offset map, but not when doing the cleaning. This patch fixes the problem by checking for control batches in the `shouldRetainRecord` callback. Reviewers: Jun Rao <junrao@gmail.com>
-
- 7月 09, 2022
-
-
由 Divij Vaidya 创作于
What: When a certificate is rotated on a broker via dynamic configuration and the previous certificate expires, the broker to controller connection starts failing with SSL Handshake failed. Why: A similar fix was earlier performed in #6721 but when BrokerToControllerChannelManager was introduced in v2.7, we didn't enable dynamic reconfiguration for it's channel. Summary of testing strategy (including rationale) Add a test which fails prior to the fix done in the PR and succeeds afterwards. The bug wasn't caught earlier because there was no test coverage to validate the scenario. Reviewers: Luke Chen <showuon@gmail.com>
-
由 Lucas Bradstreet 创作于
When running with 4th generation instances supporting EBS only, we need to use a larger volume or else we run out of disk space during a system test run. This change also parameterizes the instance type as an env variable for easier testing. Reviewers: David Jacot <djacot@confluent.io>
-