Skip to content
代码片段 群组 项目
  1. 9月 26, 2022
  2. 9月 22, 2022
    • Jeff Kim's avatar
      KAFKA-13778: Fetch from follower should never run the preferred read replica... · bd6d3698
      Jeff Kim 创作于
      KAFKA-13778: Fetch from follower should never run the preferred read replica selection (#11965) (#801)
      
      The current preferred read replica selection logic relies on `partition.leaderReplicaIdOpt` to determine if the selection must be run. The issue is that `partition.leaderReplicaIdOpt` is defined for both the leader and the followers thus the logic is ran all the time. The impact is not too bad as the leader is selected most of the time when the logic is ran by the follower and the leader is filtered out. However there are cases where the selection on a follower could redirect the consumer to another follower under certain rare conditions. For instance with the `RackAwareReplicaSelector `, the follower must have stale replica states from a previous leadership and must have other followers in the same rack for instance. Other implementation of the selection logic could be more impacted.
      
      This patch ensures that the preferred read replica selection is only ran by the leader.
      
      Reviewers: David Jacot <djacot@confluent.io>
      
      Co-authored-by: default avatarbozhao12 <102274736+bozhao12@users.noreply.github.com>
      bd6d3698
  3. 9月 15, 2022
    • Feng Min's avatar
      Remove unused import · a9b49f40
      Feng Min 创作于
      a9b49f40
    • Feng Min's avatar
      Merge remote-tracking branch 'apache-kafka/3.2' into sync-upstream-3.2-14-SEP-2022 · b6875780
      Feng Min 创作于
      * apache-kafka/3.2: (45 commits)
        MINOR: Bump version in upgrade guide to 3.2.3
        KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits (#12626)
        KAFKA-14196; Do not continue fetching partitions awaiting auto-commit prior to revocation (#12603)
        MINOR: 3.2 branch version to 3.2.3-SNAPSHOT
        Bump version to 3.2.2
        Upgrade Netty and Jackson versions for CVE fixes [KAFKA-14044] (#12376)
        KAFKA-14194: Fix NPE in Cluster.nodeIfOnline (#12584)
        MINOR: Update LICENSE-binary
        MINOR: Align Scala version to 2.13.8
        MINOR: Bump version in upgrade guide to 3.2.2
        ...
      b6875780
  4. 9月 13, 2022
    • Tom Bentley's avatar
      013d2499
    • Jason Gustafson's avatar
      KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits (#12626) · e72db098
      Jason Gustafson 创作于
      
      Asynchronous offset commits may throw an unexpected WakeupException following #11631 and #12244. This patch fixes the problem by passing through a flag to ensureCoordinatorReady to indicate whether wakeups should be disabled. This is used to disable wakeups in the context of asynchronous offset commits. All other uses leave wakeups enabled.
      
      Note: this patch builds on top of #12611.
      
      Co-Authored-By: default avatarGuozhang Wang <wangguoz@gmail.com>
      
      Reviewers: Luke Chen <showuon@gmail.com>
      e72db098
    • Philip Nee's avatar
      KAFKA-14196; Do not continue fetching partitions awaiting auto-commit prior to revocation (#12603) · 56baf644
      Philip Nee 创作于
      When auto-commit is enabled with the "eager" rebalance strategy, the consumer will commit all offsets prior to revocation. Following recent changes, this offset commit is done asynchronously, which means there is an opportunity for fetches to continue returning data to the application. When this happens, the progress is lost following revocation, which results in duplicate consumption. This patch fixes the problem by adding a flag in `SubscriptionState` to ensure that partitions which are awaiting revocation will not continue being fetched.
      
      Reviewers: Luke Chen <showuon@gmail.com>, Jason Gustafson <jason@confluent.io>
      56baf644
  5. 9月 09, 2022
  6. 9月 02, 2022
  7. 8月 26, 2022
    • Divij Vaidya's avatar
      KAFKA-14122: Fix flaky test DynamicBrokerReconfigurationTest#testKeyStoreAlter (#12452) · 2e229db6
      Divij Vaidya 创作于
      Reviewers: Mickael Maison <mickael.maison@gmail.com>
      2e229db6
    • Colin Patrick McCabe's avatar
      KAFKA-13835: Fix two bugs related to dynamic broker configs in KRaft (#12063) · 44e41972
      Colin Patrick McCabe 创作于
      Fix two bugs related to dynamic broker configs in KRaft. The first bug is that we are calling reloadUpdatedFilesWithoutConfigChange when a topic configuration is changed, but not when a
      broker configuration is changed. This is backwards. This function must be called only for broker
      configs, and never for topic configs or cluster configs.
      
      The second bug is that there were several configurations such as max.connections which are related
      to broker listeners, but which do not involve changing the registered listeners. We should support
      these configurations in KRaft. This PR fixes the configuration change validation to support this case.
      
      Reviewers: Jason Gustafson <jason@confluent.io>, Matthew de Detrich <mdedetrich@gmail.com>
      44e41972
  8. 8月 25, 2022
  9. 8月 13, 2022
  10. 8月 09, 2022
  11. 8月 06, 2022
  12. 8月 05, 2022
  13. 7月 30, 2022
  14. 7月 29, 2022
  15. 7月 22, 2022
  16. 7月 21, 2022
  17. 7月 20, 2022
    • Shawn's avatar
      KAFKA-14024: Consumer keeps Commit offset in onJoinPrepare in Cooperative rebalance (#12349) · d8541b20
      Shawn 创作于
      In KAFKA-13310, we tried to fix a issue that consumer#poll(duration) will be returned after the provided duration. It's because if rebalance needed, we'll try to commit current offset first before rebalance synchronously. And if the offset committing takes too long, the consumer#poll will spend more time than provided duration. To fix that, we change commit sync with commit async before rebalance (i.e. onPrepareJoin).
      
      However, in this ticket, we found the async commit will keep sending a new commit request during each Consumer#poll, because the offset commit never completes in time. The impact is that the existing consumer will be kicked out of the group after rebalance timeout without joining the group. That is, suppose we have consumer A in group G, and now consumer B joined the group, after the rebalance, only consumer B in the group.
      
      Besides, there's also another bug found during fixing this bug. Before KAFKA-13310, we commitOffset sync with rebalanceTimeout, which will retry when retriable error until timeout. After KAFKA-13310, we thought we have retry, but we'll retry after partitions revoking. That is, even though the retried offset commit successfully, it still causes some partitions offsets un-committed, and after rebalance, other consumers will consume overlapping records.
      
      Reviewers: RivenSun <riven.sun@zoom.us>, Luke Chen <showuon@gmail.com>
      d8541b20
    • Walker Carlson's avatar
      Revert "KAFKA-12887 Skip some RuntimeExceptions from exception handler (#11228)" (#12421) · 56a136d8
      Walker Carlson 创作于
      This reverts commit 4835c64f
      
      Reviewers: Matthias J. Sax <matthias@confluent.io>
      56a136d8
  18. 7月 19, 2022
  19. 7月 18, 2022
  20. 7月 17, 2022
  21. 7月 12, 2022
    • Kirk True's avatar
      KAFKA-14062: OAuth client token refresh fails with SASL extensions (#12398) · c873d9d7
      Kirk True 创作于
      
      - Different objects should be considered unique even with same content to support logout
      - Added comments for SaslExtension re: removal of equals and hashCode
      - Also swapped out the use of mocks in exchange for *real* SaslExtensions so that we exercise the use of default equals() and hashCode() methods.
      - Updates to implement equals and hashCode and add tests in SaslExtensionsTest to confirm
      
      Co-authored-by: default avatarPurshotam Chauhan <pchauhan@confluent.io>
      
      Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
      c873d9d7
    • Eugene Tolbakov's avatar
      KAFKA-14013: Limit the length of the `reason` field sent on the wire (#12388) · 74e24dee
      Eugene Tolbakov 创作于
      KIP-800 added the `reason` field to the JoinGroupRequest and the LeaveGroupRequest as I mean to provide more information to the group coordinator. In https://issues.apache.org/jira/browse/KAFKA-13998, we discovered that the size of the field is limited to 32767 chars by our serialisation mechanism. At the moment, the field either provided directly by the user or constructed internally is directly set regardless of its length.
      
      This patch sends only the first 255 chars of the used provided or internally generated reason on the wire. Given the purpose of this field, that seems acceptable and that should still provide enough information to operators to understand the cause of a rebalance.
      
      Reviewers: David Jacot <djacot@confluent.io>
      74e24dee
  22. 7月 11, 2022
    • Jason Gustafson's avatar
      KAFKA-14055; Txn markers should not be removed by matching records in the offset map (#12390) · b29cb162
      Jason Gustafson 创作于
      When cleaning a topic with transactional data, if the keys used in the user data happen to conflict with the keys in the transaction markers, it is possible for the markers to get removed before the corresponding data from the transaction is removed. This results in a hanging transaction or the loss of the transaction's atomicity since it would effectively get bundled into the next transaction in the log. Currently control records are excluded when building the offset map, but not when doing the cleaning. This patch fixes the problem by checking for control batches in the `shouldRetainRecord` callback.
      
      Reviewers: Jun Rao <junrao@gmail.com>
      b29cb162
  23. 7月 09, 2022
    • Divij Vaidya's avatar
      KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection (#12381) · 1180f5e1
      Divij Vaidya 创作于
      What:
      When a certificate is rotated on a broker via dynamic configuration and the previous certificate expires, the broker to controller connection starts failing with SSL Handshake failed.
      
      Why:
      A similar fix was earlier performed in #6721 but when BrokerToControllerChannelManager was introduced in v2.7, we didn't enable dynamic reconfiguration for it's channel.
      
      Summary of testing strategy (including rationale)
      Add a test which fails prior to the fix done in the PR and succeeds afterwards. The bug wasn't caught earlier because there was no test coverage to validate the scenario.
      
      Reviewers: Luke Chen <showuon@gmail.com>
      1180f5e1
    • Lucas Bradstreet's avatar
      MINOR: kafka system tests should support larger EBS volumes for newer instances (#12382) (#741) · 1a2d6e4c
      Lucas Bradstreet 创作于
      When running with 4th generation instances supporting EBS only, we need
      to use a larger volume or else we run out of  disk space during a system
      test run.
      
      This change also parameterizes the instance type as an env variable for
      easier testing.
      
      Reviewers: David Jacot <djacot@confluent.io>
      1a2d6e4c
加载中