Skip to content
代码片段 群组 项目
  1. 3月 22, 2022
  2. 3月 21, 2022
  3. 3月 20, 2022
    • 彭小漪's avatar
      KAFKA-13728: fix PushHttpMetricsReporter no longer pushes metrics when network... · 6145974f
      彭小漪 创作于
      KAFKA-13728: fix PushHttpMetricsReporter no longer pushes metrics when network failure is recovered. (#11879)
      
      The class PushHttpMetricsReporter no longer pushes metrics when network failure is recovered.
      
      I debugged the code and found the problem here: when we submit a task to the ScheduledThreadPoolExecutor that needs to be executed periodically, if the task throws an exception and is not swallowed, the task will no longer be scheduled to execute.
      
      So when an IO exception occasionally occurs on the network, we should swallow it rather than throw it in task HttpReporter.
      
      Reviewers: Guozhang Wang <wangguoz@gmail.com>
      6145974f
  4. 3月 19, 2022
    • José Armando García Sancio's avatar
      KAFKA-13682; KRaft Controller auto preferred leader election (#11893) · 8d6968e8
      José Armando García Sancio 创作于
      Implement auto leader rebalance for KRaft by keeping track of the set of topic partitions which have a leader that is not the preferred replica. If this set is non-empty then schedule a leader balance event for the replica control manager.
      
      When applying PartitionRecords and PartitionChangeRecords to the ReplicationControlManager, if the elected leader is not the preferred replica then remember this topic partition in the set of imbalancedPartitions.
      
      Anytime the quorum controller processes a ControllerWriteEvent it schedules a rebalance operation if the there are no pending rebalance operations, the feature is enabled and there are imbalance partitions.
      
      This KRaft implementation only supports the configurations properties auto.leader.rebalance.enable and leader.imbalance.check.interval.seconds. The configuration property leader.imbalance.per.broker.percentage is not supported and ignored.
      
      Reviewers: Jun Rao <junrao@gmail.com>, David Arthur <mumrah@gmail.com>
      8d6968e8
    • José Armando García Sancio's avatar
      KAFKA-13587; Implement leader recovery for KIP-704 (#11733) · 52621613
      José Armando García Sancio 创作于
      Implementation of the protocol for starting and stopping leader recovery after an unclean leader election. This includes the management of state in the controllers (legacy and KRaft) and propagating this information to the brokers. This change doesn't implement log recovery after an unclean leader election.
      
      Protocol Changes
      ================
      
      For the topic partition state znode, the new field "leader_recovery_state" was added. If the field is missing the value is assumed to be RECOVERED.
      
      ALTER_PARTITION was renamed from ALTER_ISR. The CurrentIsrVersion field was renamed to PartitionEpoch. The new field LeaderRecoveryState was added.
      
      The new field LeaderRecoverState was added to the LEADER_AND_ISR request. The inter broker protocol version is used to determine which version to send to the brokers.
      
      A new tagged field for LeaderRecoveryState was added to both the PartitionRecord and PartitionChangeRecord.
      
      Controller
      ==========
      
      For both the KRaft and legacy controller the LeaderRecoveryState is set to RECOVERING, if the leader was elected out of the ISR, also known as unclean leader election. The controller sets the state back to RECOVERED after receiving an ALTER_PARTITION request with version 0, or with version 1 and with the LeaderRecoveryState set to RECOVERED.
      
      Both controllers preserve the leader recovery state even if the unclean leader goes offline and comes back online before an RECOVERED ALTER_PARTITION is sent.
      
      The controllers reply with INVALID_REQUEST if the ALTER_PARTITION either:
      
          1. Attempts to increase the ISR while the partition is still RECOVERING
          2. Attempts to change the leader recovery state to RECOVERING from a RECOVERED state.
      
      Topic Partition Leader
      ======================
      
      The topic partition leader doesn't implement any log recovery in this change. The topic partition leader immediately marks the partition as RECOVERED and sends that state in the next ALTER_PARTITION request.
      
      Reviewers: Jason Gustafson <jason@confluent.io>
      52621613
  5. 3月 18, 2022
  6. 3月 17, 2022
    • Luke Chen's avatar
      KAFKA-9847: add config to set default store type (KIP-591) (#11705) · fbe7fb94
      Luke Chen 创作于
      Reviewers: Hao Li <hli@confluent.io>, A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Matthias J. Sax <mjsax@apache.org>
      fbe7fb94
    • Levani Kokhreidze's avatar
      KAFKA-6718 / Add rack awareness configurations to StreamsConfig (#11837) · b68463c2
      Levani Kokhreidze 创作于
      This PR is part of KIP-708 and adds rack aware standby task assignment logic.
      
      Rack aware standby task assignment won't be functional until all parts of this KIP gets merged.
      
      Splitting PRs into three smaller PRs to make the review process easier to follow. Overall plan is the following:
      
      :track_next: Rack aware standby task assignment logic #10851
      :track_next: Protocol change, add clientTags to SubscriptionInfoData #10802
      :point_right: Add required configurations to StreamsConfig (public API change, at this point we should have full functionality)
      
      This PR implements last point of the above mentioned plan.
      
      Reviewers: Luke Chen <showuon@gmail.com>, Bruno Cadonna <cadonna@apache.org>
      b68463c2
  7. 3月 16, 2022
  8. 3月 15, 2022
  9. 3月 14, 2022
  10. 3月 12, 2022
    • Hao Li's avatar
      KIP-825: Part 1, add new RocksDBTimeOrderedWindowStore (#11802) · 63ea5db9
      Hao Li 创作于
      Initial State store implementation for TimedWindow and SlidingWindow.
      
      RocksDBTimeOrderedWindowStore.java contains one RocksDBTimeOrderedSegmentedBytesStore which contains index and base schema.
      
      PrefixedWindowKeySchemas.java implements keyschema for time ordered base store and key ordered index store.
      
      Reviewers: James Hughes, Guozhang Wang <wangguoz@gmail.com>
      63ea5db9
    • Hao Li's avatar
      MINOR: fix flaky... · 17988f47
      Hao Li 创作于
      MINOR: fix flaky EosIntegrationTest.shouldCommitCorrectOffsetIfInputTopicIsTransactional[at_least_once] (#11878)
      
      In this test, we started Kafka Streams app and then write to input topic in transaction. It's possible when streams commit offset, transaction hasn't finished yet. So the streams committed offset could be less than the eventual endOffset.
      
      This PR moves the logic of writing to input topic before starting streams app.
      
      Reviewers: John Roesler <vvcephei@apache.org>
      17988f47
  11. 3月 11, 2022
加载中