Skip to content
代码片段 群组 项目
  1. 6月 30, 2022
    • Guozhang Wang's avatar
      MINOR: Use mock time in DefaultStateUpdaterTest (#12344) · 3faa6cf6
      Guozhang Wang 创作于
      For most tests we would need an auto-ticking mock timer to work with draining-with-timeout functions.
      For tests that check for never checkpoint we need no auto-ticking timer to control exactly how much time elapsed.
      
      Reviewers: Bruno Cadonna <cadonna@apache.org>
      3faa6cf6
    • Guozhang Wang's avatar
      [9/N][Emit final] Emit final for session window aggregations (#12204) · ababc426
      Guozhang Wang 创作于
      * Add a new API for session windows to range query session window by end time (KIP related).
      * Augment session window aggregator with emit strategy.
      * Minor: consolidated some dup classes.
      * Test: unit test on session window aggregator.
      
      Reviewers: Guozhang Wang <wangguoz@gmail.com>
      ababc426
  2. 6月 29, 2022
  3. 6月 28, 2022
  4. 6月 26, 2022
  5. 6月 25, 2022
    • Bruno Cadonna's avatar
      KAFKA-10199: Expose tasks in state updater (#12312) · 1ceaf300
      Bruno Cadonna 创作于
      This PR exposes the tasks managed by the state updater. The state updater manages all tasks that were added to the state updater and that have not yet been removed from it by draining one of the output queues.
      
      Reviewers: Guozhang Wang <wangguoz@gmail.com>
      1ceaf300
  6. 6月 24, 2022
  7. 6月 23, 2022
    • David Arthur's avatar
      KAFKA-13966 Prepend bootstrap metadata to controller queue (#12269) · c6c9da02
      David Arthur 创作于
      Also fixes flaky QuorumControllerTest#testInvalidBootstrapMetadata
      
      Reviewers: Jason Gustafson <jason@confluent.io>
      c6c9da02
    • Chris Egerton's avatar
      KAFKA-13987: Isolate REST request timeout changes in Connect integration tests (#12291) · d00b7875
      Chris Egerton 创作于
      This causes the artificial reductions in the Connect REST request timeout to be more isolated. Specifically, they now only take place in the tests that need them (instead of any tests that happen to be running after the reduction has taken place and before it has been reset), and they are only performed for the requests that are expected to time out, before being immediately reset. This should help reduce spurious test failures (especially in slow environments like Jenkins) for all Connect integration tests that interact with the REST API, not just the BlockingConnectorTest test suite.
      
      Reviewers: Bruno Cadonna <cadonna@apache.org>
      d00b7875
    • Bruno Cadonna's avatar
      MINOR: Fix static mock usage in NamedCacheMetricsTest (#12322) · 8026a0ed
      Bruno Cadonna 创作于
      Before this PR the call to `StreamsMetricsImpl.addAvgAndMinAndMaxToSensor()`
      was just a call and not a verification on the mock. This miss happened
      during the switch from EasyMock to Mockito.
      
      Reviewers: John Roesler <vvcephei@apache.org>, Guozhang Wang <wangguoz@gmail.com>
      8026a0ed
    • Bruno Cadonna's avatar
      MINOR: Fix static mock usage in ProcessorNodeMetricsTest (#12323) · 269277f7
      Bruno Cadonna 创作于
      Before this PR the calls to StreamsMetricsImpl.addInvocationRateAndCountToSensor()
      were just calls and not a verification on the mock. This miss happened
      during the switch from EasyMock to Mockito.
      
      Reviewers: John Roesler <vvcephei@apache.org>, Guozhang Wang <wangguoz@gmail.com>
      269277f7
    • Bruno Cadonna's avatar
      MINOR: Fix static mock usage in StateStoreMetricsTest (#12325) · 6f5843da
      Bruno Cadonna 创作于
      Before this PR the calls to the static methods on
      StreamsMetricsImpl were just calls and not a verification
      on the mock. This miss happened during the switch from
      EasyMock to Mockito.
      
      Reviewers: John Roesler <vvcephei@apache.org>, Guozhang Wang <wangguoz@gmail.com>
      6f5843da
    • Jason Gustafson's avatar
      MINOR: Add ineligible replica reason to log message (#12328) · ead66451
      Jason Gustafson 创作于
      It's useful if the message about ineligible replicas explains the reason the replica is ineligible.
      
      Reviewers: David Jacot <djacot@confluent.io>
      ead66451
  8. 6月 22, 2022
    • David Jacot's avatar
      MINOR: Fix AlterPartitionManager topic id handling in response handler (#12317) · 3072b3d2
      David Jacot 创作于
      https://github.com/apache/kafka/commit/f83d95d9a28267f7ef7a7b1e584dcdb4aa842210 introduced topic ids in the AlterPartitionRequest/Response and we just found a bug in the request handling logic. The issue is the following.
      
      When the `AlterPartitionManager` receives the response, it builds the `partitionResponses` mapping `TopicIdPartition` to its result. `TopicIdPartition` is built from the response. Therefore if version < 2 is used, `TopicIdPartition` will have the `ZERO` topic id. Then the `AlterPartitionManager` iterates over the item sent to find their response. If an item has a topic id in its `TopicIdPartition` and version < 2 was used, it cannot find it because one has it and the other one has not.
      
      This patch fixes the issue by using `TopicPartition` as a key in the `partitionResponses` map. This ensures that the result can be found regardless of the topic id being set or not.
      
      Note that the case where version 2 is used is handled correctly because we already have logic to get back the topic name from the topic id in order to construct the `TopicPartition`.
      
      `testPartialTopicIds` test was supposed to catch this but it didn't due to the ignorable topic id field being present. This patch fixes the test as well.
      
      Reviewers: Kvicii <42023367+Kvicii@users.noreply.github.com>, Jason Gustafson <jason@confluent.io>
      3072b3d2
  9. 6月 21, 2022
  10. 6月 20, 2022
  11. 6月 18, 2022
    • Guozhang Wang's avatar
      KAFKA-13880: Remove DefaultPartitioner from StreamPartitioner (#12304) · cfdd5679
      Guozhang Wang 创作于
      There are some considerata embedded in this seemingly straight-forward PR that I'd like to explain here. The StreamPartitioner is used to send records to three types of topics:
      
      1) repartition topics, where key should never be null.
      2) changelog topics, where key should never be null.
      3) sink topics, where only non-windowed key could be null and windowed key should still never be null.
      Also, the StreamPartitioner is used as part of the IQ to determine which host contains a certain key, as determined by the case 2) above.
      
      This PR's main goal is to remove the deprecated producer's default partitioner, while with those things in mind such that:
      
      We want to make sure for not-null keys, the default murmur2 hash behavior of the streams' partitioner stays consistent with producer's new built-in partitioner.
      For null-keys (which is only possible for non-window default stream partition, and is never used for IQ), we would fix the issue that we may never rotate to a new partitioner by setting the partition as null hence relying on the newly introduced built-in partitioner.
      
      Reviewers: Artem Livshits <84364232+artemlivshits@users.noreply.github.com>, Matthias J. Sax <matthias@confluent.io>
      cfdd5679
  12. 6月 17, 2022
  13. 6月 16, 2022
  14. 6月 15, 2022
    • Ron Dagostino's avatar
      MINOR: Fix force kill of KRaft colocated controllers in system tests (#11238) · b04937dc
      Ron Dagostino 创作于
      I noticed that a system test using a KRaft cluster with 3 brokers but only 1 co-located controller did not force-kill the second and third broker after shutting down the first broker (the one with the controller).  The issue was a floating point rounding error.  This patch adjusts for the rounding error and also makes the logic work for an even number of controllers.  A local run of `tests/kafkatest/sanity_checks/test_bounce.py` succeeded (and I manually increased the cluster size for the 1 co-located controller case and observed the correct kill behavior: the second and third brokers were force-killed as expected).
      
      Reviewers: Luke Chen <showuon@gmail.com>, José Armando García Sancio <jsancio@users.noreply.github.com>, David Jacot <djacot@confluent.io>
      b04937dc
    • Matthias J. Sax's avatar
      MINOR: improve description of `commit.interval.ms` config (#12169) · 44edad5b
      Matthias J. Sax 创作于
      Reviewers: Luke Chen <showuon@gmail.com>, Kvicii Y <@Kvicii>, Bruno Cadonna <bruno@confluent.io>, A. Sophie Blee-Goldman <sophie@confluent.io>
      44edad5b
    • Guozhang Wang's avatar
      KAFKA-13846: Use the new addMetricsIfAbsent API (#12287) · 39a555ba
      Guozhang Wang 创作于
      Use the newly added function to replace the old addMetric function that may throw illegal argument exceptions.
      
      Although in some cases concurrency should not be possible they do not necessarily remain always true in the future, so it's better to use the new API just to be less error-prone.
      
      Reviewers: Bruno Cadonna <cadonna@apache.org>
      39a555ba
    • Ismael Juma's avatar
      MINOR: Remove ReplicaManagerTest.initializeLogAndTopicId (#12276) · f421c008
      Ismael Juma 创作于
      The workaround is not required with mockito.
      
      Reviewers: José Armando García Sancio <jsancio@users.noreply.github.com>, Divij Vaidya <diviv@amazon.com>, Kvicii <42023367+Kvicii@users.noreply.github.com> 
      f421c008
  15. 6月 14, 2022
    • Mickael Maison's avatar
      KAFKA-13958: Expose logdirs total/usable space via Kafka API (KIP-827) (#12248) · 4fcfd9dd
      Mickael Maison 创作于
      This implements KIP-827: https://cwiki.apache.org/confluence/display/KAFKA/KIP-827%3A+Expose+logdirs+total+and+usable+space+via+Kafka+API
      
      Add TotalBytes and UsableBytes to DescribeLogDirsResponse
      Add matching getters on LogDirDescription
      
      Reviewers: Tom Bentley <tbentley@redhat.com>, Divij Vaidya<diviv@amazon.com>, Viktor Somogyi-Vass <viktorsomogyi@gmail.com>, Igor Soarez <soarez@apple.com> 
      4fcfd9dd
    • David Jacot's avatar
      KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft... · f83d95d9
      David Jacot 创作于
      KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841, Part 2) (#12181)
      
      This path implements [KIP-841](https://cwiki.apache.org/confluence/display/KAFKA/KIP-841%3A+Fenced+replicas+should+not+be+allowed+to+join+the+ISR+in+KRaft). Specifically, it implements the following:
      * It introduces INELIGIBLE_REPLICA and NEW_LEADER_ELECTED error codes.
      * The KRaft controller validates the new ISR provided in the AlterPartition request and rejects the call if any replica in the new ISR is not eligible to join the the ISR - e.g. when fenced or shutting down. The leader reverts to the last committed ISR when its request is rejected due to this.
      * The partition leader also verifies that a replica is eligible before trying to add it back to the ISR. If it is not eligible, the ISR expansion is not triggered at all.
      * Updates the AlterPartition API to use topic ids. Updates the AlterPartition manger to handle topic names/ids. Updates the ZK controller and the KRaft controller to handle topic names/ids depending on the version of the request used.
      
      Reviewers: Artem Livshits <84364232+artemlivshits@users.noreply.github.com>, José Armando García Sancio <jsancio@users.noreply.github.com>, Jason Gustafson <jason@confluent.io>
      f83d95d9
    • A. Sophie Blee-Goldman's avatar
      HOTFIX: null check keys of ProducerRecord when computing sizeInBytes (#12288) · 3189a864
      A. Sophie Blee-Goldman 创作于
      Minor followup to #12235 that adds a null check on the record key in the new ClientUtils#producerRecordSizeInBytes utility method, as there are valid cases in which we might be sending records with null keys to the Producer, such as a simple builder.stream("non-keyed-input-topic").filter(...).to("output-topic")
      
      Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>
      3189a864
    • David Arthur's avatar
      KAFKA-13935 Fix static usages of IBP in KRaft mode (#12250) · cc384054
      David Arthur 创作于
      * Set the minimum supported MetadataVersion to 3.0-IV1
      * Remove MetadataVersion.UNINITIALIZED
      * Relocate RPC version mapping for fetch protocols into MetadataVersion
      * Replace static IBP calls with dynamic calls to MetadataCache
      
      A side effect of removing the UNINITIALIZED metadata version is that the FeatureControlManager and FeatureImage will initialize themselves with the minimum KRaft version (3.0-IV1).
      
      The rationale for setting the minimum version to 3.0-IV1 is so that we can avoid any cases of KRaft mode running with an old log message format (KIP-724 was introduced in 3.0-IV1). As a side-effect of increasing this minimum version, the feature level values decreased by one.
      
      Reviewers: Jason Gustafson <jason@confluent.io>, Jun Rao <junrao@gmail.com>
      cc384054
    • vamossagar12's avatar
      KAFKA-13846: Adding overloaded metricOrElseCreate method (#12121) · 5cab11cf
      vamossagar12 创作于
      Reviewers: David Jacot <djacot@confluent.io>, Justine Olshan <jolshan@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
      5cab11cf
加载中