Skip to content
代码片段 群组 项目
  1. 8月 11, 2022
  2. 8月 06, 2022
    • Ismael Juma's avatar
      CONFLUENT: Sync from apache/kafka trunk to confluentinc/kafka master (5 August 2022) · 97f97b00
      Ismael Juma 创作于
      Version related conflicts:
      * Jenkinsfile
      * gradle.properties
      * streams/quickstart/java/pom.xml
      * streams/quickstart/java/src/main/resources/archetype-resources/pom.xml
      * streams/quickstart/pom.xml
      * tests/kafkatest/__init__.py
      * tests/kafkatest/version.py
      
      * commit 'add7cd85': (66 commits)
      KAFKA-14136 Generate ConfigRecord for brokers even if the value is
      unchanged (#12483)
        HOTFIX / KAFKA-14130: Reduce RackAwarenesssTest to unit Test (#12476)
        MINOR: Remove ARM/PowerPC builds from Jenkinsfile (#12380)
        KAFKA-14111 Fix sensitive dynamic broker configs in KRaft (#12455)
        KAFKA-13877: Fix flakiness in RackAwarenessIntegrationTest (#12468)
      KAFKA-14129: KRaft must check manual assignments for createTopics are
      contiguous (#12467)
      KAFKA-13546: Do not fail connector validation if default topic
      creation group is explicitly specified (#11615)
      KAFKA-14122: Fix flaky test
      DynamicBrokerReconfigurationTest#testKeyStoreAlter (#12452)
        MINOR; Use right enum value for broker registration change (#12236)
        MINOR; Synchronize access to snapshots' TreeMap (#12464)
        MINOR; Bump trunk to 3.4.0-SNAPSHOT (#12463)
        MINOR: Stop logging 404s at ERROR level in Connect
      KAFKA-14095: Improve handling of sync offset failures in MirrorMaker
      (#12432)
        Minor: enable index for emit final sliding window (#12461)
        MINOR: convert some more junit tests to support KRaft (#12456)
        KAFKA-14108: Ensure both JUnit 4 and JUnit 5 tests run (#12441)
        MINOR: Remove code of removed metric (#12453)
      MINOR: Update comment on verifyTaskGenerationAndOwnership method in
      DistributedHerder
      KAFKA-14012: Add warning to closeQuietly documentation about method
      references of null objects (#12321)
        MINOR: Fix static mock usage in ThreadMetricsTest (#12454)
        ...
      97f97b00
  3. 8月 05, 2022
  4. 8月 04, 2022
    • Guozhang Wang's avatar
      HOTFIX / KAFKA-14130: Reduce RackAwarenesssTest to unit Test (#12476) · 5ceaa588
      Guozhang Wang 创作于
      While working on KAFKA-13877, I feel it's an overkill to introduce the whole test class as an integration test, since all we need is to just test the assignor itself which could be a unit test. Running this suite with 9+ instances takes long time and is still vulnerable to all kinds of timing based flakiness. A better choice is to reduce it as a unit test, similar to HighAvailabilityStreamsPartitionAssignorTest that just test the behavior of the assignor itself, rather than creating many instances hence depend on various timing bombs to not explode.
      
      Since we mock everything, there's no flakiness anymore. Plus we greatly reduced the test runtime (on my local machine, the old integration takes about 35 secs to run the whole suite, while the new one take 20ms on average).
      
      Reviewers: Divij Vaidya <diviv@amazon.com>, Dalibor Plavcic
      5ceaa588
    • Jason Gustafson's avatar
      MINOR: Remove ARM/PowerPC builds from Jenkinsfile (#12380) · 64c72c2e
      Jason Gustafson 创作于
      Reviewers: Colin P. McCabe <cmccabe@apache.org>, Ismael Juma <ismael@juma.me.uk>
      64c72c2e
    • Ismael Juma's avatar
      Merge up to the Apache Kafka 3.3 branching point · affe8cd0
      Ismael Juma 创作于
      > $ git merge-base apache-github/3.3 apache-github/trunk
      > 23c92ce7
      
      > $ git show 23c92ce7
      > commit 23c92ce7
      > Author: SC <pch838811@gmail.com>
      > Date:   Mon Jul 11 11:36:56 2022 +0900
      >
      >    MINOR: Use String#format for niceMemoryUnits result (#12389)
      >
      >    Reviewers: Luke Chen <showuon@gmail.com>, Divij Vaidya <diviv@amazon.com>
      
      * commit '23c92ce7':
        MINOR: Use String#format for niceMemoryUnits result (#12389)
        KAFKA-14055; Txn markers should not be removed by matching records in the offset map (#12390)
        KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection (#12381)
        KAFKA-13996: log.cleaner.io.max.bytes.per.second can be changed dynamically (#12296)
        KAFKA-13983: Fail the creation with "/" in resource name in zk ACL (#12359)
        KAFKA-12943: update aggregating documentation (#12091)
        KAFKA-13846: Follow up PR to address review comments (#12297)
      affe8cd0
    • Aneesh Garg's avatar
      ba219265
    • David Arthur's avatar
      KAFKA-14111 Fix sensitive dynamic broker configs in KRaft (#12455) · bfd9e6ec
      David Arthur 创作于
      Enable some of the dynamic broker reconfiguration tests in KRaft mode
      bfd9e6ec
    • Guozhang Wang's avatar
      KAFKA-13877: Fix flakiness in RackAwarenessIntegrationTest (#12468) · 32024593
      Guozhang Wang 创作于
      In the current test, we check for tag distribution immediately after everyone is on the running state, however due to the fact of the follow-up rebalances, "everyone is now in running state" does not mean that the cluster is now stable. In fact, a follow-up rebalance may occur, upon which the local thread metadata would return empty which would cause the distribution verifier to fail.
      
      Reviewers: Divij Vaidya <diviv@amazon.com>, Luke Chen <showuon@gmail.com>
      32024593
  5. 8月 03, 2022
    • Colin Patrick McCabe's avatar
      KAFKA-14129: KRaft must check manual assignments for createTopics are contiguous (#12467) · 0c4da230
      Colin Patrick McCabe 创作于
      KRaft should validate that manual assignments given to createTopics are contiguous. In other words,
      they must start with partition 0, and progress through 1, 2, 3, etc. ZK mode does this, but KRaft
      mode previously did not. Also fix a null pointer exception when the placement for partition 0
      was not specified.
      
      Convert over AddPartitionsTest to use KRaft. This PR converts all of the test except for some of
      the placement logic tests, which will need to be redone for KRaft mode in a future change.
      
      Fix null pointer exception in KRaftMetadataCache#getPartitionInfo.  Specifically, we should not
      assume that the partition will be found in the hash map. This is another case where we had
      "Some(x)" but it should be "Option(x)."
      
      Fix a potential null pointer exception in BrokerServer#state.
      
      Reviewers: dengziming <dengziming1993@gmail.com>, Jason Gustafson <jason@confluent.io>
      0c4da230
    • venkatteki's avatar
      KAFKA-13546: Do not fail connector validation if default topic creation group... · a02c8d33
      venkatteki 创作于
      KAFKA-13546: Do not fail connector validation if default topic creation group is explicitly specified (#11615)
      
      Reviewers: Chris Egerton <fearthecellos@gmail.com>
      a02c8d33
    • Divij Vaidya's avatar
      KAFKA-14122: Fix flaky test DynamicBrokerReconfigurationTest#testKeyStoreAlter (#12452) · 78038bca
      Divij Vaidya 创作于
      Reviewers: Mickael Maison <mickael.maison@gmail.com>
      78038bca
  6. 8月 02, 2022
  7. 8月 01, 2022
  8. 7月 30, 2022
    • Hao Li's avatar
      Minor: enable index for emit final sliding window (#12461) · f7ac5d3d
      Hao Li 创作于
      Enable index for sliding window emit final case as it's faster to fetch windows for particular key
      
      Reviewers: Guozhang Wang <wangguoz@gmail.com>
      f7ac5d3d
    • Colin Patrick McCabe's avatar
      MINOR: convert some more junit tests to support KRaft (#12456) · e26772ef
      Colin Patrick McCabe 创作于
      * MINOR: convert some more junit tests to support KRaft
      
      Introduce TestUtils#waitUntilLeaderIsElectedOrChangedWithAdmin, a ZK-free alternative to
      TestUtils#waitUntilLeaderIsElectedOrChanged.
      
      Convert PlaintextProducerSendTest, SslProducerSendTest, TransactionsWithMaxInFlightOneTest,
      AddPartitionsToTxnRequestServerTest and KafkaMetricsReporterTest to support KRaft
      
      Reviewers: dengziming <dengziming1993@gmail.com>, David Arthur <mumrah@gmail.com>
      e26772ef
  9. 7月 29, 2022
  10. 7月 28, 2022
    • Chris Egerton's avatar
      KAFKA-14089: Only check for committed seqnos after disabling exactly-once... · 9e74f91e
      Chris Egerton 创作于
      KAFKA-14089: Only check for committed seqnos after disabling exactly-once support in Connect integration test (#12429)
      
      
      Reviewers: Mickael Maison <mickael.maison@gmail.com>

, Tom Bentley <tbentley@redhat.com>
      9e74f91e
    • vamossagar12's avatar
      KAFKA-14007: Close header converters during Connect task shutdown (#12309) · 0c5f5a7f
      vamossagar12 创作于
      The HeaderConverter interface extends Closeable, but we weren't closing them anywhere before. This change causes header converters to be closed as part of task shutdown.
      
      Reviewers: Kvicii <42023367+Kvicii@users.noreply.github.com>, Chris Egerton <fearthecellos@gmail.com>
      0c5f5a7f
    • Guozhang Wang's avatar
      KAFKA-10199: Further refactor task lifecycle management (#12439) · 06f47c3b
      Guozhang Wang 创作于
      1. Consolidate the task recycle procedure into a single function within the task. The current procedure now becomes: a) task.recycleStateAndConvert, at end of it the task is in closed while its stateManager is retained, and the manager type has been converted; 2) create the new task with old task's fields and the stateManager inside the creators.
      2. Move the task execution related metadata into the corresponding TaskExecutionMetadata class, including the task idle related metadata (e.g. successfully processed tasks); reduce the number of params needed for TaskExecutor as well as Tasks.
      3. Move the task execution related fields (embedded producer and consumer) and task creators out of Tasks and migrated into TaskManager. Now the Tasks is only a bookkeeping place without any task mutation logic.
      4. When adding tests, I realized that we should not add task to state updater right after creation, since it was not initialized yet, while state updater would validate that the task's state is already restoring / running. So I updated that logic while adding unit tests.
      
      Reviewers: Bruno Cadonna <cadonna@apache.org>
      06f47c3b
    • Alex Sorokoumov's avatar
      KAFKA-13769: Add tests for ForeignJoinSubscriptionProcessorSupplier (#12437) · d076b7ad
      Alex Sorokoumov 创作于
      Reviewers: Adam Bellemare <adam.bellemare@gmail.com>, John Roesler <vvcephei@apache.org>
      d076b7ad
    • Colin Patrick McCabe's avatar
      MINOR: Convert some junit tests to kraft (#12443) · 1914418c
      Colin Patrick McCabe 创作于
      Convert ProducerCompressionTest, MirrorMakerIntegrationTest, EdgeCaseRequestTest to kraft.
      
      Make it explicit that ServerShutdownTest#testControllerShutdownDuringSend is ZK-only.
      
      Reviewers: David Arthur <mumrah@gmail.com>
      1914418c
  11. 7月 27, 2022
  12. 7月 26, 2022
加载中