Skip to content
代码片段 群组 项目
  1. 8月 29, 2022
  2. 8月 25, 2022
  3. 8月 09, 2022
  4. 8月 05, 2022
  5. 7月 30, 2022
  6. 7月 17, 2022
  7. 7月 09, 2022
  8. 7月 08, 2022
  9. 6月 16, 2022
    • Justine Olshan's avatar
      MINOR: Pin ducktape version to < 0.9 (#12242) (#729) · dbfce612
      Justine Olshan 创作于
      
      With newer ducktape versions than < 0.9 system tests
      may run into authentication issues with the AK system test
      infrastructure.
      
      The version will be bumped up once we have infrastructure
      in place for newer paramiko versions brought in by ducktape
      0.9.
      
      Reviewers: Lucas Bradstreet <lucas@confluent.io>, Matthias J. Sax <mjsax@apache.org>, Kvicii <Karonazaba@gmail.com>
      
      Co-authored-by: default avatarBruno Cadonna <cadonna@apache.org>
      dbfce612
  10. 6月 08, 2022
    • Philip Nee's avatar
      HOTFIX: cherry-pick KAFKA-13563 compatibility fix into 3.2 (#724) · 17d73248
      Philip Nee 创作于
      
      * HOTFIX: only try to clear discover-coordinator future upon commit (#12244)
      
      This is another way of fixing KAFKA-13563 other than #11631.
      
      Instead of letting the consumer to always try to discover coordinator in pool with either mode (subscribe / assign), we defer the clearance of discover future upon committing async only. More specifically, under manual assign mode, there are only three places where we need the coordinator:
      
      * commitAsync (both by the consumer itself or triggered by caller), this is where we want to fix.
      * commitSync, which we already try to re-discovery coordinator.
      * committed (both by the consumer itself based on reset policy, or triggered by caller), which we already try to re-discovery coordinator.
      
      The benefits are that for manual assign mode that does not try to trigger any of the above three, then we never would be discovering coordinator. The original fix in #11631 would let the consumer to discover coordinator even if none of the above operations are required.
      
      Reviewers: Luke Chen <showuon@gmail.com>, David Jacot <djacot@confluent.io>
      
      * HOTFIX: add space to avoid checkstyle failure
      
      Co-authored-by: default avatarGuozhang Wang <wangguoz@gmail.com>
      17d73248
  11. 5月 09, 2022
  12. 5月 02, 2022
  13. 4月 28, 2022
    • Ismael Juma's avatar
      KAFKA-13794: Fix comparator of inflightBatchesBySequence in TransactionsManager (round 3) (#12096) · 50d88ab8
      Ismael Juma 创作于
      Conceptually, the ordering is defined by the producer id, producer epoch
      and the sequence number. This set should generally only have entries
      for the same producer id and epoch, but there is one case where
      we can have conflicting `remove` calls and hence we add this as
      a temporary safe fix.
      
      We'll follow-up with a fix that ensures the original intended invariant.
      
      Reviewers: Jason Gustafson <jason@confluent.io>, David Jacot
      <djacot@confluent.io>, Luke Chen <showuon@gmail.com>
      50d88ab8
  14. 4月 27, 2022
  15. 4月 15, 2022
  16. 4月 13, 2022
  17. 4月 12, 2022
    • Yang Yu's avatar
      KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (#11939) · 1181825b
      Yang Yu 创作于
      When a log entry is appended to a Kafka topic using KafkaLog4jAppender, the producer.send operation
      may hit a deadlock if the producer network thread also tries to append a log at the same log level.
      This issue is triggered when idempotence is enabled for the KafkaLog4jAppender and the producer
      tries to acquire the TransactionManager lock.
      
      This is a temporary workaround to avoid deadlocks by disabling idempotence explicitly in
      KafkaLog4jAppender.
      
      Reviewers: Luke Chen <showuon@gmail.com>, Ismael Juma <ismael@juma.me.uk>
      1181825b
  18. 4月 09, 2022
    • Xiaoyue Xue's avatar
      KAFKA-13794: Follow up to fix producer batch comparator (#12006) · 1483a865
      Xiaoyue Xue 创作于
      In comparator, objects that are not equal need to have a stable order otherwise, binary search may not find the objects. Improve the producer batch comparator
      
      Reviewers: Luke Chen <showuon@gmail.com>
      1483a865
    • Alok Nikhil's avatar
      MINOR: Fix DescribeLogDirs API error handling for older API versions (#12017) · 424fbc93
      Alok Nikhil 创作于
      With KAFKA-13527 / KIP-784 we introduced a new top-level error code for
      the DescribeLogDirs API for versions 3 and above. However, the change
      regressed the error handling for versions less than 3 since the response
      converter fails to write the non-zero error code out (rightly) for
      versions lower than 3 and drops the response to the client which
      eventually times out instead of receiving an empty log dirs response and
      processing that as a Cluster Auth failure.
      
      With this change, the API conditionally propagates the error code out to
      the client if the request API version is 3 and above. This keeps the
      semantics of the error handling the same for all versions and restores
      the behavior for older versions.
      
      See current behavior in the broker log:
      ```bash
      ERROR] 2022-04-08 01:22:56,406 [data-plane-kafka-request-handler-10] kafka.server.KafkaApis - [KafkaApi-0] Unexpected error handling request RequestHeader(apiKey=DESCRIBE_LOG_DIRS, apiVersion=0, clientId=sarama, correlationId=1) -- DescribeLogDirsRequestData(topics=null)
      org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default errorCode at version 0
      [ERROR] 2022-04-08 01:22:56,407 [data-plane-kafka-request-handler-10] kafka.server.KafkaRequestHandler - [Kafka Request Handler 10 on Broker 0], Exception when handling request
      org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default errorCode at version 0
      ```
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>
      424fbc93
  19. 4月 08, 2022
  20. 4月 06, 2022
  21. 4月 02, 2022
  22. 3月 31, 2022
    • Bounkong Khamphousone's avatar
      fix: make sliding window works without grace period (#kafka-13739) (#11928) · 90bd03a0
      Bounkong Khamphousone 创作于
      Fix upperbound for sliding window, making it compatible with no grace period (kafka-13739)
      
      Added unit test for early sliding window and "normal" sliding window for both events within one time difference (small input) and above window time difference (large input).
      
      Fixing this window interval may slightly change stream behavior but probability to happen is extremely slow and may not have a huge impact on the result given.
      
      Reviewers Leah Thomas <lthomas@confluent.io>, Bill Bejeck <bbejeck@apache.org>
      90bd03a0
    • Yu's avatar
      KAFKA-13772: Partitions are not correctly re-partitioned when the fetcher... · 0f25205a
      Yu 创作于
      KAFKA-13772: Partitions are not correctly re-partitioned when the fetcher thread pool is resized (#11953)
      
      Partitions are assigned to fetcher threads based on their hash modulo the number of fetcher threads. When we resize the fetcher thread pool, we basically re-distribute all the partitions based on the new fetcher thread pool size. The issue is that the logic that resizes the fetcher thread pool updates the `fetcherThreadMap` while iterating over it. The `Map` does not give any guarantee in this case - especially when the underlying map is re-hashed - and that led to not iterating over all the fetcher threads during the process and thus in leaving some partitions in the wrong fetcher threads.
      
      Reviewers: Luke Chen <showuon@gmail.com>, David Jacot <djacot@confluent.io>
      0f25205a
    • David Jacot's avatar
      KAFKA-13783; Remove reason prefixing in JoinGroupRequest and LeaveGroupRequest (#11971) · ce86a54b
      David Jacot 创作于
      KIP-800 introduced a mechanism to pass a reason in the join group request and in the leave group request. A default reason is used unless one is provided by the user. In this case, the custom reason is prefixed by the default one.
      
      When we tried to used this in Kafka Streams, we noted a significant degradation of the performances, see https://github.com/apache/kafka/pull/11873. It is not clear wether the prefixing is the root cause of the issue or not. To be on the safe side, I think that we should remove the prefixing. It does not bring much anyway as we are still able to distinguish a custom reason from the default one on the broker side.
      
      This patch removes prefixing the user provided reasons. So if a the user provides a reason, the reason is used directly. If the reason is empty or null, the default reason is used.
      
      Reviewers: Luke Chen <showuon@gmail.com>, <jeff.kim@confluent.io>, Hao Li <hli@confluent.io>
      ce86a54b
    • dengziming's avatar
      MINOR: Fix an uncompatible bug in GetOffsetShell (#11936) · 72809cce
      dengziming 创作于
      In KIP-815 we replaced KafkaConsumer with AdminClient in GetOffsetShell. In the previous implementation, partitions were just ignored if there is no offset for them, however, we will print -1 instead now, This PR fix this inconsistency.
      
      Reviewers: David Jacot <djacot@confluent.io>, Luke Chen <showuon@gmail.com>
      72809cce
    • Konstantine Karantasis's avatar
      KAFKA-13748: Do not include file stream connectors in Connect's CLASSPATH and... · 42c80400
      Konstantine Karantasis 创作于
      KAFKA-13748: Do not include file stream connectors in Connect's CLASSPATH and plugin.path by default (#11908)
      
      With this change we stop including the non-production grade connectors that are meant to be used for demos and quick starts by default in the CLASSPATH and plugin.path of Connect deployments. The package of these connector will still be shipped with the Apache Kafka distribution and will be available for explicit inclusion. 
      
      The changes have been tested through the system tests and the existing unit and integration tests. 
      
      Reviewers: Mickael Maison <mickael.maison@gmail.com>, Randall Hauch <rhauch@gmail.com>
      42c80400
    • Mike Lothian's avatar
      KAFKA-13660: Switch log4j12 to reload4j (#11743) · 77574381
      Mike Lothian 创作于
      
      This bumps the slf4j version to 1.7.36 and swaps out log4j 1.2.17 with
      reload4j 1.2.19
      
      Signed-off-by: default avatarMike Lothian <mike@fireburn.co.uk>
      
      Reviewers: Luke Chen <showuon@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Bruno Cadonna <cadonna@apache.org>
      77574381
    • Edwin's avatar
      KAFKA-13775: CVE-2020-36518 - Upgrade jackson-databind to 2.12.6.1 (#11962) · a6e4260d
      Edwin 创作于
      CVE-2020-36518 vulnerability affects jackson-databind (see GHSA-57j2-w4cx-62h2).
      
      Upgrading to jackson-databind version 2.12.6.1 addresses this CVE.
      
      Reviewers: Luke Chen <showuon@gmail.com>, Bruno Cadonna <cadonna@apache.org>
      a6e4260d
    • Bruno Cadonna's avatar
      Upgrade RocksDB from 6.27.3 to 6.29.4.1 (#11967) · 12734b14
      Bruno Cadonna 创作于
      RocksDB 6.27.3 does not run on arm64 M1 Macs which would prevent people on this platform to run Kafka Streams. Thus, this PR upgrades RocksDB to 6.29.4.1 which contains the following fix to allow to run RocksDB on arm64 M1 Macs:
      
      facebook/rocksdb#7720
      
      The source compatibility report between 6.27.3 and 6.29.4.1 (attached to the ticket) reports a couple of incompatibilities. However, the incompatibilities do not seem to affect Kafka Streams' backwards compatibility.
      
          The changes to class RocksDB only apply when inheriting from RocksDB. RocksDB is not exposed to users in Streams.
          The changes to class WriteBatch and class WriteBatchInterface also only apply with inheritance. Both classes are not exposed to users in Streams.
          -The change to enum SanityLevel seem also not to apply to Streams since SanityLevel is only used in ConfigOptions which is only used to load options from files and properties objects. Loading options from files or properties is not exposed to users in Streams.
      
      Reviewers: Bill Bejeck <bbejeck@apache.org>, Matthias J. Sax <mjsax@apache.org>, A. Sophie Blee-Goldman <ableegoldman@apache.org>
      12734b14
    • sunshujie1990's avatar
      KAFKA-13719: Fix connector restart cause duplicate tasks (#11869) · f2920b24
      sunshujie1990 创作于
      
      
      Reviewers: Mickael Maison <mickael.maison@gmail.com>, Luke Chen <showuon@gmail.com>, Chris Egerton <fearthecellos@gmail.com>
      Co-authored-by: default avatarChris Egerton <fearthecellos@gmail.com>
      f2920b24
  23. 3月 30, 2022
    • Ismael Juma's avatar
      KAFKA-13418: Support key updates with TLS 1.3 (#11966) · ecff741c
      Ismael Juma 创作于
      Key updates with TLS 1.3 trigger code paths similar to renegotiation with TLS 1.2.
      Update the read/write paths not to throw an exception in this case (kept the exception
      in the `handshake` method).
      
      With the default configuration, key updates happen after 2^37 bytes are encrypted.
      There is a security property to adjust this configuration, but the change has to be
      done before it is used for the first time and it cannot be changed after that. As such,
      it is best done via a system test (filed KAFKA-13779).
      
      To validate the change, I wrote a unit test that forces key updates and manually ran
      a producer workload that produced more than 2^37 bytes. Both cases failed without
      these changes and pass with them.
      
      Note that Shylaja Kokoori attached a patch with the SslTransportLayer fix and hence
      included them as a co-author of this change.
      
      Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
      
      Co-authored-by: Shylaja Kokoori
      ecff741c
  24. 3月 29, 2022
加载中