Skip to content
代码片段 群组 项目
  1. 9月 17, 2020
  2. 9月 16, 2020
    • Confluent Jenkins Bot's avatar
    • Bruno Cadonna's avatar
      KAFKA-10292: Set min.insync.replicas to 1 of __consumer_offsets (#9286) · 64b24cef
      Bruno Cadonna 创作于
      The test StreamsBrokerBounceTest.test_all_brokers_bounce() fails on
      2.5 because in the last stage of the test there is only one broker
      left and the offset commit cannot succeed because the
      min.insync.replicas of __consumer_offsets is set to 2 and acks is
      set to all. This causes a time out and extends the closing of the
      Kafka Streams client to beyond the duration passed to the close
      method of the client.
      
      This affects especially the 2.5 branch since there Kafka Streams
      commits offsets for each task, i.e., close() needs to wait for the
      timeout for each task. In 2.6 and trunk the offset commit is done
      per thread, so close() does only need to wait for one time out per
      stream thread.
      
      I opened this PR on trunk, since the test could also become
      flaky on trunk and we want to avoid diverging system tests across
      branches.
      
      A more complete solution would be to improve the test by defining
      a better success criteria.
      
      Reviewers: Guozhang Wang <wangguoz@gmail.com>
      64b24cef
  3. 8月 20, 2020
  4. 8月 19, 2020
  5. 8月 18, 2020
  6. 8月 16, 2020
  7. 8月 13, 2020
  8. 8月 11, 2020
    • Rajini Sivaram's avatar
    • Stanislav Kozlovski's avatar
      MINOR: Ensure a single version of scala-library is used (#9155) · f3257cf7
      Stanislav Kozlovski 创作于
      This patch ensures we use a force resolution strategy for the scala-library dependency.
      
      I've tested this locally and saw a difference in the output.
      
      With the change (using 2.4 and the jackson library 2.10.5):
      ```
      ./core/build/dependant-libs-2.12.10/scala-java8-compat_2.12-0.9.0.jar
      ./core/build/dependant-libs-2.12.10/scala-collection-compat_2.12-2.1.2.jar
      ./core/build/dependant-libs-2.12.10/scala-reflect-2.12.10.jar
      ./core/build/dependant-libs-2.12.10/scala-logging_2.12-3.9.2.jar
      ./core/build/dependant-libs-2.12.10/scala-library-2.12.10.jar
      ```
      
      Without (using 2.4 and the jackson library 2.10.5):
      ```
       find . -name 'scala*.jar'
      ./core/build/dependant-libs-2.12.10/scala-java8-compat_2.12-0.9.0.jar
      ./core/build/dependant-libs-2.12.10/scala-collection-compat_2.12-2.1.2.jar
      ./core/build/dependant-libs-2.12.10/scala-reflect-2.12.10.jar
      ./core/build/dependant-libs-2.12.10/scala-logging_2.12-3.9.2.jar
      ./core/build/dependant-libs-2.12.10/scala-library-2.12.12.jar
      ```
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>
      f3257cf7
  9. 8月 10, 2020
  10. 8月 01, 2020
  11. 7月 29, 2020
  12. 7月 28, 2020
  13. 7月 26, 2020
    • Brian Byrne's avatar
      KAFKA-10158: Fix flaky testDescribeUnderReplicatedPartitionsWhenReassignmentIsInProgress (#9022) · e9b58cf7
      Brian Byrne 创作于
      Set `replica.fetch.max.bytes` to `1` and produce multiple record batches to allow
      for throttling to take place. This helps avoid a race condition where the
      reassignment would complete more quickly than expected causing an
      assertion to fail some times.
      
      Reviewers: Lucas Bradstreet <lucas@confluent.io>, Jason Gustafson <jason@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>
      e9b58cf7
  14. 7月 24, 2020
  15. 7月 23, 2020
  16. 7月 20, 2020
    • Greg Harris's avatar
    • Greg Harris's avatar
      KAFKA-10286: Connect system tests should wait for workers to join group (#9040) · f93056ea
      Greg Harris 创作于
      
      Currently, the system tests `connect_distributed_test` and `connect_rest_test` only wait for the REST api to come up.
      The startup of the worker includes an asynchronous process for joining the worker group and syncing with other workers.
      There are some situations in which this sync takes an unusually long time, and the test continues without all workers up.
      This leads to flakey test failures, as worker joins are not given sufficient time to timeout and retry without waiting explicitly.
      
      This changes the `ConnectDistributedTest` to wait for the Joined group message to be printed to the logs before continuing with tests. I've activated this behavior by default, as it's a superset of the checks that were performed by default before.
      
      This log message is present in every version of DistributedHerder that I could find, in slightly different forms, but always with `Joined group` at the beginning of the log message. This change should be safe to backport to any branch.
      
      Signed-off-by: default avatarGreg Harris <gregh@confluent.io>
      Author: Greg Harris <gregh@confluent.io>
      Reviewer: Randall Hauch <rhauch@gmail.com>
      f93056ea
  17. 7月 18, 2020
  18. 7月 15, 2020
  19. 7月 11, 2020
    • Chia-Ping Tsai's avatar
      KAFKA-10191 fix flaky StreamsOptimizedTest (#8913) · edce73fe
      Chia-Ping Tsai 创作于
      Call KafkaStreams#cleanUp to reset local state before starting application up the second run.
      
      Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Boyang Chen <boyang@confluent.io>, John Roesler <john@confluent.io>
      edce73fe
  20. 7月 10, 2020
    • John Roesler's avatar
      MINOR: prune the metadata upgrade test matrix (#8971) · 8d810bf1
      John Roesler 创作于
      Most of the values in the metadata upgrade test matrix are just testing
      the upgrade/downgrade path between two previous releases. This is
      unnecessary. We run the tests for all supported branches, so what we
      should test is the up-/down-gradability of released versions with respect
      to the current branch.
      
      Reviewers: Guozhang Wang <wangguoz@gmail.com>
      8d810bf1
  21. 7月 09, 2020
    • Guozhang Wang's avatar
      KAFKA-10134: Use long poll if we do not have fetchable partitions (#8934) · 48446042
      Guozhang Wang 创作于
      The intention of using poll(0) is to not block on rebalance but still return some data; however, `updateAssignmentMetadataIfNeeded` have three different logic: 1) discover coordinator if necessary, 2) join-group if necessary, 3) refresh metadata and fetch position if necessary. We only want to make 2) to be non-blocking but not others, since e.g. when the coordinator is down, then heartbeat would expire and cause the consumer to fetch with timeout 0 as well, causing unnecessarily high CPU.
      
      Since splitting this function is a rather big change to make as a last minute blocker fix for 2.6, so I made a smaller change to make updateAssignmentMetadataIfNeeded has an optional boolean flag to indicate if 2) above should wait until either expired or complete, otherwise do not wait on the join-group future and just poll with zero timer.
      
      Reviewers: Jason Gustafson <jason@confluent.io>
      48446042
  22. 7月 08, 2020
    • Bruno Cadonna's avatar
      KAFKA-10221: Backport fix for KAFKA-9603 to 2.5 (#8987) · 17f9f3ae
      Bruno Cadonna 创作于
      KAFKA-9603 reports that the number of open files keeps increasing
      in RocksDB. The reason is that bulk loading is turned on but
      never turned off in segmented state stores for standby tasks.
      
      This bug was fixed in 2.6 through PR #8661 by using code
      that is not present in 2.5. So cherry-picking was not possible.
      
      Reviewers: John Roesler <vvcephei@apache.org>
      17f9f3ae
  23. 7月 07, 2020
    • Boyang Chen's avatar
      KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups (#8989) · c2f26a28
      Boyang Chen 创作于
      This is a bug fix for older admin clients using static membership and call DescribeGroups.
      By making groupInstanceId ignorable, it would not crash upon handling the response.
      
      Added test coverages for DescribeGroups, and some side cleanups.
      
      Reviewers: Jason Gustafson <jason@confluent.io>
      c2f26a28
  24. 7月 02, 2020
    • Ismael Juma's avatar
      MINOR: Update Netty to 4.1.50.Final (#8972) · 39769643
      Ismael Juma 创作于
      This includes important fixes. Netty is required by ZooKeeper if TLS is
      enabled.
      
      I verified that the netty jars were changed from 4.1.48 to 4.1.50 with
      this PR, `find . -name '*netty*'`:
      
      ```text
      ./core/build/dependant-libs-2.13.3/netty-handler-4.1.50.Final.jar
      ./core/build/dependant-libs-2.13.3/netty-transport-native-epoll-4.1.50.Final.jar
      ./core/build/dependant-libs-2.13.3/netty-codec-4.1.50.Final.jar
      ./core/build/dependant-libs-2.13.3/netty-transport-native-unix-common-4.1.50.Final.jar
      ./core/build/dependant-libs-2.13.3/netty-transport-4.1.50.Final.jar
      ./core/build/dependant-libs-2.13.3/netty-resolver-4.1.50.Final.jar
      ./core/build/dependant-libs-2.13.3/netty-buffer-4.1.50.Final.jar
      ./core/build/dependant-libs-2.13.3/netty-common-4.1.50.Final.jar
      ```
      
      Note that the previous netty exclude no longer worked since we upgraded
      to ZooKeeper 3.5.x as it switched to Netty 4 which has different module names.
      Also, the Netty dependency is needed by ZooKeeper for TLS support so we
      cannot exclude it.
      
      Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
      39769643
  25. 6月 30, 2020
    • David Jacot's avatar
      KAFKA-10212: Describing a topic with the TopicCommand fails if unauthorised to... · e643c364
      David Jacot 创作于
      KAFKA-10212: Describing a topic with the TopicCommand fails if unauthorised to use ListPartitionReassignments API
      
      Since https://issues.apache.org/jira/browse/KAFKA-8834, describing topics with the TopicCommand requires privileges to use ListPartitionReassignments or fails to describe the topics with the following error:
      
      > Error while executing topic command : Cluster authorization failed. 
      
      This is a quite hard restriction has most of the secure clusters do not authorize non admin members to access ListPartitionReassignments.
      
      This patch catches the `ClusterAuthorizationException` exception and gracefully fails back. We already do this when the API is not available so it remains consistent.
      
      Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
      e643c364
    • showuon's avatar
      KAFKA-9509: Increase timeout when consuming records to fix flaky test in MM2 (#8894) · 06341cfb
      showuon 创作于
      A simple increase in the timeout of the consumer that verifies that records have been replicated seems to fix the integration tests in `MirrorConnectorsIntegrationTest` that have been failing more often recently. 
      
      Reviewers: Ryanne Dolan <ryannedolan@gmail.com>, Sanjana Kaundinya <skaundinya@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>, Konstantine Karantasis <konstantine@confluent.io>
      06341cfb
    • Andrew Egelhofer's avatar
      Bump CCS version to 5.5.2 · c25955ca
      Andrew Egelhofer 创作于
      c25955ca
  26. 6月 27, 2020
    • John Roesler's avatar
      KAFKA-10173: Fix suppress changelog binary schema compatibility (#8905) · 1d07fb8c
      John Roesler 创作于
      We inadvertently changed the binary schema of the suppress buffer changelog
      in 2.4.0 without bumping the schema version number. As a result, it is impossible
      to upgrade from 2.3.x to 2.4+ if you are using suppression.
      
      * Refactor the schema compatibility test to use serialized data from older versions
      as a more foolproof compatibility test.
      * Refactor the upgrade system test to use the smoke test application so that we
      actually exercise a significant portion of the Streams API during upgrade testing
      * Add more recent versions to the upgrade system test matrix
      * Fix the compatibility bug by bumping the schema version to 3
      
      Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
      1d07fb8c
加载中