Skip to content
代码片段 群组 项目
  1. 4月 02, 2020
  2. 4月 01, 2020
    • Confluent Jenkins Bot's avatar
    • Ismael Juma's avatar
      MINOR: Fix Scala 2.13 compiler warnings (#8390) · 90bbeedf
      Ismael Juma 创作于
      Once Scala 2.13.2 is officially released, I will submit a follow up PR
      that enables `-Xfatal-warnings` with the necessary warning
      exclusions. Compiler warning exclusions were only introduced in 2.13.2
      and hence why we have to wait for that. I used a snapshot build to
      test it in the meantime.
      
      Changes:
      * Remove Deprecated annotation from internal request classes
      * Class.newInstance is deprecated in favor of
      Class.getConstructor().newInstance
      * Replace deprecated JavaConversions with CollectionConverters
      * Remove unused kafka.cluster.Cluster
      * Don't use Map and Set methods deprecated in 2.13:
          - collection.Map +, ++, -, --, mapValues, filterKeys, retain
          - collection.Set +, ++, -, --
      * Add scala-collection-compat dependency to streams-scala and
      update version to 2.1.4.
      * Replace usages of deprecated Either.get and Either.right
      * Replace usage of deprecated Integer(String) constructor
      * `import scala.language.implicitConversions` is not needed in Scala 2.13
      * Replace usage of deprecated `toIterator`, `Traversable`, `seq`,
      `reverseMap`, `hasDefiniteSize`
      * Replace usage of deprecated alterConfigs with incrementalAlterConfigs
      where possible
      * Fix implicit widening conversions from Long/Int to Double/Float
      * Avoid implicit conversions to String
      * Eliminate usage of deprecated procedure syntax
      * Remove `println`in `LogValidatorTest` instead of fixing the compiler
      warning since tests should not `println`.
      * Eliminate implicit conversion from Array to Seq
      * Remove unnecessary usage of 3 argument assertEquals
      * Replace `toStream` with `iterator`
      * Do not use deprecated SaslConfigs.DEFAULT_SASL_ENABLED_MECHANISMS
      * Replace StringBuilder.newBuilder with new StringBuilder
      * Rename AclBuffers to AclSeqs and remove usage of `filterKeys`
      * More consistent usage of Set/Map in Controller classes: this also fixes
      deprecated warnings with Scala 2.13
      * Add spotBugs exclusion for inliner artifact in KafkaApis with Scala 2.12.
      
      Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
      90bbeedf
    • Lucas Bradstreet's avatar
      MINOR: reduce garbage in operation and resource java conversions (#8391) · cb912510
      Lucas Bradstreet 创作于
      Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
      cb912510
    • Matthias J. Sax's avatar
      KAFKA-9441: Cleanup Streams metrics for removed task commit latency metrics (#8356) · 6a49ede9
      Matthias J. Sax 创作于
      Reviewers: Boyang Chen <boyang@confluent.io>, Guozhang Wang <guozhang@confluent.io>, Bruno Cadonna <bruno@confluent.io>
      6a49ede9
    • Guozhang Wang's avatar
      MINOR: more logs for empty assignment (#8397) · c1814329
      Guozhang Wang 创作于
      We find that brokers may send empty assignment for some members unexpectedly, and would need more logs investigating this issue.
      
      Reviewers: John Roesler <vvcephei@apache.org>
      c1814329
    • Guozhang Wang's avatar
      KAFKA-9753: Add active tasks process ratio (#8370) · 353aa620
      Guozhang Wang 创作于
      
      Measure the percentage ratio the stream thread spent on processing each task among all assigned active tasks (KIP-444). Also add unit tests to cover the added metrics in this PR and the previous #8358. Also trying to fix the flaky test reported in KAFKA-5842
      
      Co-authored-by: default avatarJohn Roesler <vvcephei@apache.org>
      
      Reviewers: Bruno Cadonna <bruno@confluent.io>, John Roesler <vvcephei@apache.org>
      353aa620
  3. 3月 31, 2020
    • Confluent Jenkins Bot's avatar
    • David Jacot's avatar
      MINOR: Fix MockAdminClient to not throw IndexOutOfBoundsException when... · 121c7692
      David Jacot 创作于
      MINOR: Fix MockAdminClient to not throw IndexOutOfBoundsException when brokerId is above the known one. (#8392)
      
      Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
      121c7692
    • Jason Gustafson's avatar
      KAFKA-9777; Remove txn purgatory to fix race condition on txn completion (#8389) · 75e8ee1d
      Jason Gustafson 创作于
      This patch addresses a locking issue with DelayTxnMarker completion. Because of the reliance on the shared read lock in TransactionStateManager and the deadlock avoidance algorithm in `DelayedOperation`, we cannot guarantee that a call to checkAndComplete will offer an opportunity to complete the job. This patch removes the reliance on this lock in two ways:
      
      1. We replace the transaction marker purgatory with a map of transaction with pending markers. We were not using purgatory expiration anyway, so this avoids the locking issue and simplifies usage.
      2. We were also relying on the read lock for the `DelayedProduce` completion when calling `ReplicaManager.appendRecords`. As far as I can tell, this was not necessary. The lock order is always 1) state read/write lock, 2) txn metadata locks. Since we only call `appendRecords` while holding the read lock, a deadlock does not seem possible. 
      
      Reviewers: Jun Rao <junrao@gmail.com>
      75e8ee1d
    • Matthias J. Sax's avatar
      KAFKA-9719: Streams with EOS-beta should fail fast for older brokers (#8367) · 6ad54073
      Matthias J. Sax 创作于
      Reviewers: Boyang Chen <boyang@confluent.io>, Guozhang Wang <guozhang@confluent.io>
      6ad54073
    • Greg Harris's avatar
      KAFKA-9706: Handle null in keys or values when Flatten transformation is used (#8279) · 2c969b14
      Greg Harris 创作于
      * Fixed DataException thrown when handling tombstone events with null value
      * Passes through original record when finding a null key when it's configured for keys or a null value when it's configured for values. 
      * Added unit tests for schema and schemaless data
      2c969b14
    • Boyang Chen's avatar
      MINOR: Update docs for KIP-530 and KIP-562 (#8388) · 0520bc6c
      Boyang Chen 创作于
      Reviewer: Matthias J. Sax <matthias@confluent.io>
      0520bc6c
    • Lucas Bradstreet's avatar
      KAFKA-9685: PT2, avoid unnecessary set creation in ACL matching (#8382) · 4fa5bdcf
      Lucas Bradstreet 创作于
      #8261 went a long way to solving some of the ACL performance issues. I don't think
      we need to create sets at all for the `find` and `isEmpty` calls. ` testAuthorizer` is
      22% to 62% of the cost after this change:
      
      ```
      Before:
      Benchmark                                (aclCount)  (resourceCount)  Mode  Cnt   Score    Error  Units
      AclAuthorizerBenchmark.testAclsIterator           5             5000  avgt   15   0.430 ±  0.004  ms/op
      AclAuthorizerBenchmark.testAclsIterator           5            10000  avgt   15   0.980 ±  0.007  ms/op
      AclAuthorizerBenchmark.testAclsIterator           5            50000  avgt   15  11.191 ±  0.032  ms/op
      AclAuthorizerBenchmark.testAclsIterator          10             5000  avgt   15   0.880 ±  0.007  ms/op
      AclAuthorizerBenchmark.testAclsIterator          10            10000  avgt   15   2.642 ±  0.029  ms/op
      AclAuthorizerBenchmark.testAclsIterator          10            50000  avgt   15  26.361 ±  0.242  ms/op
      AclAuthorizerBenchmark.testAclsIterator          15             5000  avgt   15   1.655 ±  0.024  ms/op
      AclAuthorizerBenchmark.testAclsIterator          15            10000  avgt   15   5.276 ±  0.041  ms/op
      AclAuthorizerBenchmark.testAclsIterator          15            50000  avgt   15  40.702 ±  0.574  ms/op
      AclAuthorizerBenchmark.testAuthorizer             5             5000  avgt   15   0.202 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer             5            10000  avgt   15   0.233 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer             5            50000  avgt   15   0.424 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            10             5000  avgt   15   0.202 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            10            10000  avgt   15   0.253 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            10            50000  avgt   15   0.423 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            15             5000  avgt   15   0.198 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            15            10000  avgt   15   0.242 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            15            50000  avgt   15   0.391 ±  0.002  ms/op
      JMH benchmarks done
      
      After:
      Benchmark                                (aclCount)  (resourceCount)  Mode  Cnt   Score    Error  Units
      AclAuthorizerBenchmark.testAclsIterator           5             5000  avgt   15   0.504 ±  0.164  ms/op
      AclAuthorizerBenchmark.testAclsIterator           5            10000  avgt   15   1.038 ±  0.271  ms/op
      AclAuthorizerBenchmark.testAclsIterator           5            50000  avgt   15  11.767 ±  0.028  ms/op
      AclAuthorizerBenchmark.testAclsIterator          10             5000  avgt   15   0.827 ±  0.016  ms/op
      AclAuthorizerBenchmark.testAclsIterator          10            10000  avgt   15   2.801 ±  0.027  ms/op
      AclAuthorizerBenchmark.testAclsIterator          10            50000  avgt   15  26.157 ±  0.191  ms/op
      AclAuthorizerBenchmark.testAclsIterator          15             5000  avgt   15   1.814 ±  0.053  ms/op
      AclAuthorizerBenchmark.testAclsIterator          15            10000  avgt   15   5.420 ±  0.065  ms/op
      AclAuthorizerBenchmark.testAclsIterator          15            50000  avgt   15  41.372 ±  0.659  ms/op
      AclAuthorizerBenchmark.testAuthorizer             5             5000  avgt   15   0.064 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer             5            10000  avgt   15   0.070 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer             5            50000  avgt   15   0.240 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            10             5000  avgt   15   0.055 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            10            10000  avgt   15   0.084 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            10            50000  avgt   15   0.249 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            15             5000  avgt   15   0.057 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            15            10000  avgt   15   0.084 ±  0.001  ms/op
      AclAuthorizerBenchmark.testAuthorizer            15            50000  avgt   15   0.243 ±  0.001  ms/op
      ```
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>
      4fa5bdcf
    • Confluent Jenkins Bot's avatar
    • Bill Bejeck's avatar
      MINOR: Update dependencies.gradle, Dockerfile, version.py, and bash.sh for 2.4.1 upgrade (#8387) · c725c233
      Bill Bejeck 创作于
      These files were missed in the 2.4.1 release
      
      Reviewers: Ismael Juma <ismael@confluent.io>
      c725c233
  4. 3月 30, 2020
    • Chia-Ping Tsai's avatar
      MINOR: Exclude '**/*Suite.class' from test, unitTest and integrationTest (#8381) · 923f1ec1
      Chia-Ping Tsai 创作于
      The tasks `unitTest` and `integrationTest` used to run tests don't exclude
      the ```**/*Suite``` so the tests included by Suite class are executed twice.
      For example:
      
      ```
      11:42:25 org.apache.kafka.streams.integration.StoreQuerySuite > org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleToQueryMapValuesState STARTED
      11:42:26 
      11:42:26 org.apache.kafka.streams.integration.GlobalKTableIntegrationTest > shouldKStreamGlobalKTableJoin PASSED
      11:42:30 
      11:42:30 org.apache.kafka.streams.integration.StoreQuerySuite > org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleToQueryMapValuesState PASSED
      ...
      11:48:42 org.apache.kafka.streams.integration.QueryableStateIntegrationTest > shouldBeAbleToQueryMapValuesState STARTED
      11:48:46 
      11:48:46 org.apache.kafka.streams.integration.QueryableStateIntegrationTest > shouldBeAbleToQueryMapValuesState PASSED
      ```
      
      For consistency, move the existing exclusion for the `test` task from the
      `streams` project to `subprojects`.
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>
      923f1ec1
  5. 3月 29, 2020
  6. 3月 28, 2020
  7. 3月 27, 2020
  8. 3月 26, 2020
加载中