Skip to content
代码片段 群组 项目
  1. 10月 07, 2020
  2. 9月 16, 2020
  3. 8月 21, 2020
  4. 8月 19, 2020
  5. 8月 18, 2020
    • Konstantine Karantasis's avatar
      KAFKA-10387: Fix inclusion of transformation configs when topic creation is... · 0b3f1686
      Konstantine Karantasis 创作于
      KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect (#9172)
      
      Addition of configs for custom topic creation with KIP-158 created a regression when transformation configs are also included in the configuration of a source connector. 
      
      To experience the issue, just enabling topic creation at the worker is not sufficient. A user needs to supply a source connector configuration that contains both transformations and custom topic creation properties. 
      
      The issue is that the enrichment of configs in `SourceConnectorConfig` happens on top of an `AbstractConfig` rather than a `ConnectorConfig`. Inheriting from the latter allows enrichment to be composable for both topic creation and transformations. 
      
      Unit tests and integration tests are written to test these combinations. 
      
      Reviewers: Randall Hauch <rhauch@gmail.com>
      0b3f1686
    • Ismael Juma's avatar
      CONFLUENT: Align jersey and jackson versions with `confluentinc/common` for 2.6/6.0.x (#392) · 9595e9a6
      Ismael Juma 创作于
      Update Jackson to 2.10.5 and Jersey to 2.30. Note that the versions in
      master are already aligned.
      
      Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
      9595e9a6
  6. 8月 14, 2020
  7. 8月 13, 2020
  8. 8月 11, 2020
    • Stanislav Kozlovski's avatar
      MINOR: Ensure a single version of scala-library is used (#9155) · 0a8c7b52
      Stanislav Kozlovski 创作于
      This patch ensures we use a force resolution strategy for the scala-library dependency.
      
      I've tested this locally and saw a difference in the output.
      
      With the change (using 2.4 and the jackson library 2.10.5):
      ```
      ./core/build/dependant-libs-2.12.10/scala-java8-compat_2.12-0.9.0.jar
      ./core/build/dependant-libs-2.12.10/scala-collection-compat_2.12-2.1.2.jar
      ./core/build/dependant-libs-2.12.10/scala-reflect-2.12.10.jar
      ./core/build/dependant-libs-2.12.10/scala-logging_2.12-3.9.2.jar
      ./core/build/dependant-libs-2.12.10/scala-library-2.12.10.jar
      ```
      
      Without (using 2.4 and the jackson library 2.10.5):
      ```
       find . -name 'scala*.jar'
      ./core/build/dependant-libs-2.12.10/scala-java8-compat_2.12-0.9.0.jar
      ./core/build/dependant-libs-2.12.10/scala-collection-compat_2.12-2.1.2.jar
      ./core/build/dependant-libs-2.12.10/scala-reflect-2.12.10.jar
      ./core/build/dependant-libs-2.12.10/scala-logging_2.12-3.9.2.jar
      ./core/build/dependant-libs-2.12.10/scala-library-2.12.12.jar
      ```
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>
      0a8c7b52
  9. 8月 10, 2020
  10. 8月 04, 2020
  11. 8月 01, 2020
  12. 7月 29, 2020
  13. 7月 28, 2020
    • Bruno Cadonna's avatar
      KAFKA-10287: Skip unknown offsets when computing sum of changelog offsets (#9066) · e942823c
      Bruno Cadonna 创作于
      In PR #8962 we introduced a sentinel UNKNOWN_OFFSET to mark unknown offsets in checkpoint files. The sentinel was set to -2 which is the same value used for the sentinel LATEST_OFFSET that is used in subscriptions to signal that state stores have been used by an active task. Unfortunately, we missed to skip UNKNOWN_OFFSET when we compute the sum of the changelog offsets.
      
      If a task had only one state store and it did not restore anything before the next rebalance, the stream thread wrote -2 (i.e., UNKNOWN_OFFSET) into the subscription as sum of the changelog offsets. During assignment, the leader interpreted the -2 as if the stream run the task as active although it might have run it as standby. This misinterpretation of the sentinel value resulted in unexpected task assigments.
      
      Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>, John Roesler <vvcephei@apache.org>, Matthias J. Sax <mjsax@apache.org>
      e942823c
    • Bruno Cadonna's avatar
      MINOR: Remove staticmethod tag to be able to use logger of instance (#9086) · e5c9843b
      Bruno Cadonna 创作于
      A system test failed with the following error: global name 'self' is not defined
      
      The reason was that `self` was accessed to log a message in a static method. This commit makes the method an instance method.
      
      Reviewer: Matthias J. Sax <matthias@confluent.io>
      e5c9843b
  14. 7月 27, 2020
  15. 7月 26, 2020
  16. 7月 25, 2020
    • Stanislav Kozlovski's avatar
      KAFKA-10301: Do not clear Partition#remoteReplicasMap during partition assignment updates (#9065) · fb0a79dd
      Stanislav Kozlovski 创作于
      We would previously update the map by adding the new replicas to the map and then removing the old ones.
      During a recent refactoring, we changed the logic to first clear the map and then add all the replicas to it.
      
      While this is done in a write lock, not all callers that access the map structure use a lock. It is safer to revert to
      the previous behavior of showing the intermediate state of the map with extra replicas, rather than an
      intermediate state of the map with no replicas.
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>
      fb0a79dd
    • huxi's avatar
      KAFKA-10268: dynamic config like "--delete-config log.retention.ms" doesn't work (#9051) · 67df232b
      huxi 创作于
      * KAFKA-10268: dynamic config like "--delete-config log.retention.ms" doesn't work
      
      https://issues.apache.org/jira/browse/KAFKA-10268
      
      Currently, ConfigCommand --delete-config API does not restore the config to default value, no matter at broker-level or broker-default level. Besides, Admin.incrementalAlterConfigs API also runs into this problem. This patch fixes it by removing the corresponding config from the newConfig properties when reconfiguring dynamic broker config.
      67df232b
  17. 7月 23, 2020
    • Rajini Sivaram's avatar
      MINOR: Fix deprecation version for NotLeaderForPartitionException (#9056) · 35f399e3
      Rajini Sivaram 创作于
      Reviewers: Ismael Juma <ismael@juma.me.uk>
      35f399e3
    • Jason Gustafson's avatar
      KAFKA-10274; Consistent timeouts in transactions_test (#9026) · 49ce9321
      Jason Gustafson 创作于
      KAFKA-10235 fixed a consistency issue with the transaction timeout and the progress timeout. Since the test case relies on transaction timeouts, we need to wait at last as long as the timeout in order to ensure progress. However, having a low transaction timeout makes the test prone to the issue identified in KAFKA-9802, in which the coordinator timed out the transaction while the producer was awaiting a Produce response.
      
      Reviewers: Chia-Ping Tsai <chia7712@gmail.com>,  Boyang Chen <boyang@confluent.io>, Jun Rao <junrao@gmail.com>
      49ce9321
  18. 7月 22, 2020
  19. 7月 21, 2020
    • Nitesh Mor's avatar
      MINOR: log4j migration to confluent repackaged version (#362) · 9fe1e2cd
      Nitesh Mor 创作于
      Context: log4j v1 has reached end of life many years ago, and is affected by CVE-2019-17571
      Confluent repackaged version of log4j fixes the security vulnerabilities.
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>, Jeff Kim <jeff.kim@confluent.io>
      9fe1e2cd
  20. 7月 20, 2020
  21. 7月 18, 2020
    • Brian Byrne's avatar
      Merge branch `apache/kafka/2.6' into `2.6`. · b56b8783
      Brian Byrne 创作于
      b56b8783
    • elismaga's avatar
    • Rajini Sivaram's avatar
      KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable... · f221a2e5
      Rajini Sivaram 创作于
      KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers (#8979)
      
      Brokers currently return NOT_LEADER_FOR_PARTITION to producers and REPLICA_NOT_AVAILABLE to consumers if a replica is not available on the broker during reassignments. Non-Java clients treat REPLICA_NOT_AVAILABLE as a non-retriable exception, Java consumers handle this error by explicitly matching the error code even though it is not an InvalidMetadataException. This PR renames NOT_LEADER_FOR_PARTITION to NOT_LEADER_OR_FOLLOWER and uses the same error for producers and consumers. This is compatible with both Java and non-Java clients since all clients handle this error code (6) as retriable exception. The PR also makes ReplicaNotAvailableException a subclass of InvalidMetadataException.
          - ALTER_REPLICA_LOG_DIRS continues to return REPLICA_NOT_AVAILABLE. Retained this for compatibility since this request never returned NOT_LEADER_FOR_PARTITION earlier.
         -  MetadataRequest version 0 also returns REPLICA_NOT_AVAILABLE as topic-level error code for compatibility. Newer versions filter these out and return Errors.NONE, so didn't change this.
         - Partition responses in MetadataRequest return REPLICA_NOT_AVAILABLE to indicate that one of the replicas is not available. Did not change this since NOT_LEADER_FOR_PARTITION is not suitable in this case.
      
      Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>, Bob Barrett <bob.barrett@confluent.io>
      f221a2e5
    • Jason Gustafson's avatar
      MINOR: Fix flaky system test assertion after static member fencing (#9033) · 2020262e
      Jason Gustafson 创作于
      The test case `OffsetValidationTest.test_fencing_static_consumer` fails periodically due to this error:
      ```
      Traceback (most recent call last):
        File "/home/jenkins/workspace/system-test-kafka_2.6/kafka/venv/lib/python2.7/site-packages/ducktape-0.7.8-py2.7.egg/ducktape/tests/runner_client.py", line 134, in run
          data = self.run_test()
        File "/home/jenkins/workspace/system-test-kafka_2.6/kafka/venv/lib/python2.7/site-packages/ducktape-0.7.8-py2.7.egg/ducktape/tests/runner_client.py", line 192, in run_test
          return self.test_context.function(self.test)
        File "/home/jenkins/workspace/system-test-kafka_2.6/kafka/venv/lib/python2.7/site-packages/ducktape-0.7.8-py2.7.egg/ducktape/mark/_mark.py", line 429, in wrapper
          return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
        File "/home/jenkins/workspace/system-test-kafka_2.6/kafka/tests/kafkatest/tests/client/consumer_test.py", line 257, in test_fencing_static_consumer
          assert len(consumer.dead_nodes()) == num_conflict_consumers
      AssertionError
      ```
      When a consumer stops, there is some latency between when the shutdown is observed by the service and when the node is added to the dead nodes. This patch fixes the problem by giving some time for the assertion to be satisfied.
      
      Reviewers: Boyang Chen <boyang@confluent.io>
      2020262e
加载中