- 5月 14, 2020
- 5月 13, 2020
-
-
由 Aneel Nazareth 创作于
-
由 Ismael Juma 创作于
Confluent Cloud is running v0.603.27 or newer, so we can remove these classes that had been retained for upgrade compatibility. Also removed upgrade test from 5.3.0 with tiered enabled as suggested by @lbradstreet. Upgrades from that version are not supported. Reviewers: Lucas Bradstreet <lucas@confluent.io>
-
由 Rajini Sivaram 创作于
Reviewers: Manikumar Reddy, Stanislav Kozlovski
-
由 Stanislav Kozlovski 创作于
This patch implements the DescribeBrokerRemovals protocol and Admin Client API, as described in https://confluentinc.atlassian.net/wiki/spaces/CNKAF/pages/1179098256/SBK+Drain+Broker+Debarment+One-Pager The AdminClient now has a `describeBrokerRemovals` method with which it can list all the active broker removal tasks going on in the cluster.
-
由 Ismael Juma 创作于
Update Jenkinsfile to build Scala 2.12 explicitly since the default has changed. The results of `PartitionBenchmark` and `ReplicaFetcherThreadBenchmark` show a 10% to 20% improvement with Scala 2.13. ```text Scala 2.12 Benchmark (partitionCount) Mode Cnt Score Error Units ReplicaFetcherThreadBenchmark.testFetcher 100 avgt 5 7527.180 ± 72.625 ns/op ReplicaFetcherThreadBenchmark.testFetcher 500 avgt 5 50551.081 ± 203.377 ns/op ReplicaFetcherThreadBenchmark.testFetcher 1000 avgt 5 128799.819 ± 1678.992 ns/op ReplicaFetcherThreadBenchmark.testFetcher 5000 avgt 5 751650.483 ± 13876.009 ns/op Benchmark (observerFeatureStr) Mode Cnt Score Error Units PartitionBenchmark.updateFollowerFetchStateBench false avgt 5 184.400 ± 5.351 ns/op PartitionBenchmark.updateFollowerFetchStateBench true avgt 5 182.850 ± 1.851 ns/op ``` ```text Scala 2.13 Benchmark (partitionCount) Mode Cnt Score Error Units ReplicaFetcherThreadBenchmark.testFetcher 100 avgt 5 6961.714 ± 640.678 ns/op ReplicaFetcherThreadBenchmark.testFetcher 500 avgt 5 44138.686 ± 2195.613 ns/op ReplicaFetcherThreadBenchmark.testFetcher 1000 avgt 5 101720.689 ± 4304.548 ns/op ReplicaFetcherThreadBenchmark.testFetcher 5000 avgt 5 613656.113 ± 4218.214 ns/op Benchmark (observerFeatureStr) Mode Cnt Score Error Units PartitionBenchmark.updateFollowerFetchStateBench false avgt 5 159.497 ± 10.196 ns/op PartitionBenchmark.updateFollowerFetchStateBench true avgt 5 165.496 ± 2.775 ns/op ``` Reviewers: Gwen Shapira <gwen@confluent.io>, Lucas Bradstreet <lucas@confluent.io>
-
由 Ismael Juma 创作于
* origin/master: chore: minor version bump v0.952.0-6.0.0-ce-SNAPSHOT [ci skip] MINOR: Remove unused methods in KafkaCruiseControl (#1691) chore: minor version bump v0.951.0-6.0.0-ce-SNAPSHOT [ci skip] KCORE-436: Client interceptor for destination tenant prefixing in cluster link clients (#1667) CIAM-25 Allow CRNs with Paths but no Clusters (#1697) emit deltas as cumulative metrics (#1705) chore: minor version bump v0.950.0-6.0.0-ce-SNAPSHOT [ci skip] CNKAF-649: (part 1/2) Introduce ShutdownManager for SBK broker removal (#1673) MINOR: Fix flaky testTwoConsecutiveShutdownCallExitTwice test (#1709) chore: minor version bump v0.949.0-6.0.0-ce-SNAPSHOT [ci skip] CNKAF-697 (part1): Increase min broker quota constants (#1701) chore: minor version bump v0.948.0-6.0.0-ce-SNAPSHOT [ci skip] KCORE-459: Generate license config docs, use license prefix for timeout config (#1708) chore: minor version bump v0.947.0-6.0.0-ce-SNAPSHOT [ci skip] CONFLUENT: Improve error message when cluster linking is disabled (#1684) chore: minor version bump v0.946.0-6.0.0-ce-SNAPSHOT [ci skip] KSTORAGE-412 Server side handling for the unclean leader flag from LeaderAndIsr re… (#1344)
-
由 Stanislav Kozlovski 创作于
This patch removes a couple of unused methods in KafkaCruiseControl and simplifies the class
-
由 Rajini Sivaram 创作于
Multi-tenant transformations for cluster linking. This includes transformations for the broker interceptor and a new client interceptor that will be used for cluster link clients when the destination is a Cloud cluster. Reviewers: Brian Byrne
-
由 Aneel Nazareth 创作于
-
由 Xavier Léauté 创作于
In line with the changes to our metrics api, delta metrics represent a cumulative value since the last reset (i.e. the "start" time)
-
由 Stanislav Kozlovski 创作于
This patch adds a shutdown manager in SBK which encapsulates the logic for shutting down a broker as part of https://confluentinc.atlassian.net/wiki/spaces/CNKAF/pages/1219931556/SBK+Remove+Broker+Mega-Pager It adds three new configs (internal, no public docs): (confluent.balancer.)default.api.timeout.ms - 1 minute default (confluent.balancer.)describe.cluster.response.timeout.ms - 10 second default (confluent.balancer.)broker.removal.shutdown.timeout.ms - 10 minute default
-
由 Stanislav Kozlovski 创作于
The `testTwoConsecutiveShutdownCallExitTwice` would sometimes fail by seeing the metric equal to 3, rather than 2 as expected. This is likely due to the static variable being incremented as part of a previous test which initiates the same shutdown code, like `InitiateShutdownRequestIntegrationTest`
- 5月 12, 2020
-
-
由 Anna Povzner 创作于
Per-broker quota for a tenant is set to 32KB/s if the broker has not tenant's leaders. During a roll, when a broker restarted, initially it has no leaders and will get 32KB/s quota. When a client start send requests to the new broker after preferred leader change, there maybe a small window when the quota is not updated. This causes large throttle times. This PR changes the constants on a broker to be 10MB/s so it works OK for multi-tenant clusters (there maybe a small window where a tenant may get more quota than it should), and works for enterprise clusters. We will create a follow up PR to fix related broker configs ("confluent.quota.tenant.broker.min.consumer.rate", "confluent.quota.tenant.broker.min.producer.rate") because looks like they are not declared in KafkaConfig and then set those configs in cc-spec-kafka to larger values for enterprise (single-tenant) clusters. Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
-
由 Ismael Juma 创作于
Minor conflicts and some compiler errors in `KafkaApisTest`. Almost every file had import conflicts: * AdminManager.scala: take upstream changes, keep local changes related to observers, cluster linking and the principal parameter to the alter configs methods. * KafkaApis.scala: take upstream changes, keep local changes related to buffer caching for LZ4. * LogTest.scala: take upstream changes, one conflict due to AbstractLog in the signature. * KafkaApisTest.scala: take upstream changes, pass principal to describe/alter config methods in KafkaApis. * confluent/master: KAFKA-9466: Update Kafka Streams docs for KIP-447 (#8621) KAFKA-9949: Fix flaky GlobalKTableIntegrationTest (#8635) KAFKA-9921: explicit handling of null values with retainDuplicates (#8626) KAFKA-9956: Authorizer APIs may be invoked more than once for a given request (#8643) KAFKA-9972: Only commit tasks with valid states (#8632) MINOR: Use `forEach` and `ifPresent` to simplify Scala code (#8642) KAFKA-9942: ConfigCommand fails to set client quotas for default users with --bootstrap-server. (#8628) MINOR: Improve formatting in docs (#8611) KAFKA-9290: Update IQ related JavaDocs (#8114) KAFKA-9928: Fix flaky GlobalKTableEOSIntegrationTest (#8600) KAFKA-6145: Set HighAvailabilityTaskAssignor as default in streams_upgrade_test.py (#8613) KAFKA-9667: Connect JSON serde strip trailing zeros (#8230) MINOR: Log4j Improvements on Fetcher (#8629) KAFKA-9865: Expose output topic names from TopologyTestDriver (#8483)
-
由 Rajini Sivaram 创作于
Reviewers: Manikumar Reddy
-
由 Rajini Sivaram 创作于
Reviewers: Brian Byrne
-
由 Raman Verma 创作于
Server side handling for the unclean leader flag from LeaderAndIsr request - Handles the unclean leader flag in LeaderAndIsrRequest to start a recovery process at the partition layer. - Recovery is mainly geared towards creating a correct view of merged log at the new leader, considering the tiered segments of the log. For non tiered partitions, recovery simply means resetting the unclean leader state at zookeeper. - Recovery also involves verifying that local log segments at the new leader do not diverge from the tiered segments(leader epoch state and producer state) and does not create a hole with respect to tiered portion of log. Local log is truncated in case of divergence or hole. - Till the recovery is completed, we block some of the API requests like OFFSET_FOR_LEADER_EPOCH, DELETE_RECORDS, REPLICA_STATUS, PRODUCE, FETCH, LIST_OFFSETS, TIER_LIST_OFFSET. Rationale behind returning a retriable exception to clients is that we need to ascertain log start and end offsets before opening log for IO or list operations. - While the unclean leader flag is set by the controller as it elects an out of sync replica to be partition leader, it is reset at the end of the recovery process by the unclean leader.
-
-
由 Xavier Léauté 创作于
- collect meter 1-min rate as a gauge - collect "count" values only once for both deltas and total
-
由 Matthias J. Sax 创作于
Reviewers: Boyang Chen <boyang@confluent.io>, Jim Galasyn <jim.galasyn@confluent.io>, Guozhang Wang <guozhang@confluent.io>
-
由 Guozhang Wang 创作于
Reviewer: Matthias J. Sax <matthias@confluent.io>
-
由 A. Sophie Blee-Goldman 创作于
Reviewer: Matthias J. Sax <matthias@confluent.io>
-
由 Ismael Juma 创作于
* Fix describeConfigs and alterConfigs not to invoke authorizer more than once * Add tests to KafkaApisTest to verify the fixes * Rename `filterAuthorized` to `filterByAuthorized` * Tweak `filterByAuthorized` to take resources instead of resource names and improve implementation * Introduce `partitionMapByAuthorized` and `partitionSeqByAuthorized` and simplify code by using it * Replace List with Seq in some AdminManager methods * Remove stray `println` in `KafkaApisTest` Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>