- 10月 04, 2022
-
-
由 Mathew Hogan 创作于
Reviewers: @zhengyd2014
-
由 Vikas Singh 创作于
This change updates the default list of goals in KCCConfig to include the SystemTopicEvenDistributionGoal. This makes the list of goals same as one in CCloud (except in CCloud we remove ReplicaPlacementGoal). Simple change, tests pass.
-
由 Vikas Singh 创作于
We are setting a followers replication bytes in using the leader of the partition (bytes in of leader). However, there are cases where leader is null (cluster is in unhealthy state). In those cases we need to skip those partitions when setting follower's replication bytes in. This test adds a filter to filter these partitions out and also adds couple of tests to validate correct behavior. These tests were failing before but pass now.
-
由 David Mao 创作于
In RCCA-6639, RCCA-6233, RCCA-6651 we saw that bursts of topic creations were taking down the controller broker for reads/writes. A profile taken showed that high time was spent in garbage collection and topic name collision validation (profiles attached in the associated JIRA). A big contributor to this is that we're enumerating all 100k+ topics in the cluster for topic name collision. In particular, for each topic in a request we are retrieving the full set of topics in a cluster which ends up being O(N * M) work where N is the number of topics in the request and M is the total number of topics in the cluster. This PR cuts us over to using the tenant-aware MetadataCache for topic name collision validation so that M is # of tenant topics. This is slightly more prone to race conditions than validating topics through ZK directly in the sense that there is a larger window where concurrent CreateTopic requests can create conflicting topic names, but this seems like a reasonable tradeoff given the performance impact and that the race already existed previously.
-
由 Vikas Singh 创作于
CruiseControlProcessor::setFollowerReplicationBytesIn was setting replication bytes in of follower based on topic byes in at the leader. But there are times when the metrics of the topic at leader node isn't complete (as when topic is empty with no data). In that case the code was throwing IllegalArgumentException. This change adds a check for this condition and ignore setting follower replication byes for this case. Added a unit test that hits this case and then made sure after the fix the test pases.
- 10月 03, 2022
-
-
由 Matthew Wong 创作于
This PR creates a supplier class of OffsetMap objects that can be used for CompactionTasks. The OffsetMapSupplier maintains a pool of OffsetMap objects that can be used across the individual TierLogCleanerMananger threads that are executing CompactionTasks Originally, each CompactionTask would allocate its own OffsetMap object when it began to perform a cleaning pass. However, this is proving to be costly in the amount of resources used for garbage collecting these maps when their respective cleanings are finished.
-
由 Rohit Shekhar 创作于
Co-authored-by: Alok Thatikunta <athatikunta@confluent.io> reviewer @alok123t
- 10月 01, 2022
-
-
由 David Mao 创作于
-
由 Vincent Jiang 创作于
* Add ResetTierPartitionState tool * Store output ftps in a new file * add integration test case testClearAllTierData * fix TierMetadataComparatorTest.testValidationFailsOnOffsetScanEnable * Fence segment in SEGMENT_DELETE_INITIATE state
- 9月 30, 2022
-
-
由 Colin P. McCabe 创作于
-
由 Justine Olshan 创作于
In #7373 we discussed potentially promoting brokers when mitigation was disabled but decided to do as a followup. I took a look at the logic, and if we check if we can transition to promoting, we will target brokers in demoting and demoted states to promote. (Which is what we want.) Any inflight demotions will cease (the issue in #7373) since our demotion status changed. We also won't try to demote again, since we gate that logic on the change of the config (which is done before we check if we can promote.) I adjusted the state expectations from the test added in #7373, but the dynamic code still works as we want We also promote on startup if the broker is demoted and mitigation is off. This covers the case where we disable via LD.
-
由 Ron Dagostino 创作于
- 9月 29, 2022
-
-
由 Ron Dagostino 创作于
-
由 andymg3 创作于
KGLOBAL-2216: Handle SocketException in ClusterLinkConnectionChecker when checking TCP connection (#7593)
-
由 andymg3 创作于
-
由 alysha 创作于
-
由 kpatelatwork 创作于
* KGLOBAL-1578 validate that security protocol of a cloud to cloud link during create/alter should be SSL * added tests for update config * added check for update clusterlink in kraft mode and added missing listenername validate * added end to end validation for security protocol minor reformatting * rewrote to validate in plugin to simplify the code removed whitespace to make the commit clean minor cleanups * resolved minor review comments * minor review comments
-
由 Aishwarya Gune 创作于
* Fix reading goals configs from static configs - originals, add integration test for dynamic updates - adding configs, deleting configs. Testing path include - fetch static config on dynamic deletion and fetching default configs from KCCConfig on dynamic config deletion and no static configs present.
-
由 Akhilesh C 创作于
* KGLOBAL-2218: Eliminate stronger validation on ACL sync enable and filter flags. When ACL sync enable filter is enabled, and filters are not provided, we fail the cluster link. We do not follow the same approach for other background tasks. Removed this extra validation to ensure we don't fail cluster link under such circumstances and just log a warning message about the wrong link configuration.
- 9月 28, 2022
-
-
由 Aneesh Garg 创作于
This adds a new configuration `sasl.server.max.receive.size` that sets the maximum receive size for requests before and during authentication. Reviewers: Tom Bentley <tbentley@redhat.com>, Mickael Maison <mickael.maison@gmail.com> Co-authored-by: Manikumar Reddy <manikumar.reddy@gmail.com> Co-authored-by: Mickael Maison <mickael.maison@gmail.com> Co-authored-by: Manikumar Reddy <manikumar.reddy@gmail.com> Co-authored-by: Mickael Maison <mickael.maison@gmail.com>
-
由 Feng Min 创作于
-
由 Feng Min 创作于
Sync ccs/master into ce-kafka/master up to 08/30/2022 Fairly easy conflict to resolve as most of PR has been cherry-picked. Conflicts to be solved: both modified: core/src/main/scala/kafka/admin/FeatureCommand.scala both modified: core/src/main/scala/kafka/server/ControllerServer.scala both modified: core/src/test/java/kafka/testkit/KafkaClusterTestKit.java both modified: core/src/test/scala/integration/kafka/server/MetadataVersionIntegrationTest.scala both modified: core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala both modified: core/src/test/scala/other/kafka/StressTestLog.scala both modified: core/src/test/scala/other/kafka/TestLinearWriteSpeed.scala both modified: core/src/test/scala/unit/kafka/admin/FeatureCommandTest.scala both modified: core/src/test/scala/unit/kafka/coordinator/group/GroupCoordinatorTest.scala both modified: core/src/test/scala/unit/kafka/coordinator/group/GroupMetadataManagerTest.scala both modified: core/src/test/scala/unit/kafka/log/AbstractLogCleanerIntegrationTest.scala both modified: core/src/test/scala/unit/kafka/log/BrokerCompressionTest.scala both modified: core/src/test/scala/unit/kafka/log/LogCleanerManagerTest.scala both modified: core/src/test/scala/unit/kafka/log/LogCleanerTest.scala both modified: core/src/test/scala/unit/kafka/log/LogConcurrencyTest.scala both modified: core/src/test/scala/unit/kafka/log/LogLoaderTest.scala both modified: core/src/test/scala/unit/kafka/log/LogManagerTest.scala both modified: core/src/test/scala/unit/kafka/log/LogTestUtils.scala deleted by us: core/src/test/scala/unit/kafka/log/UnifiedLogTest.scala both modified: core/src/test/scala/unit/kafka/server/metadata/BrokerMetadataListenerTest.scala both modified: core/src/test/scala/unit/kafka/tools/DumpLogSegmentsTest.scala both modified: core/src/test/scala/unit/kafka/tools/StorageToolTest.scala both modified: core/src/test/scala/unit/kafka/utils/SchedulerTest.scala both modified: core/src/test/scala/unit/kafka/utils/TestUtils.scala both modified: docs/upgrade.html both modified: metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java both modified: metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java both modified: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java both modified: metadata/src/main/java/org/apache/kafka/image/FeaturesImage.java both added: metadata/src/main/java/org/apache/kafka/metadata/bootstrap/BootstrapMetadata.java both modified: metadata/src/test/java/org/apache/kafka/controller/ClusterControlManagerTest.java both modified: metadata/src/test/java/org/apache/kafka/controller/FeatureControlManagerTest.java both modified: metadata/src/test/java/org/apache/kafka/controller/QuorumControllerTest.java both added: metadata/src/test/java/org/apache/kafka/metadata/bootstrap/BootstrapDirectoryTest.java both added: metadata/src/test/java/org/apache/kafka/metadata/bootstrap/BootstrapMetadataTest.java both modified: server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java both modified: server-common/src/test/java/org/apache/kafka/server/common/MetadataVersionTest.java both modified: shell/src/main/java/org/apache/kafka/shell/MetadataNodeManager.java both modified: shell/src/test/java/org/apache/kafka/shell/MetadataNodeManagerTest.java both modified: tests/kafkatest/services/kafka/kafka.py * commit '28d5a059': (13 commits) KAFKA-14187: kafka-features.sh: add support for --metadata (#12571) KAFKA-13036: Replace EasyMock and PowerMock with Mockito for RocksDBMetricsRecorderTest (#12459) MINOR: Add KRaft broker api to protocol docs (#11786) MINOR: Displaying default entity name in MetadataShell (#12053) MINOR: MINOR: Remove redundant error log in ChannelBuilder (#12539) KAFKA-10199: Remove tasks from state updater on shutdown (#12562) KAFKA-10199: Handle restored tasks output by state updater (#12554) MINOR: Small code cleanups in GroupCoordinator (#12563) MINOR: Small javadoc/code cleanups in connect api and transforms (#12558) KAFKA-14183; Cluster metadata bootstrap file should use header/footer (#12565) ...