- 12月 01, 2022
-
-
由 Jeff Kim 创作于
-
由 andymg3 创作于
-
由 Ashish Malgawa 创作于
This reverts commit 9c4cb9b3.
-
由 Jeff Kim 创作于
-
由 Sarat Kakarla 创作于
* Adding cells admin command
-
由 Jeff Kim 创作于
-
由 Ashish Malgawa 创作于
* Added describe permission to DataSteward and DataDiscovery and added lineage fix
- 11月 30, 2022
-
-
由 Ning Shan 创作于
* Keep SBC enabled and processing metrics when demoting a broker
-
由 Raman Verma 创作于
This PR adds the following for handling the deletion of FTPS snapshots for topic partitions - Add `lastCommittedSnapshotId` in `TierPartitionState` interface which will be used by MetadataSnapshotTask to avoid deletion of the latest snapshot and implementation of this in `FileTierPartitionState` - Kafka config `confluent.tier.metadata.snapshots.retention.days` indicates the number of days for which a snapshot should be retained. Used in `TierTasks` as part of `SnapshotTasksConfig` and is dynamically configurable to increase/decrease the duration - Add a new state `MetadataSnapshotGarbageCollectionState` in `MetadataSnapshotTask` responsible for garbage collection of FileTierPartitionState snapshots for live partitions. This is called subsequent to `AfterMetadataSnapshotState`. This state once completed transactions to `BeforeMetadataSnapshotState` - Add `garbageCollectAllSnapshots` as part of `CollectDeletableObjects` in `DeletionTask` to handle the deletion of snapshots as part of the cleanup routine for the deletion of topic - Minor rename `CollectDeletableSegments` to `CollectDeletableObjects` as the scope of this method increased to handle deletion of FTPS snapshots as well - Minor helper method `extractEncodedName` in `TierObjectStore` for parsing file naming nomenclature used in the object store Testing - Enhance `MetadataSnapshotTaskTest` for testing deletion logic of live partitions - Below was tested on devel cluster - Deletion logic works i.e., delete only FTPS snapshots when retention time has passed - Changing retention time ms - dynamic config and static via LD flag-based ones reviewers: @junrao @alok123t
-
由 drumiljaswani 创作于
Integrating KC-1786 with Kraft. As part of this change, we turn on Broker health manager for KRaft and start monitoring the storage issues for the broker's running in KRAFT mode. The mitigation enablement for KRaft is behind a separate flag until we finish the test plan for kraft+kc-1786
-
由 Yang Yu 创作于
Refactor KRaft stray partition deletion to use the delayed stray partition deletion mechanism.
-
由 Jason Gustafson 创作于
We moved the metadata encryptor to ImageWriterOptions here: [https://github.com/confluentinc/ce-kafka/commit/c2be380f19639146a1d2e1646e2ea5eb2915ad84 - Connect to preview](https://github.com/confluentinc/ce-kafka/commit/c2be380f19639146a1d2e1646e2ea5eb2915ad84). However, we never updated the logic to make use of it in `BrokerMetadataSnapshotter`. This means that sensitive configurations are not encrypted in snapshots. Additionally, I found that the controller was lacking logic to encrypt sensitive records when generating snapshots (although it does have logic to encrypt these records when they are first written to the log). The patch fixes both issues. Reviewers: Colin Patrick McCabe <cmccabe@apache.org>
-
由 Lingnan Liu 创作于
* MINOR: Perform replica actions before leadership actions in incremental balancing This patch changes the sequence of operations in proposing balancing actions during incremental balancing. Now it will first try replica movement first and it that doesn't work out (for example due to rack being overloaded and no replica movements can address the imbalance), the goal falls back to leadership movement. The reason of doing this change is that we observed replica movement has a much bigger impact on cluster balance than leadership movement.
-
由 Jae Wie 创作于
KCFUN-595: Implements tenant APIs: adminClient.assignTenantsToCell and adminClient.deleteTenants (#8020)
-
由 Jae Wie 创作于
- 11月 29, 2022
-
-
由 David Jacot 创作于
KAFKA-14422; Consumer rebalance stuck after new static member joins a group with members not supporting static members (#12909) (#8160) When a consumer group on a version prior to 2.3 is upgraded to a newer version and static membership is enabled in the meantime, the consumer group remains stuck, iff the leader is still on the old version. The issue is that setting `GroupInstanceId` in the response to the leader is only supported from JoinGroup version >= 5 and that `GroupInstanceId` is not ignorable nor handled anywhere else. Hence is there is at least one static member in the group, sending the JoinGroup response to the leader fails with a serialization error. ``` org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default groupInstanceId at version 2 ``` When this happens, the member stays around until the group coordinator is bounced because a member with a non-null `awaitingJoinCallback` is never expired. This patch fixes the issue by making `GroupInstanceId` ignorable. A unit test has been modified to cover this. Reviewers: Jason Gustafson <jason@confluent.io> (cherry picked from commit be032735)
-
由 David Jacot 创作于
We recently had a bug causing the JoinGroup callback to thrown an exception (https://github.com/apache/kafka/pull/12909). When it happens, the exception is propagated to the caller and the JoinGroup callback is never completed. To make it worst, the member whose callback failed become a zombie because the group coordinator does not expire member with a pending callback. This patch catch exceptions for both invocation of JoinGroup and SyncGroup callbacks and retry to complete them with a `UNKNOWN_SERVER_ERROR` error if they failed. Reviewers: Jason Gustafson <jason@confluent.io> (cherry picked from commit c2c8b246)
-
由 Jason Gustafson 创作于
Builds are consistently bumping against the 2 hour limit and often getting aborted.
-
由 Vikas Singh 创作于
-
由 Stanislav Vodetskyi 创作于
* ignore throttling test for now * put ignore in the correct spot