- 4月 02, 2020
- 4月 01, 2020
-
-
-
由 Ismael Juma 创作于
Once Scala 2.13.2 is officially released, I will submit a follow up PR that enables `-Xfatal-warnings` with the necessary warning exclusions. Compiler warning exclusions were only introduced in 2.13.2 and hence why we have to wait for that. I used a snapshot build to test it in the meantime. Changes: * Remove Deprecated annotation from internal request classes * Class.newInstance is deprecated in favor of Class.getConstructor().newInstance * Replace deprecated JavaConversions with CollectionConverters * Remove unused kafka.cluster.Cluster * Don't use Map and Set methods deprecated in 2.13: - collection.Map +, ++, -, --, mapValues, filterKeys, retain - collection.Set +, ++, -, -- * Add scala-collection-compat dependency to streams-scala and update version to 2.1.4. * Replace usages of deprecated Either.get and Either.right * Replace usage of deprecated Integer(String) constructor * `import scala.language.implicitConversions` is not needed in Scala 2.13 * Replace usage of deprecated `toIterator`, `Traversable`, `seq`, `reverseMap`, `hasDefiniteSize` * Replace usage of deprecated alterConfigs with incrementalAlterConfigs where possible * Fix implicit widening conversions from Long/Int to Double/Float * Avoid implicit conversions to String * Eliminate usage of deprecated procedure syntax * Remove `println`in `LogValidatorTest` instead of fixing the compiler warning since tests should not `println`. * Eliminate implicit conversion from Array to Seq * Remove unnecessary usage of 3 argument assertEquals * Replace `toStream` with `iterator` * Do not use deprecated SaslConfigs.DEFAULT_SASL_ENABLED_MECHANISMS * Replace StringBuilder.newBuilder with new StringBuilder * Rename AclBuffers to AclSeqs and remove usage of `filterKeys` * More consistent usage of Set/Map in Controller classes: this also fixes deprecated warnings with Scala 2.13 * Add spotBugs exclusion for inliner artifact in KafkaApis with Scala 2.12. Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
-
由 Lucas Bradstreet 创作于
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
-
由 Matthias J. Sax 创作于
Reviewers: Boyang Chen <boyang@confluent.io>, Guozhang Wang <guozhang@confluent.io>, Bruno Cadonna <bruno@confluent.io>
-
由 Guozhang Wang 创作于
We find that brokers may send empty assignment for some members unexpectedly, and would need more logs investigating this issue. Reviewers: John Roesler <vvcephei@apache.org>
-
由 Guozhang Wang 创作于
Measure the percentage ratio the stream thread spent on processing each task among all assigned active tasks (KIP-444). Also add unit tests to cover the added metrics in this PR and the previous #8358. Also trying to fix the flaky test reported in KAFKA-5842 Co-authored-by: John Roesler <vvcephei@apache.org> Reviewers: Bruno Cadonna <bruno@confluent.io>, John Roesler <vvcephei@apache.org>
-
- 3月 31, 2020
-
-
-
由 David Jacot 创作于
MINOR: Fix MockAdminClient to not throw IndexOutOfBoundsException when brokerId is above the known one. (#8392) Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
-
由 Jason Gustafson 创作于
This patch addresses a locking issue with DelayTxnMarker completion. Because of the reliance on the shared read lock in TransactionStateManager and the deadlock avoidance algorithm in `DelayedOperation`, we cannot guarantee that a call to checkAndComplete will offer an opportunity to complete the job. This patch removes the reliance on this lock in two ways: 1. We replace the transaction marker purgatory with a map of transaction with pending markers. We were not using purgatory expiration anyway, so this avoids the locking issue and simplifies usage. 2. We were also relying on the read lock for the `DelayedProduce` completion when calling `ReplicaManager.appendRecords`. As far as I can tell, this was not necessary. The lock order is always 1) state read/write lock, 2) txn metadata locks. Since we only call `appendRecords` while holding the read lock, a deadlock does not seem possible. Reviewers: Jun Rao <junrao@gmail.com>
-
由 Matthias J. Sax 创作于
Reviewers: Boyang Chen <boyang@confluent.io>, Guozhang Wang <guozhang@confluent.io>
-
由 Greg Harris 创作于
* Fixed DataException thrown when handling tombstone events with null value * Passes through original record when finding a null key when it's configured for keys or a null value when it's configured for values. * Added unit tests for schema and schemaless data
-
由 Boyang Chen 创作于
Reviewer: Matthias J. Sax <matthias@confluent.io>
-
由 Lucas Bradstreet 创作于
#8261 went a long way to solving some of the ACL performance issues. I don't think we need to create sets at all for the `find` and `isEmpty` calls. ` testAuthorizer` is 22% to 62% of the cost after this change: ``` Before: Benchmark (aclCount) (resourceCount) Mode Cnt Score Error Units AclAuthorizerBenchmark.testAclsIterator 5 5000 avgt 15 0.430 ± 0.004 ms/op AclAuthorizerBenchmark.testAclsIterator 5 10000 avgt 15 0.980 ± 0.007 ms/op AclAuthorizerBenchmark.testAclsIterator 5 50000 avgt 15 11.191 ± 0.032 ms/op AclAuthorizerBenchmark.testAclsIterator 10 5000 avgt 15 0.880 ± 0.007 ms/op AclAuthorizerBenchmark.testAclsIterator 10 10000 avgt 15 2.642 ± 0.029 ms/op AclAuthorizerBenchmark.testAclsIterator 10 50000 avgt 15 26.361 ± 0.242 ms/op AclAuthorizerBenchmark.testAclsIterator 15 5000 avgt 15 1.655 ± 0.024 ms/op AclAuthorizerBenchmark.testAclsIterator 15 10000 avgt 15 5.276 ± 0.041 ms/op AclAuthorizerBenchmark.testAclsIterator 15 50000 avgt 15 40.702 ± 0.574 ms/op AclAuthorizerBenchmark.testAuthorizer 5 5000 avgt 15 0.202 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 5 10000 avgt 15 0.233 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 5 50000 avgt 15 0.424 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 10 5000 avgt 15 0.202 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 10 10000 avgt 15 0.253 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 10 50000 avgt 15 0.423 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 15 5000 avgt 15 0.198 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 15 10000 avgt 15 0.242 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 15 50000 avgt 15 0.391 ± 0.002 ms/op JMH benchmarks done After: Benchmark (aclCount) (resourceCount) Mode Cnt Score Error Units AclAuthorizerBenchmark.testAclsIterator 5 5000 avgt 15 0.504 ± 0.164 ms/op AclAuthorizerBenchmark.testAclsIterator 5 10000 avgt 15 1.038 ± 0.271 ms/op AclAuthorizerBenchmark.testAclsIterator 5 50000 avgt 15 11.767 ± 0.028 ms/op AclAuthorizerBenchmark.testAclsIterator 10 5000 avgt 15 0.827 ± 0.016 ms/op AclAuthorizerBenchmark.testAclsIterator 10 10000 avgt 15 2.801 ± 0.027 ms/op AclAuthorizerBenchmark.testAclsIterator 10 50000 avgt 15 26.157 ± 0.191 ms/op AclAuthorizerBenchmark.testAclsIterator 15 5000 avgt 15 1.814 ± 0.053 ms/op AclAuthorizerBenchmark.testAclsIterator 15 10000 avgt 15 5.420 ± 0.065 ms/op AclAuthorizerBenchmark.testAclsIterator 15 50000 avgt 15 41.372 ± 0.659 ms/op AclAuthorizerBenchmark.testAuthorizer 5 5000 avgt 15 0.064 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 5 10000 avgt 15 0.070 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 5 50000 avgt 15 0.240 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 10 5000 avgt 15 0.055 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 10 10000 avgt 15 0.084 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 10 50000 avgt 15 0.249 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 15 5000 avgt 15 0.057 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 15 10000 avgt 15 0.084 ± 0.001 ms/op AclAuthorizerBenchmark.testAuthorizer 15 50000 avgt 15 0.243 ± 0.001 ms/op ``` Reviewers: Ismael Juma <ismael@juma.me.uk>
-
-
由 Bill Bejeck 创作于
These files were missed in the 2.4.1 release Reviewers: Ismael Juma <ismael@confluent.io>
-
- 3月 30, 2020
-
-
由 Chia-Ping Tsai 创作于
The tasks `unitTest` and `integrationTest` used to run tests don't exclude the ```**/*Suite``` so the tests included by Suite class are executed twice. For example: ``` 11:42:25 org.apache.kafka.streams.integration.StoreQuerySuite > org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleToQueryMapValuesState STARTED 11:42:26 11:42:26 org.apache.kafka.streams.integration.GlobalKTableIntegrationTest > shouldKStreamGlobalKTableJoin PASSED 11:42:30 11:42:30 org.apache.kafka.streams.integration.StoreQuerySuite > org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleToQueryMapValuesState PASSED ... 11:48:42 org.apache.kafka.streams.integration.QueryableStateIntegrationTest > shouldBeAbleToQueryMapValuesState STARTED 11:48:46 11:48:46 org.apache.kafka.streams.integration.QueryableStateIntegrationTest > shouldBeAbleToQueryMapValuesState PASSED ``` For consistency, move the existing exclusion for the `test` task from the `streams` project to `subprojects`. Reviewers: Ismael Juma <ismael@juma.me.uk>
-
- 3月 29, 2020
-
-
由 Svend Vanderveken 创作于
MINOR: Fix error message in exception when records have schemas in Connect's Flatten transformation (#3982) In case of an error while flattening a record with schema, the Flatten transformation was reporting an error about a record without schema, as follows: ``` org.apache.kafka.connect.errors.DataException: Flatten transformation does not support ARRAY for record without schemas (for field ...) ``` The expected behaviour would be an error message specifying "with schemas". This looks like a simple copy/paste typo from the schemaless equivalent methods, in the same file Reviewers: Ewen Cheslack-Postava <me@ewencp.org>, Konstantine Karantasis <konstantine@confluent.io>
-
-
由 Scott 创作于
Simple doc fix in a code snippet in connect.html Co-authored-by: Scott Ferguson <smferguson@gmail.com> Reviewers: Ewen Cheslack-Postava <me@ewencp.org>, Konstantine Karantasis <konstantine@confluent.io>
-
由 17hao 创作于
Add missing @Override annotations and use lambdas when declaring threads to suppress warnings in IDEs and improve readability. Reviewers: Ron Dagostino <rdagostino@confluent.io>, Konstantine Karantasis <konstantine@confluent.io>
-
-
由 Boyang Chen 创作于
This PR enhances the epoch checking logic for endTransaction call in TransactionCoordinator. Previously it relaxes the checking by allowing a producer epoch bump, which is error-prone since there is no reason to see a producer epoch bump from client. Reviewers: Jason Gustafson <jason@confluent.io>
-
- 3月 28, 2020
-
-
由 Bruno Cadonna 创作于
When a caching state store is closed it calls its flush() method. If flush() throws an exception the underlying state store is not closed. This commit ensures that state stores underlying a wrapped state stores are closed even when preceding operations in the close method throw. Co-authored-by: John Roesler <vvcephei@apache.org> Reviewers: John Roesler <vvcephei@apache.org>, Guozhang Wang <wangguoz@gmail.com>, Matthias J. Sax <matthias@confluent.io>
-
由 Grace Chen 创作于
* Publish preview jobs with preview release settings * Clean up artifactory publish and include skip signing * Fix skip signing
-
由 Chris Egerton 创作于
For reasons outlined in https://issues.apache.org/jira/browse/KAFKA-9771 we can't upgrade to a version of Jetty with the bug fixed, or downgrade to one prior to the introduction of the bug. Luckily, the actual fix is pretty straightforward and can be ported over to Connect for use until it's possible to upgrade to a version of Jetty with that bug fixed: https://github.com/eclipse/jetty.project/pull/4404/files#diff-58640db0f8f2cd84b7e653d1c1540913R2188-R2193 The changes here have been verified locally; a test with multiple certificates/multiple hostnames will be submitted in a follow up. Reviewers: Jeff Huang <47870461+jeffhuang26@users.noreply.github.com>, Konstantine Karantasis <konstantine@confluent.io>
-
- 3月 27, 2020
-
-
-
由 John Roesler 创作于
* delete topics before tearing down multi-node clusters to avoid leader elections during shutdown * tear down all nodes concurrently instead of sequentially Reviewers: Matthias J. Sax <matthias@confluent.io>
-
-
由 Guozhang Wang 创作于
1. Within a single while loop, process the tasks in AAABBBCCC instead of ABCABCABC. This also helps the follow-up PR to time the per-task processing ratio to record less time, hence less overhead. 2. Add thread-level process / punctuate / poll / commit ratio metrics. 3. Fixed a few issues discovered (inline commented). Reviewers: John Roesler <vvcephei@apache.org>
-
-
由 Stanislav Kozlovski 创作于
1. Defaults state-change log level to INFO. 2. INFO level state-change log includes (a) request level logging with just partition counts; (b) the leader/isr changes per partition in the controller and in the broker (reduced to mostly just 1 logging per partition). Reviewers: Jun Rao <junrao@gmail.com>
-
由 Steve Rodrigues 创作于
An exception is caught and marked 'ignored' even though it's actually used in the catch block. Declare the caught Exception.
-
由 Greg Harris 创作于
* KAFKA-9707: Fix InsertField.Key not applying to tombstone events * Fix typo that hardcoded .value() instead of abstract operatingValue * Add test for Key transform that was previously not tested Signed-off-by: Greg Harris <gregh@confluent.io> * Add null value assertion to tombstone test * Remove mis-named function and add test for passing-through a null-keyed record. Signed-off-by: Greg Harris <gregh@confluent.io> * Simplify unchanged record assertion Signed-off-by: Greg Harris <gregh@confluent.io> * Replace assertEquals with assertSame Signed-off-by: Greg Harris <gregh@confluent.io> * Fix checkstyleTest indent issue Signed-off-by: Greg Harris <gregh@confluent.io>
-
由 David Arthur 创作于
Reviewers: José Armando García Sancio <jsancio@gmail.com>
-
由 Lucas Bradstreet 创作于
A write lock is currently taken out whenever an ACL update is triggered. This update requires a round trip to ZK to get the ACLs for the resource (https://github.com/apache/kafka/pull/7882/files#diff-852b9cb2ceb2b85ec25b422f72c42620R489). This round trip to ZK can block any ACL lookups, which will block any requests and that require authorization, and their corresponding handler threads. This PR attempts to avoid these read locks by snapshotting the aclCache which is a threadsafe scala immutable.TreeMap. Author: Lucas Bradstreet <lucas@confluent.io> Author: Manikumar Reddy <manikumar.reddy@gmail.com> Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk Closes #7882 from lbradstreet/acl-read-lock
-
由 THREE LEVEL HELMET 创作于
Reviewers: Colin P. McCabe <cmccabe@apache.org>
-
由 Boyang Chen 创作于
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
-
- 3月 26, 2020
-
-
由 Rajini Sivaram 创作于
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
-
-