Skip to content
代码片段 群组 项目
  1. 12月 20, 2022
  2. 12月 19, 2022
  3. 12月 18, 2022
  4. 12月 17, 2022
    • ConfluentSemaphore's avatar
    • Anastasia Vela's avatar
      KCFUN-602: Implement the basic producer id throttle mechanism (#8109) · 693caf1f
      Anastasia Vela 创作于
      This PR adds the implementation for producer id throttling. The feature
      is gated by a feature flag which defaults to not enabled the throttling.
      Throttling is done in the ProduceRequest pipeline before appending to
      log. There is a check to ensure the producer id count is below the
      throttling threshold. If it's below the threshold, we append to log like
      normally. If it's above the threshold, we send the error response
      REQUEST_TIME_OUT back to the client and throttle them for some time
      period by muting the channel. Once the throttle time is exceeded, the
      messages will be processed again checking that the quota has not been
      exceeded.
      Recording to the count/rate metrics is done in the ProducerStateManager
      to keep track of the expired producer ids as well. So as soon as
      producer ids expire, the count decrements. And when a producer id is
      inserted into the map, the count will increment. This means the
      ProducerIdQuotaManager needed to be ported into the Producer
      StateManager, which is done in this PR.
      This PR also adds the following configs:
      - `confluent.producer.id.throttle.enable` to gate whether we want
      producer id throttling to occur. Defaults to not enabled.
      - `confluent.producer.id.quota.manager.enable` to gate whether the quota
      manager will be initialized or not. Defaults to not enabled.
      
      [JIRA](https://confluentinc.atlassian.net/browse/KCFUN-602)
      
      The following tests were added:
      - MultiTenantQuotaIntegrationTest.java - integration tests to: 1) test
      the pipeline updates the metric accordingly and throttles as expected,
      and 2) test that the rate quota is dynamic
      - DynamicBrokerConfigTest - test that the producer id quota rate is
      dynamic
      - KafkaProducerTest.java - test client's behavior when encountering the
      REQUEST_TIME_OUT exception
      
      ### Committer Checklist (excluded from commit message)
      - [ ] Verify design and implementation 
      - [ ] Verify test coverage and CI build status
      - [ ] Verify documentation (including upgrade notes)
      
      ### Merge requirements
      
      **Branch protections have been put into place which will prevent PRs
      from being merged to master when the build is failing.**
      
      Please review the build in jenkins. If your own change is to blame,
      please fix as required.
      Be careful to not simply retry the build if you suspect your change has
      made tests flakier or added flaky tests.
      
      If you suspect another change is the cause of failures or you're unclear
      about the cause, please read
      
      https://confluentinc.atlassian.net/wiki/spaces/KAFKA/pages/2719875296/ce-kafka+build+stability+unblocking+PR+merges.
      693caf1f
    • ConfluentSemaphore's avatar
    • Confluent Jenkins Bot's avatar
      chore: delete project_onprem.yml · 1749e779
      Confluent Jenkins Bot 创作于
      1749e779
    • ConfluentSemaphore's avatar
    • Zhongyin Zhang's avatar
      [KCFUN-689] Validate Server cets in ConfluentTrustManager (#8162) · 7e7dac66
      Zhongyin Zhang 创作于
      ### About
      The ConfluentTrustManager currently only support client cert validation.
      In order to support inter-broker ssl, we need to add the ability for it
      to validate server certs, and make the behavior configurable so that it
      can be used as needed depending on the use case.
      
      ### Major changes in this PR
      Add server verification in ConfluentTrustManager
      Make Confluent Host Suffix configurable
      
      The [engineer One Page
      ](https://confluentinc.atlassian.net/wiki/spaces/K/pages/2936971376/Using+a+Custom+TrustManager+for+Internal+TLS)
      describes the detail about this change.
      
      [Jira](https://confluentinc.atlassian.net/browse/KCFUN-689)
      
      ### Testing
      
      - The Confluent domain suffix is configurable
      - The Host name validation is disabled under client mode
      - Validate the inter broker ssl handshake
      
      ### Committer Checklist (excluded from commit message)
      - [ ] Verify design and implementation 
      - [ ] Verify test coverage and CI build status
      - [ ] Verify documentation (including upgrade notes)
      
      ### Merge requirements
      
      **Branch protections have been put into place which will prevent PRs
      from being merged to master when the build is failing.**
      
      Please review the build in jenkins. If your own change is to blame,
      please fix as required.
      Be careful to not simply retry the build if you suspect your change has
      made tests flakier or added flaky tests.
      
      If you suspect another change is the cause of failures or you're unclear
      about the cause, please read
      
      https://confluentinc.atlassian.net/wiki/spaces/KAFKA/pages/2719875296/ce-kafka+build+stability+unblocking+PR+merges.
      7e7dac66
    • ConfluentSemaphore's avatar
    • Sanjana Kaundinya's avatar
      KGLOBAL-2442: Disallow cluster link deletion when mirror topics are in... · 0ac040dd
      Sanjana Kaundinya 创作于
      KGLOBAL-2442: Disallow cluster link deletion when mirror topics are in PENDING_STOPPED state (#8264)
      
      0ac040dd
    • ConfluentSemaphore's avatar
    • Kowshik Prakasam's avatar
      KSTORAGE-2577: Dump headers of tier state files using DumpTierPartitionState tool (#8295) · 8e1c4736
      Kowshik Prakasam 创作于
      Modified the `DumpTierPartitionState` to be able to print (to standard
      out) the headers (in JSON format) of all checkpointed tier state files
      under a provided root log directory.
      
      **Test:**
      Built the tool and ran it to test 2 cases:
      
      1. Ran the tool against all tier state files from one of the Kafka
      brokers in tier soak. Tool worked fine, and `jq` command was able to
      prettify its output meaning that the JSON was valid. Also tried
      introducing few errors and the tool still worked fine printing the
      errors correctly to stderr while still printing the JSON output for the
      remaining valid partitions.
      
      2. As a regression test, ran the tool against a single user partition's
      log directory. The tool behaved just like it was prior to this PR and
      printed all contents of the tier state file.
      8e1c4736
    • chern's avatar
      KGLOBAL-1852: Reject cluster link request on Confluent Cloud unauthen… (#7747) · bbc789a1
      chern 创作于
      …ticated listener
      
      Customers can set cluster link bootstrap server to an unauthenticated
      listener of the source cluster. There are network connectivity if the
      source and destination cluster are in the same network. This is bad
      because through cluster link, customers can access another cluster
      without authentication on Confluent Cloud. To mitigate this, we disallow
      bootstrap server that has localhost or site local address + list of
      unauthenticated ports. The downside is we have to update new IP address
      ranges and unauthenticated ports used for Confluent Cloud.
      
      To solve this problem permanently, Confluent Cloud brokers should reject
      cluster linking requests on unauthenticated listener. The code is
      stricter as it only allows SASL_SSL, which is the only option for cloud
      currently. The change introduces ConfluentCloudBrokerInterceptor which
      will be used by unauthenticated listeners on Confluent Cloud.
      
      After destination cluster detecting such scenario, destination cluster
      will fail the cluster link.
      bbc789a1
    • ConfluentSemaphore's avatar
  5. 12月 16, 2022
  6. 12月 15, 2022
加载中