Skip to content
代码片段 群组 项目
  1. 11月 10, 2022
  2. 11月 09, 2022
    • ConfluentSemaphore's avatar
    • Xavier Léauté's avatar
      METRICS-4145 Migrate TelemetryReporter to OpenTelemetry (#5931) · f437c9db
      Xavier Léauté 创作于
      This change migrates telemetry reporter to send data in OpenTelemetry format instead of OpenCensus format.
      
      - refactors SinglePointMetric methods to accept generic Number classes to reduce code duplication
      - rename SinglePointMetric methods sum / counter to sum / deltaSum, to reflect the OpenTelemetry metric naming   
      - converts the OpenCensus specific "resource type" field into a generic OpenTelemetry "type" resource attribute
      - filters out the resource `type` attribute from events to prevent it from clashing with the existing "event type" field.
      - simplifies `Instant.clock(timestamp)` to `clock.timestamp()`
      - remove unnecessary references to `this.` to keep the code style consistent
      
      This is dependent on downstream consumers (Druid, Connect, SBC, Telemetry Receiver) to support both formats
      for the duration of the upgrade from the old version to the new version.
      
      Dependencies:
      - [x] update SBC telemetry sampler to support opentelemetry https://github.com/confluentinc/ce-kafka/pull/7678
      - [x] define upgrade path for SBC – SBC will ignore newer format metrics, and pause balancing during roll.
      - [x] Druid support https://github.com/confluentinc/druid/pull/63
      - [x] Connect support https://github.com/confluentinc/opencensus-protobuf-converter/pull/10
      - [x] Telemetry Receiver support for OpenTelemetry https://github.com/confluentinc/schroedinger/pull/1230
      f437c9db
    • ConfluentSemaphore's avatar
    • Panagiotis Garefalakis's avatar
    • Colin Patrick McCabe's avatar
      MINOR; Introduce ImageWriter and ImageWriterOptions (#12715) · c2be380f
      Colin Patrick McCabe 创作于
      This PR adds a new ImageWriter interface which replaces the generic Consumer interface which
      accepted lists of records. It is better to do batching in the ImageWriter than to try to deal with
      that complexity in the MetadataImage#write functions, especially since batching is not semantically
      meaningful in KRaft snapshots. The new ImageWriter interface also supports freeze and close, which
      more closely matches the semantics of the underlying Raft classes.
      
      The PR also adds an ImageWriterOptions class which we can use to pass parameters to control how the
      new image is written. Right now, the parameters that we are interested in are the target metadata
      version (which may be more or less than the original image's version) and a handler function which
      is invoked whenever metadata is lost due to the target version.
      
      Convert over the MetadataImage#write function (and associated functions) to use the new ImageWriter
      and ImageWriterOptions. In particular, we now have a way to handle metadata losses by invoking
      ImageWriterOptions#handleLoss. This allows us to handle writing an image at a lower version, for
      the first time. This support is still not enabled externally by this PR, though. That will come in
      a future PR.
      
      Get rid of the use of SOME_RECORD_TYPE.highestSupportedVersion() in several places. In general, we
      do not want to "silently" change the version of a record that we output, just because a new version
      was added. We should be explicit about what record version numbers we are outputting.
      
      Implement ProducerIdsDelta#toString, to make debug logs look better.
      
      Move MockRandom to the server-common package so that other internal broker packages can use it.
      
      Reviewers: José Armando García Sancio <jsancio@apache.org>
      Conflicts: Added ImageWriter / ImageWriterOptions to Confluent-specific metadata image classes.
          Handled conflicts related to metadata encryption existing in ce-kafka but not AK.
          Fix a bug in BrokerMetadataSnapshotter and BrokerMetadataListener where we were not calling
          close(true) on the image writers.
      c2be380f
    • ConfluentSemaphore's avatar
    • Yiran's avatar
    • ConfluentSemaphore's avatar
    • Michael Li's avatar
      [METRICS-4647] Deduplicate RemoteConfiguration POJOs (#7786) · 2b25cfc1
      Michael Li 创作于
      In order to meet the code freeze for CP 7.3, we duplicated a few classes in both ce-kafka and telemetry-api. Now that we have some time, we need to consolidate these classes so that we reduce tech debt and only make future changes in one class. The following classes have been moved from ce-kafka to telemetry-api:
      
      NamedFilter
      RemoteConfiguration
      RemoteConfigurationResponse
      RemoteConfigurationRequest
      We also introduce a new dependency on com.hubspot.jackson.datatype.protobuf to register the ProtobufModule() with the HttpRemoteConfigurationSource's ObjectMapper since the RemoteConfigurationRequest that is shared is an Otel resource proto and the object mapper needs to know how to serialize that object here.
      
      The corresponding changes in telemetry-api can be found in this PR.
      2b25cfc1
  3. 11月 08, 2022
  4. 11月 07, 2022
加载中