Skip to content
代码片段 群组 项目
  1. 9月 16, 2020
  2. 9月 15, 2020
  3. 9月 13, 2020
  4. 9月 12, 2020
  5. 9月 11, 2020
  6. 9月 10, 2020
    • ConfluentSemaphore's avatar
    • Lucas Bradstreet's avatar
    • ConfluentSemaphore's avatar
    • Tim Fox's avatar
    • ConfluentSemaphore's avatar
    • Anastasia Vela's avatar
      CNKAF-1138: Emit JSONs with auto-generated schema (#2487) · 40a2c5eb
      Anastasia Vela 创作于
      Currently request and response samples are outputted in a format that is JSON-like, so they aren't easily parseable. Once they're emitted as JSONs via the auto-generated schema, it'll be easier to load and parse them into other tools like druid and angle grinder.
      
      Included tests that ensure the outputted string is a parseable JSON.
      
      The following are some benchmark tests before and after the change respectively (will keep adding as I run more)
      BEFORE:
      ```
      Benchmark                                   (autoCreateTopic)  (partitionCount)  (topicCount)  Mode  Cnt    Score    Error  Units
      MetadataRequestBenchmark.testRequestToJson               true                10           500  avgt   15  638.312 ±  5.816  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                10          1000  avgt   15  659.132 ± 10.050  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                10          5000  avgt   15  633.611 ±  3.839  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                20           500  avgt   15  657.749 ± 18.178  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                20          1000  avgt   15  658.238 ±  1.981  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                20          5000  avgt   15  645.021 ±  1.231  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                50           500  avgt   15  655.577 ±  4.638  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                50          1000  avgt   15  625.718 ±  3.041  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                50          5000  avgt   15  588.551 ±  2.465  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                10           500  avgt   15  621.131 ±  5.757  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                10          1000  avgt   15  594.636 ±  1.447  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                10          5000  avgt   15  611.533 ±  0.716  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                20           500  avgt   15  633.064 ±  2.681  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                20          1000  avgt   15  625.159 ±  2.476  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                20          5000  avgt   15  634.090 ±  0.941  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                50           500  avgt   15  648.524 ±  4.881  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                50          1000  avgt   15  630.730 ±  2.502  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                50          5000  avgt   15  631.709 ±  2.584  ns/op
      ```
      ```
      Benchmark                                (partitionCount)  (topicCount)  Mode  Cnt         Score        Error  Units
      FetchRequestBenchmark.testRequestToJson                 3            10  avgt   15     14438.239 ±    467.117  ns/op
      FetchRequestBenchmark.testRequestToJson                 3           500  avgt   15    700629.960 ±  49586.125  ns/op
      FetchRequestBenchmark.testRequestToJson                 3          1000  avgt   15   1442081.215 ±  75845.206  ns/op
      FetchRequestBenchmark.testRequestToJson                10            10  avgt   15     46798.987 ±    995.295  ns/op
      FetchRequestBenchmark.testRequestToJson                10           500  avgt   15   2519359.200 ± 118128.637  ns/op
      FetchRequestBenchmark.testRequestToJson                10          1000  avgt   15   4840100.254 ± 182686.228  ns/op
      FetchRequestBenchmark.testRequestToJson                20            10  avgt   15     90212.311 ±    664.419  ns/op
      FetchRequestBenchmark.testRequestToJson                20           500  avgt   15   4918544.705 ± 223759.389  ns/op
      FetchRequestBenchmark.testRequestToJson                20          1000  avgt   15  10009446.657 ± 381793.614  ns/op
      ```
      ```
      Benchmark                                  (partitionCount)  (topicCount)  Mode  Cnt   Score   Error  Units
      ProduceRequestBenchmark.testRequestToJson                 3            10  avgt   15  54.673 ± 0.152  ns/op
      ProduceRequestBenchmark.testRequestToJson                 3           500  avgt   15  58.601 ± 0.252  ns/op
      ProduceRequestBenchmark.testRequestToJson                 3          1000  avgt   15  59.004 ± 0.257  ns/op
      ProduceRequestBenchmark.testRequestToJson                10            10  avgt   15  55.197 ± 0.383  ns/op
      ProduceRequestBenchmark.testRequestToJson                10           500  avgt   15  58.182 ± 0.267  ns/op
      ProduceRequestBenchmark.testRequestToJson                10          1000  avgt   15  62.475 ± 0.158  ns/op
      ProduceRequestBenchmark.testRequestToJson                20            10  avgt   15  57.036 ± 0.133  ns/op
      ProduceRequestBenchmark.testRequestToJson                20           500  avgt   15  60.630 ± 0.558  ns/op
      ProduceRequestBenchmark.testRequestToJson                20          1000  avgt   15  59.172 ± 0.380  ns/op
      ```
      ```
      Benchmark                                  (partitionCount)  (topicCount)  Mode  Cnt   Score   Error  Units
      ProduceRequestBenchmark.testRequestToJson                 3            10  avgt   15  68.553 ± 1.481  ns/op
      ProduceRequestBenchmark.testRequestToJson                 3           500  avgt   15  57.801 ± 0.116  ns/op
      ProduceRequestBenchmark.testRequestToJson                 3          1000  avgt   15  57.402 ± 0.761  ns/op
      ProduceRequestBenchmark.testRequestToJson                10            10  avgt   15  56.241 ± 0.623  ns/op
      ProduceRequestBenchmark.testRequestToJson                10           500  avgt   15  56.714 ± 0.552  ns/op
      ProduceRequestBenchmark.testRequestToJson                10          1000  avgt   15  58.318 ± 0.583  ns/op
      ProduceRequestBenchmark.testRequestToJson                20            10  avgt   15  54.154 ± 0.181  ns/op
      ProduceRequestBenchmark.testRequestToJson                20           500  avgt   15  58.505 ± 0.113  ns/op
      ProduceRequestBenchmark.testRequestToJson                20          1000  avgt   15  59.857 ± 0.224  ns/op
      ```
      
      AFTER:
      ```
      Benchmark                                   (autoCreateTopic)  (partitionCount)  (topicCount)  Mode  Cnt    Score   Error  Units
      MetadataRequestBenchmark.testRequestToJson               true                10           500  avgt   15  759.007 ± 5.810  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                10          1000  avgt   15  760.656 ± 1.718  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                10          5000  avgt   15  767.757 ± 3.772  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                20           500  avgt   15  743.840 ± 5.150  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                20          1000  avgt   15  743.902 ± 3.262  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                20          5000  avgt   15  743.548 ± 1.922  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                50           500  avgt   15  725.522 ± 1.401  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                50          1000  avgt   15  813.193 ± 5.587  ns/op
      MetadataRequestBenchmark.testRequestToJson               true                50          5000  avgt   15  755.759 ± 5.628  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                10           500  avgt   15  758.947 ± 1.367  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                10          1000  avgt   15  746.381 ± 5.538  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                10          5000  avgt   15  764.001 ± 2.020  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                20           500  avgt   15  762.420 ± 3.982  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                20          1000  avgt   15  763.226 ± 4.754  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                20          5000  avgt   15  755.896 ± 3.370  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                50           500  avgt   15  750.817 ± 1.920  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                50          1000  avgt   15  743.267 ± 1.704  ns/op
      MetadataRequestBenchmark.testRequestToJson              false                50          5000  avgt   15  767.195 ± 4.538  ns/op
      ```
      ```
      Benchmark                                (partitionCount)  (topicCount)  Mode  Cnt         Score        Error  Units
      FetchRequestBenchmark.testRequestToJson                 3            10  avgt   15     15621.968 ±    311.178  ns/op
      FetchRequestBenchmark.testRequestToJson                 3           500  avgt   15    979861.270 ±   4145.852  ns/op
      FetchRequestBenchmark.testRequestToJson                 3          1000  avgt   15   2014514.490 ±  19232.641  ns/op
      FetchRequestBenchmark.testRequestToJson                10            10  avgt   15     51342.178 ±    664.855  ns/op
      FetchRequestBenchmark.testRequestToJson                10           500  avgt   15   3365754.881 ±  35635.229  ns/op
      FetchRequestBenchmark.testRequestToJson                10          1000  avgt   15   6878298.687 ± 149076.429  ns/op
      FetchRequestBenchmark.testRequestToJson                20            10  avgt   15    103642.356 ±    233.205  ns/op
      FetchRequestBenchmark.testRequestToJson                20           500  avgt   15   6741431.033 ±  34171.026  ns/op
      FetchRequestBenchmark.testRequestToJson                20          1000  avgt   15  14141536.794 ±  27466.924  ns/op
      ```
      ```
      Benchmark                                  (partitionCount)  (topicCount)  Mode  Cnt    Score    Error  Units
      ProduceRequestBenchmark.testRequestToJson                 3            10  avgt   15  394.597 ± 10.652  ns/op
      ProduceRequestBenchmark.testRequestToJson                 3           500  avgt   15  397.004 ±  8.379  ns/op
      ProduceRequestBenchmark.testRequestToJson                 3          1000  avgt   15  391.991 ±  6.674  ns/op
      ProduceRequestBenchmark.testRequestToJson                10            10  avgt   15  396.227 ±  3.773  ns/op
      ProduceRequestBenchmark.testRequestToJson                10           500  avgt   15  396.368 ±  8.007  ns/op
      ProduceRequestBenchmark.testRequestToJson                10          1000  avgt   15  404.346 ± 13.188  ns/op
      ProduceRequestBenchmark.testRequestToJson                20            10  avgt   15  393.117 ± 10.497  ns/op
      ProduceRequestBenchmark.testRequestToJson                20           500  avgt   15  396.988 ±  6.357  ns/op
      ProduceRequestBenchmark.testRequestToJson                20          1000  avgt   15  397.248 ±  1.790  ns/op
      ```
      ```
      Benchmark                                     (partitionCount)  (topicCount)  Mode  Cnt    Score   Error  Units
      ListOffsetRequestBenchmark.testRequestToJson                 3            10  avgt   15  341.671 ± 0.704  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                 3           500  avgt   15  322.988 ± 0.972  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                 3          1000  avgt   15  328.373 ± 0.589  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                10            10  avgt   15  313.740 ± 0.848  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                10           500  avgt   15  349.826 ± 0.358  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                10          1000  avgt   15  348.513 ± 6.119  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                20            10  avgt   15  339.147 ± 1.250  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                20           500  avgt   15  315.552 ± 0.444  ns/op
      ListOffsetRequestBenchmark.testRequestToJson                20          1000  avgt   15  348.114 ± 0.780  ns/op
      ```
      40a2c5eb
    • ConfluentSemaphore's avatar
    • David Mao's avatar
      CNKAF-546: Reinitialize expired tenant metrics sensors (#2343) · 93d80a1d
      David Mao 创作于
      The various TenantMetrics sensor classes can keep around references to expired Sensors resulting in dropped metrics. We should be aware of sensors potentially expiring and reinitialize them if necessary.
      93d80a1d
    • Soumyarka Mondal's avatar
      KSTORAGE-909: Follow-up PR to durability audit PR (#2489) · d8fe9127
      Soumyarka Mondal 创作于
      * AuditJob reports durability lapse for relevant S3 error codes
      
      * Introduced time.sleep within AuditJob for throttled tierObjStore access
      
      * TierMetadataValidator uses read method to drain the buffer
      
      Relevant unit tests have been added, in addition to DEVEL cluster testing.
      
      reviewer: @rohitshekhar29 
      d8fe9127
加载中