标签

This edge release introduces a number of different fixes and improvements. More
notably, it introduces a new `cni-repair-controller` binary to the CNI plugin
image. The controller will automatically restart pods that have not received
their iptables configuration.

* Removed shortnames from Tap API resources to avoid colliding with existing
  Kubernetes resources ([#11816]; fixes [#11784])
* Introduced a new ExternalWorkload CRD to support upcoming mesh expansion
  feature ([#11805])
* Changed `MeshTLSAuthentication` resource validation to allow SPIFFE URI
  identities ([#11882])
* Introduced a new `cni-repair-controller` to the `linkerd-cni` DaemonSet to
  automatically restart misconfigured pods that are missing iptables rules
  ([#11699]; fixes [#11073])
* Fixed a `"duplicate metrics"` warning in the multicluster service-mirror
  component ([#11875]; fixes [#11839])
* Added metric labels and weights to `linkerd diagnostics endpoints` json
  output ([#11889])
* Changed how `Server` updates are handled in the destination service. The
  change will ensure that during a cluster resync, consumers won't be
  overloaded by redundant updates ([#11907])
* Changed `linkerd install` error output to add a newline when a Kubernetes
  client cannot be successfully initialised ([#11917])

[#11816]: https://github.com/linkerd/linkerd2/pull/11816
[#11784]: https://github.com/linkerd/linkerd2/issues/11784
[#11805]: https://github.com/linkerd/linkerd2/pull/11805
[#11882]: https://github.com/linkerd/linkerd2/pull/11882
[#11699]: https://github.com/linkerd/linkerd2/pull/11699
[#11073]: https://github.com/linkerd/linkerd2/issues/11073
[#11875]: https://github.com/linkerd/linkerd2/pull/11875
[#11839]: https://github.com/linkerd/linkerd2/issues/11839
[#11889]: https://github.com/linkerd/linkerd2/pull/11889
[#11907]: https://github.com/linkerd/linkerd2/pull/11907
[#11917]: https://github.com/linkerd/linkerd2/pull/11917

## stable-2.14.8

This stable release fixes an issue in the control plane where discovery for pod
IP addresses could hang indefinitely ([#11815]).

[#11815]: https://github.com/linkerd/linkerd2/pull/11815
-----BEGIN SSH SIGNATURE-----
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgKsHYzTTiJc883LmaPRYSoBhwFm
oAaV1b0j4q1C1YYDsAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5
AAAAQPCs5ShukiLX1IyOXDmoNsDXf6O7uhukS8SMj89AIfbH6wtptu2o3nZBFJ7WK1t37i
Z0bRQKp/hMF9skKC15Swg=
-----END SSH SIGNATURE-----

This edge release contains improvements to the logging and diagnostics of the
destination controller.

* Added a control plane metric to count errors talking to the Kubernetes API
  ([#11774])
* Fixed an issue causing spurious destination controller error messages for
  profile lookups on unmeshed pods with port in default opaque list ([#11550])

[#11774]: https://github.com/linkerd/linkerd2/pull/11774
[#11550]: https://github.com/linkerd/linkerd2/pull/11550

This stable release fixes two bugs in the Linkerd control plane.

* Fixed an issue in the destination controller where the metadata API was not
  properly initialized for jobs, leading to error messages and unnecessary API
  calls ([#11541])
* Fixed an issue in the policy controller where it was overriding statuses on
  HTTPRoute resources from other controllers ([#11705])

[#11541]: https://github.com/linkerd/linkerd2/pull/11541
[#11705]: https://github.com/linkerd/linkerd2/pull/11705

## edge-23.12.2

This edge release includes a restructuring of the proxy's balancer along with
accompanying new metrics. The new minimum supported Kubernetes version is 1.22.

* Restructured the proxy's balancer ([#11750]): balancer changes may now occur
  independently of request processing. Fail-fast circuit breaking is enforced on
  the balancer's queue so that requests can't get stuck in a queue indefinitely.
  This new balancer is instrumented with new metrics: request (in-queue) latency
  histograms, failfast states, discovery updates counts, and balancer endpoint
  pool sizes.
* Changed how the policy controller updates HTTPRoute status so that it doesn't
  affect statuses from other non-linkerd controllers ([#11705]; fixes [#11659])

[#11750]: https://github.com/linkerd/linkerd2/pull/11750
[#11705]: https://github.com/linkerd/linkerd2/pull/11705
[#11659]: https://github.com/linkerd/linkerd2/pull/11659
-----BEGIN SSH SIGNATURE-----
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgKsHYzTTiJc883LmaPRYSoBhwFm
oAaV1b0j4q1C1YYDsAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5
AAAAQJ7En6fd9MFH022zea4nvy/qW7uzvE5/zMJNc0z02Sgha5A4MDBaMjbW2VkZF5W2QI
S/gi9sCcPmku4D90PPhgw=
-----END SSH SIGNATURE-----

stable-2.14.6

This stable release back-ports bugfixes and improvements from recent edge
releases.

* multicluster: Added an `imagePullSecrets` configuration to
  linkerd-multicluster Helm chart (thanks @lhaussknecht!). ([#11287])
* multicluster: Updated the service mirror to support gateways exposed on
  multiple IP addresses (thanks @MrFreezeex!) ([#11499])
* Updated control plane logging so that client-go may emit error logs. This will
  also ensures that all logs are emitted in JSON when the json log format is
  enabled. ([#11632])
* Added `kubeAPI.clientBurst` and `kubeAPI.clientQPS` configurations that allow
  users to configure the burst and QPS rate limits for the Kubernetes API
  clients used by the control plane. The default burst and qps values are now
  set at 200 and 100, respectively. The prior defaults limited bursts 10 and QPS
  to 5, which could cause throttling issues in clusters that schedule many pods
  quickly. ([#11644])
* viz: Update the default prometheus version to v2.48.0. ([#11633])

[#11287]: https://github.com/linkerd/linkerd2/pull/11287
[#11499]: https://github.com/linkerd/linkerd2/pull/11499
[#11632]: https://github.com/linkerd/linkerd2/pull/11632
[#11644]: https://github.com/linkerd/linkerd2/pull/11644
[#11633]: https://github.com/linkerd/linkerd2/pull/11633
-----BEGIN SSH SIGNATURE-----
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgd81rZBOcvM2l+Y/wYc9F7RfLtw
mSDbBgt7nNnPPHXKQAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5
AAAAQCcXWz5KBSe4PzSai8cf5kc2uDxVENh6hU1GAHYoT+pHZG7UDvO8F4yKUATkUPrqOZ
raJqRcFhtef4FY+Kam/AU=
-----END SSH SIGNATURE-----

This edge release introduces new configuration values in the identity
controller for client-go's `QPS` and `Burst` settings. Default values for these
settings have also been raised from `5` (QPS) and `10` (Burst) to `100` and
`200` respectively.

* Added `namespaceSelector` fields for the tap-injector and jaeger-injector
  webhooks. The webhooks are now configured to skip `kube-system` by default
  ([#11649]; fixes [#11647]) (thanks @mikutas!)
* Added the ability to configure client-go's `QPS` and `Burst` settings in the
  identity controller ([#11644])
* Improved client-go logging visibility throughout the control plane's
  components ([#11632])
* Introduced `PodDisruptionBudgets` in the linkerd-viz Helm chart for tap and
  tap-injector ([#11628]; fixes [#11248]) (thanks @mcharriere!)

[#11649]: https://github.com/linkerd/linkerd2/pull/11649
[#11647]: https://github.com/linkerd/linkerd2/issues/11647
[#11644]: https://github.com/linkerd/linkerd2/pull/11644
[#11632]: https://github.com/linkerd/linkerd2/pull/11632
[#11628]: https://github.com/linkerd/linkerd2/pull/11628
[#11248]: https://github.com/linkerd/linkerd2/issues/11248

## edge-23.11.4

This edge release introduces support for the native sidecar containers entering
beta support in Kubernetes 1.29. This improves the startup and shutdown ordering
for the proxy relative to other containers, fixing the long-standing
shutdown issue with injected `Job`s. Furthermore, traffic from other
`initContainer`s can now be proxied by Linkerd.

In addition, this edge release includes Helm chart improvements, and improvements
to the multicluster extension.

* Added a new `config.alpha.linkerd.io/proxy-enable-native-sidecar` annotation
  and `Proxy.NativeSidecar` Helm option that causes the proxy container to run
  as an init-container (thanks @teejaded!) ([#11465]; fixes [#11461])
* Fixed broken affinity rules for the multicluster `service-mirror` when running
  in HA mode ([#11609]; fixes [#11603])
* Added a new check to `linkerd check` that ensures all extension namespaces are
  configured properly ([#11629]; fixes [#11509])
* Updated the Prometheus Docker image used by the `linkerd-viz` extension to
  v2.48.0, resolving a number of CVEs in older Prometheus versions ([#11633])
* Added `nodeAffinity` to `deployment` templates in the `linkerd-viz` and
  `linkerd-jaeger` Helm charts (thanks @naing2victor!) ([#11464]; fixes
  [#10680])

[#11465]: https://github.com/linkerd/linkerd2/pull/11465
[#11461]: https://github.com/linkerd/linkerd2/issues/11461
[#11609]: https://github.com/linkerd/linkerd2/pull/11609
[#11603]: https://github.com/linkerd/linkerd2/issues/11603
[#11629]: https://github.com/linkerd/linkerd2/pull/11629
[#11509]: https://github.com/linkerd/linkerd2/issues/11509
[#11633]: https://github.com/linkerd/linkerd2/pull/11633
[#11464]: https://github.com/linkerd/linkerd2/pull/11464
[#10680]: https://github.com/linkerd/linkerd2/issues/10680

## stable-2.14.5

This stable release fixes a proxy regression where bursts of TCP connections
could result in EOF errors, due to an incorrect queue capacity. In addition, it
includes fixes for the control plane, dependency upgrades, and support for image
digests in Linkerd manifests.

* Added a controlPlaneVersion override to the `linkerd-control-plane` Helm chart
  to support including SHA256 image digests in Linkerd manifests (thanks
  @cromulentbanana!) ([#11406]; fixes [#11312])
* Added a `checksum/config` annotation to the destination and proxy injector
  deployment manifests, to force restarting those workloads whenever their
  webhook secrets change during upgrade (thanks @iAnomaly!) ([#11440]; fixes
  [#6940])
* Updated the Policy controller's OpenSSL dependency to v3, as OpenSSL 1.1.1 is
  EOL ([#11625])
* proxy: Increased `DEFAULT_OUTBOUND_TCP_QUEUE_CAPACITY` to prevent EOF errors
  during bursts of TCP connections (proxy PR [#2521][proxy-2521])

[#11406]: https://github.com/linkerd/linkerd2/pull/11406
[#11312]: https://github.com/linkerd/linkerd2/issues/11312
[#11440]: https://github.com/linkerd/linkerd2/pull/11440
[#6940]: https://github.com/linkerd/linkerd2/issues/6940
[#11625]: https://github.com/linkerd/linkerd2/pull/11625
[proxy-2521]: https://github.com/linkerd/linkerd2-proxy/pull/2521

This edge release fixes a bug where Linkerd could cause EOF errors during bursts
of TCP connections.

* Fixed a bug where the `linkerd multicluster link` command's
  `--gateway-addresses` flag was not respected when a remote gateway exists
  ([#11564])
* proxy: Increased DEFAULT_OUTBOUND_TCP_QUEUE_CAPACITY to prevent EOF errors
  during bursts of TCP connections

[#11564]: https://github.com/linkerd/linkerd2/pull/11564

This stable release improves observability for the control plane by adding
additional logging to the destination controller and by adding histograms which
can detect Kubernetes informer lag. It also adds the ability to configure
protocol detection.

* Improved logging in the destination controller by adding the client pod's
  name to the logging context. This will improve visibility into the messages
  sent and received by the control plane from a specific proxy ([#11532])
* helm: Introduce configurable values for protocol detection ([#11536])
* Fixed an issue where the Destination controller could stop processing service
  profile updates, if a proxy subscribed to those updates stops reading them;
  this is a followup to the issue [#11491] fixed in [stable-2.14.2] ([#11546])
* In the Destination controller, added informer lag histogram metrics to track
  whenever the Kubernetes objects watched by the controller are falling behind
  the state in the kube-apiserver ([#11534])
* proxy: Fix grpc_status metric labels for inbound traffic

[stable-2.14.2]: https://github.com/linkerd/linkerd2/releases/tag/stable-2.14.2
[#11532]: https://github.com/linkerd/linkerd2/pull/11532
[#11536]: https://github.com/linkerd/linkerd2/pull/11536
[#11546]: https://github.com/linkerd/linkerd2/pull/11546
[#11534]: https://github.com/linkerd/linkerd2/pull/11534

## edge-23.11.2

This edge release contains observability improvements and bug fixes to the
Destination controller, and a refinement to the multicluster gateway resolution
logic.

* Fixed an issue where the Destination controller could stop processing service
  profile updates, if a proxy subscribed to those updates stops reading them;
  this is a followup to the issue [#11491] fixed in [edge-23.10.3] ([#11546])
* In the Destination controller, added informer lag histogram metrics to track
  whenever the Kubernetes objects watched by the controller are falling behind
  the state in the kube-apiserver ([#11534])
* In the multicluster service mirror, extended the target gateway resolution
  logic to take into account all the possible IPs a hostname might resolve to,
  rather than just the first one (thanks @MrFreezeex!) ([#11499])
* Added probes to the debug container to appease environments requiring probes
  for all containers ([#11308])

[edge-23.10.3]: https://github.com/linkerd/linkerd2/releases/tag/edge-23.10.3
[#11546]: https://github.com/linkerd/linkerd2/pull/11546
[#11534]: https://github.com/linkerd/linkerd2/pull/11534
[#11499]: https://github.com/linkerd/linkerd2/pull/11499
[#11308]: https://github.com/linkerd/linkerd2/pull/11308

## stable-2.14.3

This stable release fixes an issue in the Destination controller that was
forbidding to route traffic to opaque ports on unmeshed pods. Also, it increases
the log level from debug to warning when the outbound proxy faces this type of
events.

* Fixed `GetProfiles` error in the Destination controller when address is opaque
  and unmeshed ([#11556], fixes[#11555])
* Started logging at warning level in the proxy when the controller clients
  receive an error ([#2499])

[#11556]: https://github.com/linkerd/linkerd2/pull/11556
[#11555]: https://github.com/linkerd/linkerd2/pull/11555
[#2499]: https://github.com/linkerd/linkerd2-proxy/pull/2499
-----BEGIN SSH SIGNATURE-----
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgKsHYzTTiJc883LmaPRYSoBhwFm
oAaV1b0j4q1C1YYDsAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5
AAAAQFmYVWp3NRKfDnf/bsN91okhoPrGep1AaUhsyRf81LCHkaCvf65uaBNu2/E2qtzbAi
DhlQVt7cQNnA+a2aQcWg4=
-----END SSH SIGNATURE-----

This edge release includes a fix for the `ServiceProfile` CRD resource schema.
The schema incorrectly required `not` response matches to be arrays, while the
in-cluster validator parsed `not` response matches as objects. In addition, an
issues has been fixed in `linkerd profile`. When used with the `--open-api`
flag, it would not strip trailing slashes when generating a resource from
swagger specifications.

* Fixed an issue where trailing slashes wouldn't be stripped when generating
  `ServiceProfile` resources through `linkerd profile --open-api` ([#11519])
* Fixed an issue in the `ServiceProfile` CRD schema. The schema incorrectly
  required that a `not` response match should be an array, which the service
  profile validator rejected since it expected an object. The schema has been
  updated to properly indicate that `not` values should be an object ([#11510];
  fixes [#11483])
* Improved logging in the destination controller by adding the client pod's
  name to the logging context. This will improve visibility into the messages
  sent and received by the control plane from a specific proxy ([#11532])
* Fixed an issue in the destination controller where the metadata API would not
  initialize a `Job` informer. The destination controller uses the metadata API
  to retrieve `Job` metadata, and relies mostly on informers. Without an
  initialized informer, an error message would be logged, and the controller
  relied on direct API calls ([#11541]; fixes [#11531])

[#11541]: https://github.com/linkerd/linkerd2/pull/11532
[#11532]: https://github.com/linkerd/linkerd2/pull/11532
[#11531]: https://github.com/linkerd/linkerd2/issues/11531
[#11519]: https://github.com/linkerd/linkerd2/pull/11519
[#11510]: https://github.com/linkerd/linkerd2/pull/11510
[#11483]: https://github.com/linkerd/linkerd2/issues/11483

This stable release fixes issues in the proxy and Destination controller which
can result in Linkerd proxies sending traffic to stale endpoints. In addition,
it contains a bug fix for profile resolutions for pods bound on host ports and
includes patches for security advisory [CVE-2023-44487]/GHSA-qppj-fm5r-hxr3

* Control Plane
  * Fixed an issue where the Destination controller could stop processing
    changes in the endpoints of a destination, if a proxy subscribed to that
    destination stops reading service discovery updates. This issue results in
    proxies attempting to send traffic for that destination to stale endpoints
    ([#11491], fixes [#11480], [#11279], [#10590])
  * Fixed an issue where the Destination controller would not update pod
    metadata for profile resolutions for a pod accessed via the host network
    (e.g. HostPort endpoints) ([#11334])
  * Addressed [CVE-2023-44487]/GHSA-qppj-fm5r-hxr3 by upgrading several
    dependencies (including Go's gRPC and net libraries)

* Proxy
  * Fixed a regression where the proxy rendered `grpc_status` metric labels as
    a string rather than as the numeric status code ([linkerd2-proxy#2480];
    fixes [#11449])
  * Fixed a regression introduced in stable-2.13.0 where proxies would not
    terminate unused service discovery watches, exerting backpressure on the
    Destination controller, potentially causing it to become
    stuck ([linkerd2-proxy#2484])

[#10590]: https://github.com/linkerd/linkerd2/issues/10590
[#11279]: https://github.com/linkerd/linkerd2/issues/11279
[#11491]: https://github.com/linkerd/linkerd2/issues/11491
[#11480]: https://github.com/linkerd/linkerd2/issues/11480
[#11334]: https://github.com/linkerd/linkerd2/pull/11334
[#11449]: https://github.com/linkerd/linkerd2/issues/11449
[CVE-2023-44487]: https://github.com/advisories/GHSA-qppj-fm5r-hxr3
[linkerd2-proxy#2480]: https://github.com/linkerd/linkerd2-proxy/pull/2480
[linkerd2-proxy#2484]: https://github.com/linkerd/linkerd2-proxy/pull/2484

## edge-23.10.3

This edge release fixes issues in the proxy and Destination controller which can
result in Linkerd proxies sending traffic to stale endpoints. In addition, it
contains other bugfixes and updates dependencies to include patches for the
security advisories [CVE-2023-44487]/GHSA-qppj-fm5r-hxr3 and GHSA-c827-hfw6-qwvm.

* Fixed an issue where the Destination controller could stop processing
  changes in the endpoints of a destination, if a proxy subscribed to that
  destination stops reading service discovery updates. This issue results in
  proxies attempting to send traffic for that destination to stale endpoints
  ([#11483], fixes [#11480], [#11279], and [#10590])
* Fixed a regression introduced in stable-2.13.0 where proxies would not
  terminate unused service discovery watches, exerting backpressure on the
  Destination controller which could cause it to become stuck
  ([linkerd2-proxy#2484] and [linkerd2-proxy#2486])
* Added `INFO`-level logging to the proxy when endpoints are added or removed
  from a load balancer. These logs are enabled by default, and can be disabled
  by [setting the proxy log level][proxy-log-level] to
  `warn,linkerd=info,linkerd_proxy_balance=warn` or similar
  ([linkerd2-proxy#2486])
* Fixed a regression where the proxy rendered `grpc_status` metric labels as a
  string rather than as the numeric status code ([linkerd2-proxy#2480]; fixes
  [#11449])
* Extended `linkerd-jaeger`'s `imagePullSecrets` Helm value to also apply to
the `namespace-metadata` ServiceAccount ([#11504])
* Updated the control plane's dependency on the `golang.google.org/grpc` Go
  package to include patches for [CVE-2023-44487]/GHSA-qppj-fm5r-hxr3 ([#11496])
* Updated dependencies on `rustix` to include patches for GHSA-c827-hfw6-qwvm
  ([linkerd2-proxy#2488] and [#11512]).

[#10590]: https://github.com/linkerd/linkerd2/issues/10590
[#11279]: https://github.com/linkerd/linkerd2/issues/11279
[#11483]: https://github.com/linkerd/linkerd2/issues/11483
[#11449]: https://github.com/linkerd/linkerd2/issues/11449
[#11480]: https://github.com/linkerd/linkerd2/issues/11480
[#11504]: https://github.com/linkerd/linkerd2/issues/11504
[#11512]: https://github.com/linkerd/linkerd2/issues/11512
[linkerd2-proxy#2480]: https://github.com/linkerd/linkerd2-proxy/pull/2480
[linkerd2-proxy#2484]: https://github.com/linkerd/linkerd2-proxy/pull/2484
[linkerd2-proxy#2486]: https://github.com/linkerd/linkerd2-proxy/pull/2486
[linkerd2-proxy#2488]: https://github.com/linkerd/linkerd2-proxy/pull/2488
[proxy-log-level]: https://linkerd.io/2.14/tasks/modifying-proxy-log-level/
[CVE-2023-44487]: https://github.com/advisories/GHSA-qppj-fm5r-hxr3

## edge-23.10.2

This edge release includes a fix addressing an issue during upgrades for
instances not relying on automated webhook certificate management (like
cert-manager provides).

* Added a `checksum/config` annotation to the destination and proxy injector
  deployment manifests, to force restarting those workloads whenever their
  webhook secrets change during upgrade (thanks @iAnomaly!) ([#11440])
* Fixed policy controller error when deleting a Gateway API HTTPRoute resource
  ([#11471])

[#11440]: https://github.com/linkerd/linkerd2/pull/11440
[#11471]: https://github.com/linkerd/linkerd2/pull/11471
-----BEGIN SSH SIGNATURE-----
U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgKsHYzTTiJc883LmaPRYSoBhwFm
oAaV1b0j4q1C1YYDsAAAADZ2l0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5
AAAAQLKKWtOSc/YJ6vRw7E7+LNBO1cqkkM4gMbCxAjMZMN2RfIE0C2uEmWDhTRl82qGrgg
akVmjywf/hhGiWDh1xJAI=
-----END SSH SIGNATURE-----

This edge release adds additional configurability to Linkerd's viz and
multicluster extensions.

* Added a `podAnnotations` Helm value to allow adding additional annotations to
  the Linkerd-Viz Prometheus Deployment ([#11365]) (thanks @cemenson)
* Added `imagePullSecrets` Helm values to the multicluster chart so that it can
  be installed in an air-gapped environment. ([#11285]) (thanks @lhaussknecht)

[#11365]: https://github.com/linkerd/linkerd2/issues/11365
[#11285]: https://github.com/linkerd/linkerd2/issues/11285