This exposes the correlation ID in the 'GitLab is currently unable to
handle this request due to load' in Workhorse, which should make
investigating these failures easier.

Changelog: changed

This MR adds an API endpoint to allow a user to perform bulk placeholder
reassignments by CSV upload without interacting with the GitLab UI.

Download endpoint MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/183360
Related issue: https://gitlab.com/gitlab-org/gitlab/-/issues/513794

This fixes some lint errors in `workhorse/internal/git/archive.go`.

When a user requests a Git archive, the resulting file is cached
in `Gitlab.config.gitlab.repository_downloads_path` by Workhorse
and cleaned up by the `RepositoryArchiveCleanUpService`, which runs
as a cronjob every hour.

However, on multi-node systems where Sidekiq and Workhorse are
on different nodes, `Gitlab.config.gitlab.repository_downloads_path`
may not be a shared volume. As a result, Sidekiq will not see any
stale files in that path, while archives build up over time.

Previously admins must make this a shared path for this to work, but
this introduces a single point of failure and more configuration to
get right. Alternatively, they can disable the cache.

Instead we observe that because Workhorse writes these files, we
can also have Workhorse clean the path periodically.

To avoid having to set a configuration variable, Workhorse gets the
`StoragePath` upon an archive request, which then initiates the
cleaner to run. However, this has the downside of requiring a single
user request to initiate the cleaning, but the tradeoff seems worth
it. We can always add the path as a configuration parameter later.

Relates to https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6005

Changelog: fixed

This reverts commit c8987b95.

Contributes to https://gitlab.com/gitlab-org/gitlab/-/issues/455049

Add log entry to track HTTP git operations in Workhorse.

Changelog: other

See
https://gitlab.com/gitlab-org/quality/engineering-productivity/master-broken-incidents/-/issues/10443#note_2343355904
. This library just bumped `go.mod` to require go
1.23 and it's breaking our CI pipeline.

Merge branch 'security-hide-sensitive-workhorse-send-data-header' into 'master'

See merge request gitlab-org/security/gitlab!4702

Changelog: security

A customer reported the Docker client returns `missing or empty
Content-Length header`:

```
$ docker pull gitlab.example.com:443/cloud/dependency_proxy/containers/alpine:latest
Error response from daemon: missing or empty Content-Length header
```

I suspect the issue relates to:

* https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/264
* https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/267

When a client requests an object to NetApp StorageGRID with
`Accept-Encoding: gzip`, StorageGRID uses HTTP chunked encoding and
omits `Content-Length`.

As mentioned with `go doc http.Transport.DisableCompression`, the Go
client will automatically add `Accept-Encoding: gzip` if
`Accept-Encoding` is not included.

It seems that for ZIP artifacts, we disabled gzip compression in
https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/508.

However, we should do the same and leave it to the Docker client to
determine whether gzip encoding can be used. On HEAD requests, the
Docker client omits `Accept-Encoding`, while with GET requests it
includes it. This is good because it means large objects will still be
transferred via gzip, while the small metadata won't necessarily be
compressed.

Changelog: fixed

Otherwise, Workhorse would return a 500, which could trip our infra
alerts needlessly.

Related to https://gitlab.com/gitlab-com/gl-infra/production/-/issues/19136

https://gitlab.com/gitlab-org/gitlab/-/merge_requests/177057 added new
fields to the Redis KeyWatcher structure that caused a `panic:
unaligned 64-bit atomic operation` on Raspberry Pi 32-bit builds for
Workhorse.

I was able to reproduce this by on a Raspberry Pi 64-bit system:

1. Install the 32-bit libraries:
   `apt install libc6:armhf libstdc++6:armhf`.

2. Build `gitlab-workhorse` with `GOARCH=arm` and `GOARM=7`
   environment variable set.

3. Spin up a Redis server.

4. Run `gitlab-workhorse` with this `config.toml`:

```
[redis]
URL = "redis://localhost:6379"
```

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/514010

Changelog: fixed

Related issue: https://gitlab.com/gitlab-org/gitlab/-/issues/460410

Previously a Redis PubSub channel would constantly be maintained
during startup. If a Redis `TIMEOUT` value were configured, we could
get into this repeated sequence:

1. Redis tears down the channel.
2. Workhorse reports an EOF error.
3. Workhrose reconnects after a backoff time.

We can avoid this by lazily initiating the publish/subscribe channel
only when we need it. Note that since we don't block on establishing
the channel, the first CI request might not be held in a long poll,
but subsequent requests should.

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/426006

Changelog: fixed

This enables us to drop all references to AWS SDK v1 now.

https://gitlab.com/gitlab-org/gitlab/-/merge_requests/164597
introduced support for AWS SDK v2 under a feature flag that has
been enabled by default for the last 5 months. We should be able
to drop this feature flag now.

Note that we still have some legacy test code that uses AWS SDK
v1. Updating this test code to use AWS SDK v2 will be done in a
separate commit.

Changelog: changed

Bumps the version to 0.33.0 to fix CVE-2024-45338

Changelog: security

Admins can now leave out the Azure account key and allow the Azure SDK
to retrieve it via the managed or workload identity process.

We have some hacks in our Makefile that allow for the generation of
reproducible GNU build IDs. The way this is implemented is kind of
ugly:

1. Build the required Workhorse binaries.

2. To compute a unique and deterministic value for GNU build-id, we
   build the Go binaries a second time.

3. From the first build, we extract its unique and deterministic Go
   build-id, and use that to derive a comparably unique and
   deterministic GNU build-id to inject into the final binary.

4. If we cannot extract a Go build-id, we punt and fallback to using a
   random 32-byte hex string.

This is rather complex and also wasteful, as we essentially have to build
the binaries twice.

To address this, we have upstreamed 089cc68676 (cmd/link: allow deriving
GNU build ID from Go build ID ID, 2023-09-16) that does this for us in
the Go build chain. When passing the `-B gobuildid` linker flag, then
the linker will compute the GNU build ID from the Go build ID and thus
do essentially the same as we did manually in Workhorse. This new feature
has been released with Go v1.22.

Now that we are on Go v1.23, we can finally use this new flag. This
allows us to simplify our build instructions.

In addition, this fixes fixes macOS build failures in Go v1.23.3,
introudced by 54728538 (cmd/link: generate Mach-O UUID when -B flag is
specified). On macOS, builds with our previous scheme failed:

```
link: -B option too long (max 16 digits): 0x<SHA>
```

This change aligns with what we have done in other projects:

- Pages:
  https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/1082

- Gitaly: https://gitlab.com/gitlab-org/gitaly/-/merge_requests/7221

Improved the parsing for adding the definitions on the frontend for code navigation.
Instead of just relying on the `start_char` there is a new property `end_char` that will
also be used. This new property will allow us to parse multiple definitions within
the same text node.
This new parsing also helps improves cases where text would disappear, syntax
highlighting would be lost. Instead of changing the elements we just update the
text node - keeping the syntax highlighting nodes.

https://gitlab.com/gitlab-org/gitlab/-/issues/386697

* replace broken reopen dependency https://gitlab.com/gitlab-org/labkit/-/merge_requests/191
* make semantic-release work again ([a97b341](https://gitlab.com/gitlab-org/labkit/commit/a97b3417c6420407f217f9d3baf7191d100c65b6))

See https://gitlab.com/gitlab-org/labkit/-/releases/v1.21.1

Add support in the workhorse senddependency action to include the
upload authorization response.

This way, the upload handler doesn't need to get the upload
authorization from the rails backend, it's already there.

This saves one request to the rails backend and shorten the overall
upload processing time.

Apply this shortcut in the maven virtual registry feature.