Changelog: other

Deprecated in https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/2760
in favor of the new `concurrency` option.

Changelog: removed

This reverts merge request !149690

Deprecated in https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/2760
in favor of the new `concurrency` option.

Changelog: removed

Removal of queue selector and negate options for 17.0

https://gitlab.com/gitlab-org/gitlab/-/merge_requests/149370

Changelog: removed

When routing rules is not specified, all jobs go to `default` and
`mailers` queue by default. We can safely override
sidekiq queue_groups to default,mailers queues only so that Redis don't
incur overhead of BRPOPing 600+ queues.

Changelog: changed

The new concurrency option is a fixed value that doesnt depend on the
queues length. It is means to be a simple replacement over the
min-concurrency and max-concurrency.
https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/2760

Changelog: added

- ee/lib/**
- rubocop/**
- scripts/**
- sidekiq_cluster/**
- tooling/**

The purpose of this Gem is to move all `Gitlab::Utils` code
into the Gem, and use this for all common functions that
are not monolith specific.

The purpose of this Gem is to move all `Gitlab::Utils` code
into the Gem, and use this for all common functions that
are not monolith specific.

This gem is supposed to contain all source
code required to configure GitLab specific
rspec environment.

Changelog: added

Update default initializer value for Sidekiq routing_rules

See merge request gitlab-org/gitlab!97908

It was previously updated from 50 to 20, but there are still some places
defining the default as 50.

Fixed automatically via:
  rubocop -A --only Performance/StringIdentifierArgument

Verified via:
  rake rubocop:todo:generate[Performance/StringIdentifierArgument]

sidekiq-cluster handles process supervision for its child Sidekiq
processes, and terminates itself and all child processes if any child
Sidekiq process exits.

Previously, it always exited with a 0 status code (i.e. success), no
matter how the child process had terminated.

Now it exits with 1 if any child process had a non-zero exit code. This
allows a process supervisor one level up (like systemd) to detect
failures and restart.

Changelog: changed

The previous default of 50 was far too high. We don't use this on
GitLab.com and it's mostly inertia that has left it at this value.

Changelog: changed

Fixed:
- Do not determine `alive` status from the pidset returned
  by caller, since this can lead to "bouncing" state.
- Swap loop-check with sleep call, since sleep suspends
  the calling thread, which may lead to the callback
  being invoked if `alive` is invalidated by another
  thread.
- Do not trap INT and TERM by default; this was swallowing
  these signals for the Puma master.

Changelog: fixed

It has been a long-standing issue that for Sidekiq,
serving metrics and health-checks was coupled into
a single server.

We had already separated settings keys for these two
concerns previously, in anticipation of splitting
these endpoints apart, which is a breaking change.

We can now do this with 15.0.

Changelog: changed

This change:

- Adds a rake task to install gitlab-metrics-exporter
  into a given directory.
- Allows to run it in place of the existing Ruby based
  exporters; this is guarded by an environment variable.
- Extends the existing end-to-end test to test both
  Ruby and Go implementations.

There was a slight chance that the metrics directory got
wiped after workers had started up, which would result
in their metrics missing.

We were accidentally deleting metrics the exporter
server exported about itself. Actual sidekiq worker
metrics were not affected.

This happened because we would wipe the metrics dir
whenever sidekiq_0 starts up, but this happens after
the metrics server starts, so the worker was deleting
those existing metrics.

Changelog: fixed

This moves us away from a process-internal server thread
and toward an external process that we supervise instead.

This extracts the process supervision logic
from sidekiq-cluster into a helper module.

We will re-use this for Puma soon.

This provides better memory use for Puma, and we
will require this for running a non-Ruby server
in the future.

This does not actually start the server for Puma
yet.

These methods are not directly related to SidekiqCluster process
management. They are currently only used by sidekiq cluster,
but in the future they may be used in other places as well.

This prevents name clashes with library code that
is auto-loaded in workers from lib/