Previously if a separate metrics server were configured and
Puma were stopped, the metrics server would continue to run. When
Puma is started again, it attempts to spawn a new metrics server,
but this fails repeatedly because the port has already been taken.

To fix this, add an `at_exit` handler that will shutdown the process
supervisor if Puma exists. This won't help the case if Puma is
abruptly killed, however.

Relates to https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8109

Changelog: fixed

This system was never in productive use and not
officially supported. We are retiring it now.

See https://gitlab.com/gitlab-org/gitlab-metrics-exporter/-/issues/31

The purpose of this Gem is to move all `Gitlab::Utils` code
into the Gem, and use this for all common functions that
are not monolith specific.

The purpose of this Gem is to move all `Gitlab::Utils` code
into the Gem, and use this for all common functions that
are not monolith specific.

- For better backward compatibility use string keys on
  GitlabSettings::Options
- Remove all the Settingslogic references from code (not comments)
- Ensure to use GitlabSettings::Options when pushing new options to
  Array values
- Instead of inheriting from Hash, wrap a hash in
  GitlabSettings::Options and make it respond to all the requirements.
    * PROS: This provides more autonomy on what needs to be done
    * CONS: Some places were checking by Hash, we need to tackle these
    scenarios
- Ensure to deep_merge return nested Options
- Use the _raw_ version of options when passing to Fog::Storage
- Ensure to load required active_support libraries
- Show the option name in the GitlabSettings::MissingSetting error
- Ensure to have an option `source` for the path of the `gitlab.yml`
- Fix GitlabSettings::Options#deep_merge and
  GitlabSettings::Options#merge to return a new GitlabSettings::Options
  object with the options deep_merged/merged
- Add GitlabSettings#reload! for backward compatibility
- Make file/namespace initializer arguments
- Avoid mutating Settings in sidekiq_cluster/cli tests
- Fix configuration stubbing

Related to: https://gitlab.com/gitlab-org/gitlab/-/issues/286873

Avoid using `[]` notation, use `.`.

Fixed automatically via:
  rubocop -A --only Performance/StringIdentifierArgument

Verified via:
  rake rubocop:todo:generate[Performance/StringIdentifierArgument]

This can reduce memory at the expense of higher
latencies.

Changelog: changed

We need to know to what extent the Ruby heap is
fragmented since this can lead to higher and sustained
memory use in production.

We add this metric to the existing RubySampler.

Changelog: added

This is required for FIPS compliance.

Changelog: added

Fixed:
- Do not determine `alive` status from the pidset returned
  by caller, since this can lead to "bouncing" state.
- Swap loop-check with sleep call, since sleep suspends
  the calling thread, which may lead to the callback
  being invoked if `alive` is invalidated by another
  thread.
- Do not trap INT and TERM by default; this was swallowing
  these signals for the Puma master.

Changelog: fixed

This is not in use yet, but we should already
pass in the correct settings.

It has been a long-standing issue that for Sidekiq,
serving metrics and health-checks was coupled into
a single server.

We had already separated settings keys for these two
concerns previously, in anticipation of splitting
these endpoints apart, which is a breaking change.

We can now do this with 15.0.

Changelog: changed

This change:

- Adds a rake task to install gitlab-metrics-exporter
  into a given directory.
- Allows to run it in place of the existing Ruby based
  exporters; this is guarded by an environment variable.
- Extends the existing end-to-end test to test both
  Ruby and Go implementations.

We were accidentally deleting metrics the exporter
server exported about itself. Actual sidekiq worker
metrics were not affected.

This happened because we would wipe the metrics dir
whenever sidekiq_0 starts up, but this happens after
the metrics server starts, so the worker was deleting
those existing metrics.

Changelog: fixed

This moves us away from a process-internal server thread
and toward an external process that we supervise instead.

This provides better memory use for Puma, and we
will require this for running a non-Ruby server
in the future.

This does not actually start the server for Puma
yet.

So far we had only tested this with Sidekiq.
This change makes sure that WebExporter and
the puma metrics directory can be used.

This is a prerequisite for extracting WebExporter
into a separate process.

This prevents metric names from clashing in cases
where multiple processes emit the same metrics
to the same scraper job and which do not have
a pid label.

This cuts down on long-term memory use with
a small increase in request latency.

This adds new Prometheus metrics such as
request count for the metrics server itself,
so we can observe scraper requests to
metrics server instances.

We move override_rails_constants.rb to another location so that it's
closer to its related code in metrics_server.
We also remove the usage in setup-test-env as they differ in usage, and
reusing the code does not make a lot of sense.