This change requeues SyncUnlinkedSecurityPolicyProjectLinks
as it failed for few records in gitlab.com.

EE: true
Changelog: fixed

Changelog: other

Along with the relevant classes, tests, and factories,
and adjust references to point to their new namespapce in EE.

Revert "Merge branch 'pedropombeiro/504963/replace-ci_runners-with-partitioned-table' into 'master'"

This reverts merge request !182549

New column with name dpop_check_for_manage_api_endpoints is added
to namespace_settings. This setting will control whether we need
to enforce dpop authentication on /manage endpoints or not

Changelog: added
EE: true

Added new column and controller setting

Add LFK on ci_running_builds/ci_runners

Changelog: other

Adds a sharding key and makes sure the AR model sets
it in a before_validation callback

Changelog: other

This MR adds a flag to the application settings for instance wide
 token prefixes. It also adds support for feed tokens as an example.

Changelog: added

Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/388379

Changelog: other

Changelog: other

Changelog: other

Changelog: changed
EE: true

## What does this MR do and why?

Add and backfill project_id for push_event_payloads.

This table has a
[desired sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#define-a-desired_sharding_key-to-automatically-backfill-a-sharding_key)
configured ([view configuration](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/docs/push_event_payloads.yml)).

This merge request is the first step towards transforming the desired sharding key into a
[sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-sharding-key-for-all-cell-local-tables).

This involves the following changes:

- Adding a new column that will serve as the sharding key (along with the relevant asynchronous index).
- Populating the sharding key when new records are created by adding a database function and trigger.
- Scheduling a [batched background migration](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html)
  to set the sharding key for existing records.

Once the background migration has completed, a second merge request will be created to finalize the background
migration, and add a foreign key and not null constraint.

## How to verify

We have assigned a random backend engineer from ~"group::source code" to review these changes. Please review this merge
request from a ~backend perspective. The main thing we are looking to verify is that the added column and association
match the values specified by the [desired sharding key](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/docs/push_event_payloads.yml)
configuration and that backfilling the column from this other table makes sense in the context of this feature.

When you are finished, please:

1. Trigger the [database testing pipeline](https://docs.gitlab.com/ee/development/database/database_migration_pipeline.html)
   as instructed by Danger.
1. Request a review from the ~backend maintainer and ~database reviewer suggested by Danger.

If you have any questions or concerns, reach out to @tigerwnz or @shubhamkrai.

This merge request was generated by a once off keep implemented in
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/143774

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::BackfillDesiredShardingKeyLargeTable keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

This migration was finished at `2024-08-20 00:52:02 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000651
```

The last time this background migration was triggered was in [db/post_migrate/20240815083838_queue_backfill_protected_env_deploy_access_levels_protected_env_project_id.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240815083838_queue_backfill_protected_env_deploy_access_levels_protected_env_project_id.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

Return distinct years and distinct runner for dropdowns
in visualisations

Add specs and update GQL documentation

Changelog: added
EE: true
MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/181383

In this MR, we add a new setting to Namespace::PackageSetting so that users can toggle it in order to enable the audit events for package registry.

Changelog: added

Validate the NOT NULL constraint on the project_id
column on the approval_merge_request_rules table

Changelog: other

## What does this MR do and why?

Add and backfill project_id for scan_result_policies.

This table has a
[desired sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#define-a-desired_sharding_key-to-automatically-backfill-a-sharding_key)
configured ([view configuration](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/docs/scan_result_policies.yml)).

This merge request is the first step towards transforming the desired sharding key into a
[sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-sharding-key-for-all-cell-local-tables).

This involves three changes:

- Adding a new column that will serve as the sharding key (along with the relevant index and foreign key).
- Populating the sharding key when new records are created by adding a database function and trigger.
- Scheduling a [batched background migration](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html)
  to set the sharding key for existing records.

Once the background migration has completed, a second merge request will be created to finalize the background
migration and validate the not null constraint.

## How to verify

We have assigned a random backend engineer from ~"group::security policies" to review these changes. Please review this merge
request from a ~backend perspective. The main thing we are looking to verify is that the added column and association
match the values specified by the [desired sharding key](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/docs/scan_result_policies.yml)
configuration and that backfilling the column from this other table makes sense in the context of this feature.

When you are finished, please:

1. Trigger the [database testing pipeline](https://docs.gitlab.com/ee/development/database/database_migration_pipeline.html)
   as instructed by Danger.
1. Request a review from the ~backend maintainer and ~database reviewer suggested by Danger.

If you have any questions or concerns, reach out to `@tigerwnz` or @shubhamkrai.

This merge request was generated by a once off keep implemented in
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/143774

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::BackfillDesiredShardingKeySmallTable keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

This migration was finished at `2024-06-21 21:20:04 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000598
```

The last time this background migration was triggered was in [db/post_migrate/20240618122511_queue_backfill_protected_tag_create_access_levels_project_id.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240618122511_queue_backfill_protected_tag_create_access_levels_project_id.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

This adds `organization_id` column to the `fork_networks` table to
support sharding on Cells 1.0. It also provides an index to the newly
added organization_id column.

Changelog: added

Changelog: changed

This migration was finished at `2024-09-11 04:51:07 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000663
```

The last time this background migration was triggered was in [db/post_migrate/20240906131415_queue_backfill_ml_candidate_params_project_id.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240906131415_queue_backfill_ml_candidate_params_project_id.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

This migration was finished at `2024-08-06 05:38:06 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000636
```

The last time this background migration was triggered was in [db/post_migrate/20240729124632_queue_make_security_policy_bots_private.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240729124632_queue_make_security_policy_bots_private.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

This migration was finished at `2024-08-20 06:36:03 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000652
```

The last time this background migration was triggered was in [db/post_migrate/20240815083843_queue_backfill_protected_env_deploy_access_levels_protected_env_group_id.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240815083843_queue_backfill_protected_env_deploy_access_levels_protected_env_group_id.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

- If the feature flag is enabled globally then we'll
  assume that the instance wants to seemlessly
  transition to the feature being enabled within
  the application settings.

Changelog: changed

Update file-based Project/Group Import to remove import_file
used to do the import to a cron job and not during the import
process to allow future restarts of the process.

Changelog: other

This migration was finished at `2024-08-06 23:32:01 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000638
```

The last time this background migration was triggered was in [db/post_migrate/20240801195253_queue_backfill_reserved_storage_bytes.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240801195253_queue_backfill_reserved_storage_bytes.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

We currently have no way to expose an ancestor dependency path for a
component. This MR is a POC of hwo we could achieve this.

Changelog: added
EE: true

Changelog: other

This migration was finished at `2024-10-04 10:00:02 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000340
```

The last time this background migration was triggered was in [db/post_migrate/20240930144643_queue_backfill_ci_builds_runner_session_project_id.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240930144643_queue_backfill_ci_builds_runner_session_project_id.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

This migration was finished at `2024-09-16 21:26:11 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000666
```

The last time this background migration was triggered was in [db/post_migrate/20240830093237_queue_backfill_ml_candidate_metrics_project_id.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240830093237_queue_backfill_ml_candidate_metrics_project_id.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

This adds a Batched Background Migration that migrates the CI/CD
setting `restrict_user_defined_variables: false` to `true` while
also keeping the behavior backwards compatible by updating
`pipeline_variables_minimum_override_role: :developer`.

This migrations is necessary in order to deprecate the setting
`restrict_user_defined_variables` in 18.0.
In order to do that we need to have set the setting
`restrict_user_defined_variables: true` everywhere and solely
rely on `pipeline_variables_minimum_override_role`.

Changelog: changed

Removing the confidence and  confidence_overridden
columns from the vulernerabilities table with this migration,
as this column has already been ignored in a past release.

Changelog: removed

Changelog: changed
EE: true

Changelog: added

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/516947

Changelog: changed
EE: true

This migration was finished at `2024-07-27 00:15:03 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000625
```

The last time this background migration was triggered was in [db/post_migrate/20240618123929_queue_backfill_epic_issues_namespace_id.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20240618123929_queue_backfill_epic_issues_namespace_id.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other