It's a scaffold for product usage activity tracking
for GitLab early access program. Only for participants
of the program and only who agreed on optional tracking.

Changelog: added

Creates indexes for the bigint version of pipeline_id
and swaps the columns.

Changelog: other

Creates indexes for the bigint version of pipeline_id
and swaps the columns.

Changelog: other

Adds column tickets_confidential_by_default.
Defaults to confidential (true). Only sets the
visibility to public if setting is false and project
is not public.

This index has already been removed asynchronously
on GitLab.com and needs to be also ran for other
installations synchronously.

Merge branch 'security-issue-452426' into 'master'

See merge request gitlab-org/security/gitlab!3962

Changelog: security

This migration was finished at `2023-09-19 05:04:03 UTC`, you can confirm
the status using our
[batched background migration chatops commands](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#monitor-the-progress-and-status-of-a-batched-background-migration).
  To confirm it is finished you can run:

  ```
/chatops run batched_background_migrations status 1000456
```

The last time this background migration was triggered was in [db/post_migrate/20230907155247_queue_backfill_has_merge_request_of_vulnerability_reads.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/post_migrate/20230907155247_queue_backfill_has_merge_request_of_vulnerability_reads.rb)

  You can read more about the process for finalizing batched background migrations in
https://docs.gitlab.com/ee/development/database/batched_background_migrations.html .

  As part of our process we want to ensure all batched background migrations have had at least one
[required stop](https://docs.gitlab.com/ee/development/database/required_stops.html)
to process the migration. Therefore we can finalize any batched background migration that was added before the
last required stop.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::OverdueFinalizeBackgroundMigration keep.

To provide feedback on your experience with `gitlab-housekeeper` please comment in
<https://gitlab.com/gitlab-org/gitlab/-/issues/442003>.

Changelog: other

Co-authored-by: Brian Williams <bwilliams@gitlab.com>

- Removes onboarding_step_url column

Changelog: removed
EE: true

This MR removes the support for newly_detected
vulnerability status for scan_result_policies

Changelog: removed
EE: true

- Changing to positive logic for
  the attributes related to access levels
- Renaming the fields `push_protected_up_to_access_level` and
  `delete_protected_up_to_access_level` in order to
  apply a positive logic (instead of negative logic), i.e.
  https://gitlab.com/gitlab-org/gitlab/-/merge_requests/135969#note_1688854082
- Renaming the field `push_protected_up_to_access_level`
  is renamed to field `minimum_access_level_for_push`
- Renaming the field `delete_protected_up_to_access_level`
  is renamed to field `minimum_access_level_for_delete`

Changelog: other

Add desired sharding keys for feature category `metrics`.

These tables have been identified as a [cell local tables](https://docs.gitlab.com/ee/development/database/multiple_databases.html#guidelines-on-choosing-between-gitlab_main_cell-and-gitlab_main_clusterwide-schema).
All cell local tables require a [sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-sharding-key-for-all-cell-local-tables)
or a [desired sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-desired_sharding_key-for-automatically-backfilling-a-sharding_key)

A desired sharding key has been automatically selected for these tables.
These keys were chosen as the desired sharding keys because the
table has a :belongs_to relationship to a table that itself has a `NOT NULL` sharding key.

Additionally, `gitlab_schema` has been set to `gitlab_main_cell` for any tables didn't use this schema already.
For these tables we have also added `allow_cross_joins`, `allow_cross_transactions` and
`allow_cross_foreign_keys`. These will silence any existing violations, allowing the pipeline to pass without
requiring further changes. In the future, we'll remove these `allow_...` statements and fix any violations as
they arise. You can read more about this in the [documentation for multiple databases](https://docs.gitlab.com/ee/development/database/multiple_databases.html)

We have assigned a random backend engineer from ~"" to review these changes.

Please confirm that:

- each of these tables can be classified as cell local
- the selected desired sharding key is appropriate
- the backfill configuration for the desired sharding key is correct

When you are finished, please request a review from the database maintainer suggested by Danger.
If you have any questions or concerns, reach out to @tigerwnz or @manojmj.

If you would like to go through similar merged MRs so as to gather an understanding on this topic, you
can use [this](https://gitlab.com/gitlab-org/gitlab/-/merge_requests?scope=all&state=merged&label_name[]=automation%3Agitlab-housekeeper-authored) link.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::DetermineDesiredShardingKeyFeatureCategory keep.

To provide feedback on your experience with `gitlab-housekeeper` please comment in
<https://gitlab.com/gitlab-org/gitlab/-/issues/442003>.

Changelog: other

Changelog: added

This reverts merge request !149730

- Moved Unique constraint validation to EE
- Applied review comments

EE: true
Changelog: added

## What does this MR do and why?

Add and backfill project_id for workspace_variables.

This table has a
[desired sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-desired_sharding_key-for-automatically-backfilling-a-sharding_key)
configured ([view configuration](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/docs/workspace_variables.yml)).

This merge request is the first step towards transforming the desired sharding key into a
[sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-sharding-key-for-all-cell-local-tables).

This involves three changes:

- Adding a new column that will serve as the sharding key (along with the relevant index and foreign key).
- Populating the sharding key when new records are created by adding a database function and trigger.
- Scheduling a [batched background migration](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html)
  to set the sharding key for existing records.

Once the background migration has completed, a second merge request will be created to finalize the background
migration and validate the not null constraint.

## How to verify

We have assigned a random backend engineer from ~"group::ide" to review these changes. Please review this merge
request from a ~backend perspective. The main thing we are looking to verify is that the added column and association
match the values specified by the [desired sharding key](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/docs/workspace_variables.yml)
configuration and that backfilling the column from this other table makes sense in the context of this feature.

When you are finished, please:

1. Trigger the [database testing pipeline](https://docs.gitlab.com/ee/development/database/database_migration_pipeline.html)
   as instructed by Danger.
1. Request a review from the ~backend maintainer and ~database reviewer suggested by Danger.

If you have any questions or concerns, reach out to @tigerwnz or @manojmj.

This merge request was generated by a once off keep implemented in
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/143774

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::BackfillDesiredShardingKeySmallTable keep.

To provide feedback on your experience with `gitlab-housekeeper` please comment in
<https://gitlab.com/gitlab-org/gitlab/-/issues/442003>.

Changelog: other

Changelog: removed

Added new placeholder user_type, added Imports::SourceUser, and service
to create them from imported user attributes.

EE: true
Changelog: changed

- See https://gitlab.com/gitlab-org/gitlab/-/issues/455158

This adds several migrations that add an imported column
to resources that will display an imported icon in the UI
if they are imported. The column is a smallint type that
will use an enum and default to 0. The default value 0
will indicate the resource was not imported. Any other
value will indicate the importer used (e.g. github,
bitbucket, etc.)

Changelog: added

The column project_keys is going to be used to fetch
jira issues instead of project_key.

For Service Ping

The asynchornous indexes were added in !gitlab-org/gitlab/-/merge_requests/147297

Changelog: removed

As titled, this is the first step of creating indexes asynchronously.
See guideline at https://docs.gitlab.com/ee/development/database/adding_database_indexes.html#schedule-the-index-to-be-created

Changelog: added

Changelog: added

This is the second step to synchronously create indexes for p_ci_builds
See https://docs.gitlab.com/ee/development/database/adding_database_indexes.html#add-a-migration-to-create-the-index-synchronously

Changelog: added

Changelog: added

The `inputs` column is no longer used, and all the data in it has been
migrated to the new `spec` column

Changelog: other
Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/455431

This change adds a new merge request approval policy action that allows
bot comment to be disabled for a given policy.

Changelog: other

Changelog: added
EE: true

Add relaxed sharding keys for feature category `groups_and_projects`.

These tables have been identified as
[cell local tables](https://docs.gitlab.com/ee/development/database/multiple_databases.html#guidelines-on-choosing-between-gitlab_main_cell-and-gitlab_main_clusterwide-schema).

All cell local tables require a
[sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-sharding-key-for-all-cell-local-tables).

A "relaxed" sharding key has been automatically selected for these tables, referencing either `projects` or `namespaces`, or a combination of both.

The term "relaxed" is used because:

- normally, a sharding key needs to have a NOT NULL constraint on the database.
- But, "relaxed" sharding keys do not have NOT NULL constraints. However, we have verified via database-lab that these
  columns do not contain any NULL values across any rows in the table.

This means that while the NOT NULL constraint is missing, the data itself is clean, so we can always add a NOT NULL constraint after the classification of these tables are completed.

Reviwer, please confirm that each table should indeed be cell local, and that the selected column is an appropriate
sharding key. When you are finished, please request a review from the database maintainer suggested by Danger.

If you have any questions or concerns, reach out to `@manojmj` or `@tigerwnz`.

If you would like to go through similar merged MRs so as to gather an understanding on this topic, you
can use [this](https://gitlab.com/gitlab-org/gitlab/-/merge_requests?scope=all&state=merged&label_name[]=automation%3Agitlab-housekeeper-authored) link.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::DetermineRelaxedShardingKey keep.

To provide feedback on your experience with `gitlab-housekeeper` please comment in
<https://gitlab.com/gitlab-org/gitlab/-/issues/442003>.

Changelog: other

- Reverify keys when pushing new commits
- Add a flash message when a user has a GPG key is not successfully
reverified by integrations
- Prevent pushes that contain commits that were signed with GPG keys
that are not successfully reverified

Contributes to: https://gitlab.com/gitlab-org/gitlab/-/issues/442098
Changelog: changed

Allow admins of enterprise groups to disable personal access token
generation for the enterprise users. Token which are created before
disabling personal access tokens will be invalid.

Changelog: fixed
EE: true

Add relaxed sharding keys for feature category `source_code_management`.

These tables have been identified as
[cell local tables](https://docs.gitlab.com/ee/development/database/multiple_databases.html#guidelines-on-choosing-between-gitlab_main_cell-and-gitlab_main_clusterwide-schema).

All cell local tables require a
[sharding key](https://docs.gitlab.com/ee/development/database/multiple_databases.html#defining-a-sharding-key-for-all-cell-local-tables).

A "relaxed" sharding key has been automatically selected for these tables, referencing either `projects` or `namespaces`, or a combination of both.

The term "relaxed" is used because:

- normally, a sharding key needs to have a NOT NULL constraint on the database.
- But, "relaxed" sharding keys do not have NOT NULL constraints. However, we have verified via database-lab that these
  columns do not contain any NULL values across any rows in the table.

This means that while the NOT NULL constraint is missing, the data itself is clean, so we can always add a NOT NULL constraint after the classification of these tables are completed.

Reviwer, please confirm that each table should indeed be cell local, and that the selected column is an appropriate
sharding key. When you are finished, please request a review from the database maintainer suggested by Danger.

If you have any questions or concerns, reach out to `@manojmj` or `@tigerwnz`.

If you would like to go through similar merged MRs so as to gather an understanding on this topic, you
can use [this](https://gitlab.com/gitlab-org/gitlab/-/merge_requests?scope=all&state=merged&label_name[]=automation%3Agitlab-housekeeper-authored) link.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::DetermineRelaxedShardingKey keep.

To provide feedback on your experience with `gitlab-housekeeper` please comment in
<https://gitlab.com/gitlab-org/gitlab/-/issues/442003>.

Changelog: other