From b309f5d07381832ebac4b74e86b5b984236df072 Mon Sep 17 00:00:00 2001
From: Rutger Wessels <rwessels@gitlab.com>
Date: Mon, 15 Jan 2024 16:01:21 +0000
Subject: [PATCH] [Docs] Migrate to Decomposed Database Setup
---
.../postgresql/multiple_databases.md | 98 ++++++++++++++++++-
1 file changed, 93 insertions(+), 5 deletions(-)
diff --git a/doc/administration/postgresql/multiple_databases.md b/doc/administration/postgresql/multiple_databases.md
index 2caa4f147c4b..b91b0e573f00 100644
--- a/doc/administration/postgresql/multiple_databases.md
+++ b/doc/administration/postgresql/multiple_databases.md
@@ -1,4 +1,5 @@
---
+
stage: Data Stores
group: Tenant Scale
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
@@ -35,8 +36,95 @@ databases. Some examples:
## Known issues
- Once data is migrated to the `ci` database, you cannot migrate it back.
+- HA setups or PgBouncer setups are not yet supported by this procedure.
+
+## Migrate existing installations using a script
+
+> A script for migrating existing Linux package installations was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368729) in GitLab 16.8.
+
+NOTE:
+If something unexpected happens during the migration, it is safe to start over.
+
+### Existing Linux package installations
+
+#### Preparation
+
+1. Verify available disk space:
+
+ - The database node that will store the `gitlabhq_production_ci` database needs enough space to store a copy of the existing database: we _duplicate_ `gitlabhq_production`. Run the following SQL query to find out how much space is needed. Add 25%, to ensure you will not run out of disk space.
+
+ ```shell
+ sudo gitlab-psql -c "SELECT pg_size_pretty( pg_database_size('gitlabhq_production') );"
+ ```
-## Migrate existing installations
+ - During the process, a dump of the `gitlabhq_production` database needs to be temporarily stored on the filesystem of the node that will run the migration. Execute the following SQL statement to find out how much local disk space will be used. Add 25%, to ensure you will not run out of disk space.
+
+ ```shell
+ sudo gitlab-psql -c "select sum(pg_table_size(concat(table_schema,'.',table_name))) from information_schema.tables where table_catalog = 'gitlabhq_production' and table_type = 'BASE TABLE'"
+ ```
+
+1. Plan for downtime. The downtime is dependent on the size of the `gitlabhq_production` database.
+
+ - We dump `gitlabhq_production` and restore it into a new `gitlabhq_production_ci` database. Database sizes below 100 GB should be done within 30 minutes.
+ - We advise to also plan some time for smaller tasks like modifying the configuration.
+
+1. Create the new `gitlabhq_production_ci` database:
+
+ ```shell
+ sudo gitlab-psql -c "CREATE DATABASE gitlabhq_production_ci WITH OWNER 'gitlab'"
+ ```
+
+#### Migration
+
+This process includes downtime. Running the migration script will stop the GitLab instance. After the migration has been finished, the instance is restarted.
+
+1. Create a backup of the configuration:
+
+ ```shell
+ sudo cp /etc/gitlab/gitlab.rb /etc/gitlab/gitlab.rb.org
+ ```
+
+1. Edit `/etc/gitlab/gitlab.rb` and save the changes. Do **not** run the reconfigure command, the migration script will run that for you.
+
+ ```ruby
+ gitlab_rails['env'] = { 'GITLAB_ALLOW_SEPARATE_CI_DATABASE' => 'true' }
+ gitlab_rails['databases']['ci']['enable'] = true
+ gitlab_rails['databases']['ci']['db_database'] = 'gitlabhq_production_ci'
+ ```
+
+1. Run the migration script:
+
+ ```shell
+ sudo gitlab-ctl pg-decomposition-migration
+ ```
+
+At this point, the GitLab instance should start and be functional.
+
+If you want to abort the procedure and you want to start GitLab without changing anything, run the following commands:
+
+```shell
+sudo cp /etc/gitlab/gitlab.rb.org /etc/gitlab/gitlab.rb
+sudo gitlab-ctl reconfigure
+sudo gitlab-ctl restart
+```
+
+#### Cleaning up
+
+If everything works as expected, we can clean up unneeded data:
+
+- Delete the CI data in Main database:
+
+```shell
+sudo gitlab-rake gitlab:db:truncate_legacy_tables:main
+```
+
+- Delete the Main data in CI database:
+
+```shell
+sudo gitlab-rake gitlab:db:truncate_legacy_tables:ci
+```
+
+## Migrate existing installations (manual procedure)
To migrate existing data from the `main` database to the `ci` database, you can
copy the database across.
@@ -46,9 +134,9 @@ If something unexpected happens during the migration, it is safe to start over.
### Existing self-compiled installation
-1. [Disable background migrations](../../development/database/batched_background_migrations.md#enable-or-disable-background-migrations)
+1. [Disable background migrations](../../development/database/batched_background_migrations.md#enable-or-disable-background-migrations).
-1. [Ensure all background migrations are finished](../../update/background_migrations.md#check-the-status-of-batched-background-migrations)
+1. [Ensure all background migrations are finished](../../update/background_migrations.md#check-the-status-of-batched-background-migrations).
1. Stop GitLab, except for PostgreSQL:
@@ -112,7 +200,7 @@ the other way around.
### Self-compiled installations
1. For existing installations,
- [migrate the data](#migrate-existing-installations) first.
+ [migrate the data](#migrate-existing-installations-manual-procedure) first.
1. [Back up GitLab](../../administration/backup_restore/index.md)
in case of unforeseen issues.
@@ -168,7 +256,7 @@ the other way around.
### Linux package installations
1. For existing installations,
- [migrate the data](#migrate-existing-installations) first.
+ [migrate the data](#migrate-existing-installations-manual-procedure) first.
1. [Back up GitLab](../../administration/backup_restore/index.md)
in case of unforeseen issues.
--
GitLab