Merge branch 'docs/bprescott/20221209-geodb' into 'master'

Locale and PostgreSQL indexes

See merge request https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106537



Merged-by: Achilleas Pipinellis <axil@gitlab.com>
Reviewed-by: Achilleas Pipinellis <axil@gitlab.com>
Co-authored-by: Ben Prescott <bprescott@gitlab.com>

doc/administration/geo/index.md

@@ -124,7 +124,9 @@ The following are required to run Geo:
 - Git 2.9 or later
 - Git-lfs 2.4.2 or later on the user side when using LFS
 - All sites must run [the same GitLab and PostgreSQL versions](setup/database.md#postgresql-replication).
-- If using different operating system versions between Geo sites, [check OS locale data compatibility](replication/troubleshooting.md#check-os-locale-data-compatibility) across Geo sites.
+  - If using different operating system versions between Geo sites,
+    [check OS locale data compatibility](replication/troubleshooting.md#check-os-locale-data-compatibility)
+    across Geo sites to avoid silent corruption of database indexes.
 
 Additionally, check the GitLab [minimum requirements](../../install/requirements.md),
 and use the latest version of GitLab for a better experience.

doc/administration/geo/replication/troubleshooting.md

@@ -1535,9 +1535,12 @@ If the above steps are **not successful**, proceed through the next steps:
 
 ## Check OS locale data compatibility
 
-If different operating systems or different operating system versions are deployed across Geo sites, we recommend that you perform a locale data compatibility check setting up Geo.
+If different operating systems or different operating system versions are deployed across Geo sites, you should perform a locale data compatibility check before setting up Geo.
 
-Geo uses PostgreSQL and Streaming Replication to replicate data across Geo sites. PostgreSQL uses locale data provided by the operating system’s C library for sorting text. If the locale data in the C library is incompatible across Geo sites, erroneous query results that lead to [incorrect behavior on secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/360723). See [here](https://wiki.postgresql.org/wiki/Locale_data_changes) for more details.
+Geo uses PostgreSQL and Streaming Replication to replicate data across Geo sites. PostgreSQL uses locale data provided by the operating system's C library for sorting text. If the locale data in the C library is incompatible across Geo sites, erroneous query results that lead to [incorrect behavior on secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/360723).
+
+For example, Ubuntu 18.04 (and earlier) and RHEL/Centos7 (and earlier) are incompatible with their later releases.
+See the [PostgreSQL wiki for more details](https://wiki.postgresql.org/wiki/Locale_data_changes).
 
 On all hosts running PostgreSQL, across all Geo sites, run the following shell command:
 
@@ -1561,4 +1564,8 @@ or the reverse order:
 
 If the output is identical on all hosts, then they running compatible versions of locale data.
 
-If the output differs on some hosts, then PostgreSQL replication will not work properly. We advise that you select operating system versions that are compatible.
+If the output differs on some hosts, PostgreSQL replication does not work properly: indexes are corrupted on the database replicas. You should select operating system versions that are compatible.
+
+A full index rebuild is required if the on-disk data is transferred 'at rest' to an operating system with an incompatible locale, or through replication.
+
+This check is also required when using a mixture of GitLab deployments. The locale might be different between an Linux package install, a GitLab Docker container, a Helm chart deployment, or external database services.

doc/install/requirements.md

@@ -120,6 +120,18 @@ test based on those. We try to be compatible with most external (not managed by
 Omnibus GitLab) databases (for example, [AWS Relational Database Service (RDS)](https://aws.amazon.com/rds/)),
 but we can't guarantee compatibility.
 
+#### Operating system locale compatibility and silent index corruption
+
+Changes to locale data in `glibc` means that PostgreSQL database files are not fully compatible
+between different OS releases.
+
+To avoid index corruption, [check for locale compatibility](../administration/geo/replication/troubleshooting.md#check-os-locale-data-compatibility)
+when:
+
+- Moving binary PostgreSQL data between servers.
+- Upgrading your Linux distribution.
+- Updating or changing third party container images.
+
 #### Gitaly Cluster database requirements
 
 [Read more in the Gitaly Cluster documentation](../administration/gitaly/praefect.md).