From 1d29f3cd0388e58c3ee0ca5bf8e73869c7ccf73a Mon Sep 17 00:00:00 2001 From: Aakriti Gupta <agupta@gitlab.com> Date: Tue, 6 Feb 2024 22:03:46 +0000 Subject: [PATCH] Add documentation on object storage replication introduced in 16.4 --- .../geo/replication/datatypes.md | 46 ++++++++++--------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index 99d42d33a2a6f..30540f1cec61e 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -193,33 +193,35 @@ successfully, you must replicate their data using some other means. |Feature | Replicated (added in GitLab version) | Verified (added in GitLab version) | GitLab-managed object storage replication (added in GitLab version) | GitLab-managed object storage verification (added in GitLab version) | Notes | |:--------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------|:---------------------------------------------------------------------------|:--------------------------------------------------------------------|:----------------------------------------------------------------|:------| |[Application data in PostgreSQL](../../postgresql/index.md) | **Yes** (10.2) | **Yes** (10.2) | Not applicable | Not applicable | | -|[Project repository](../../../user/project/repository/index.md) | **Yes** (10.2) | **Yes** (10.7) | Not applicable | Not applicable | Migrated to [self-service framework](../../../development/geo/framework.md) in 16.2. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details.<br /><br />Behind feature flag `geo_project_repository_replication`, enabled by default in (16.3).<br /><br /> All projects, including [archived projects](../../../user/project/settings/migrate_projects.md#archive-a-project), are replicated. | -|[Project wiki repository](../../../user/project/wiki/index.md) | **Yes** (10.2)<sup>2</sup> | **Yes** (10.7)<sup>2</sup> | Not applicable | Not applicable | Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details.<br /><br />Behind feature flag `geo_project_wiki_repository_replication`, enabled by default in (15.11). | -|[Group wiki repository](../../../user/project/wiki/group.md) | [**Yes** (13.10)](https://gitlab.com/gitlab-org/gitlab/-/issues/208147) | [**Yes** (16.3)](https://gitlab.com/gitlab-org/gitlab/-/issues/323897) | Not applicable | Not applicable | Behind feature flag `geo_group_wiki_repository_replication`, enabled by default. | -|[Uploads](../../uploads.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Replication is behind the feature flag `geo_upload_replication`, enabled by default. Verification was behind the feature flag `geo_upload_verification`, removed in 14.8. | -|[LFS objects](../../lfs/index.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | GitLab versions 11.11.x and 12.0.x are affected by [a bug that prevents any new LFS objects from replicating](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).<br /><br />Replication is behind the feature flag `geo_lfs_object_replication`, enabled by default. Verification was behind the feature flag `geo_lfs_object_verification`, removed in 14.7. | -|[Personal snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | Not applicable | Not applicable | | -|[Project snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | Not applicable | Not applicable | | -|[CI job artifacts](../../../ci/jobs/job_artifacts.md) | **Yes** (10.4) | **Yes** (14.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Verification is behind the feature flag `geo_job_artifact_replication`, enabled by default in 14.10. | -|[CI Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/pipeline_artifact.rb) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Persists additional artifacts after a pipeline completes. | -|[CI Secure Files](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/secure_file.rb) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (16.4)](object_storage.md) | Verification is behind the feature flag `geo_ci_secure_file_replication`, enabled by default in 15.3. | +|[Project repository](../../../user/project/repository/index.md) | **Yes** (10.2) | **Yes** (10.7) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Migrated to [self-service framework](../../../development/geo/framework.md) in 16.2. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details.<br /><br />Behind feature flag `geo_project_repository_replication`, enabled by default in (16.3).<br /><br /> All projects, including [archived projects](../../../user/project/settings/migrate_projects.md#archive-a-project), are replicated. | +|[Project wiki repository](../../../user/project/wiki/index.md) | **Yes** (10.2)<sup>2</sup> | **Yes** (10.7)<sup>2</sup> | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details.<br /><br />Behind feature flag `geo_project_wiki_repository_replication`, enabled by default in (15.11). | +|[Group wiki repository](../../../user/project/wiki/group.md) | [**Yes** (13.10)](https://gitlab.com/gitlab-org/gitlab/-/issues/208147) | [**Yes** (16.3)](https://gitlab.com/gitlab-org/gitlab/-/issues/323897) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Behind feature flag `geo_group_wiki_repository_replication`, enabled by default. | +|[Uploads](../../uploads.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Replication is behind the feature flag `geo_upload_replication`, enabled by default. Verification was behind the feature flag `geo_upload_verification`, removed in 14.8. | +|[LFS objects](../../lfs/index.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | GitLab versions 11.11.x and 12.0.x are affected by [a bug that prevents any new LFS objects from replicating](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).<br /><br />Replication is behind the feature flag `geo_lfs_object_replication`, enabled by default. Verification was behind the feature flag `geo_lfs_object_verification`, removed in 14.7. | +|[Personal snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | | +|[Project snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | | +|[CI job artifacts](../../../ci/jobs/job_artifacts.md) | **Yes** (10.4) | **Yes** (14.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Verification is behind the feature flag `geo_job_artifact_replication`, enabled by default in 14.10. | +|[CI Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/pipeline_artifact.rb) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Persists additional artifacts after a pipeline completes. | +|[CI Secure Files](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/secure_file.rb) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Verification is behind the feature flag `geo_ci_secure_file_replication`, enabled by default in 15.3. | |[Container registry](../../packages/container_registry.md) | **Yes** (12.3)<sup>1</sup> | **Yes** (15.10) | **Yes** (12.3)<sup>1</sup> | **Yes** (15.10) | See [instructions](container_registry.md) to set up the container registry replication. | -|[Terraform Module Registry](../../../user/packages/terraform_module_registry/index.md) | **Yes** (14.0) | **Yes** (14.0) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | **Yes** (16.1) | Not applicable | Not applicable | Designs also require replication of LFS objects and Uploads. | -|[Package registry](../../../user/packages/package_registry/index.md) | **Yes** (13.2) | **Yes** (13.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Versioned Terraform State](../../terraform_state.md) | **Yes** (13.5) | **Yes** (13.12) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Replication is behind the feature flag `geo_terraform_state_version_replication`, enabled by default. Verification was behind the feature flag `geo_terraform_state_version_verification`, which was removed in 14.0. | -|[External merge request diffs](../../merge_request_diffs.md) | **Yes** (13.5) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Replication is behind the feature flag `geo_merge_request_diff_replication`, enabled by default. Verification was behind the feature flag `geo_merge_request_diff_verification`, removed in 14.7.| -|[Versioned snippets](../../../user/snippets.md#versioned-snippets) | [**Yes** (13.7)](https://gitlab.com/groups/gitlab-org/-/epics/2809) | [**Yes** (14.2)](https://gitlab.com/groups/gitlab-org/-/epics/2810) | Not applicable | Not applicable | Verification was implemented behind the feature flag `geo_snippet_repository_verification` in 13.11, and the feature flag was removed in 14.2. | -|[GitLab Pages](../../pages/index.md) | [**Yes** (14.3)](https://gitlab.com/groups/gitlab-org/-/epics/589) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)](object_storage.md) | Behind feature flag `geo_pages_deployment_replication`, enabled by default. Verification was behind the feature flag `geo_pages_deployment_verification`, removed in 14.7. | -|[Project-level Secure files](../../../ci/secure_files/index.md) | **Yes** (15.3) | **Yes** (15.3) | **Yes** (15.3) | [**Yes** (16.4)](object_storage.md) | | -| [Incident Metric Images](../../../operations/incident_management/incidents.md#metrics) | **Yes** (15.5) | **Yes** (15.5) | **Yes** (15.5) | [**Yes** (16.4)](object_storage.md) | Replication/Verification is handled via the Uploads data type. | | -|[Alert Metric Images](../../../operations/incident_management/alerts.md#metrics-tab) | **Yes** (15.5) | **Yes** (15.5) | **Yes** (15.5) | [**Yes** (16.4)](object_storage.md) | Replication/Verification is handled via the Uploads data type. | +|[Terraform Module Registry](../../../user/packages/terraform_module_registry/index.md) | **Yes** (14.0) | **Yes** (14.0) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Behind feature flag `geo_package_file_replication`, enabled by default. | +|[Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | **Yes** (16.1) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Designs also require replication of LFS objects and Uploads. | +|[Package registry](../../../user/packages/package_registry/index.md) | **Yes** (13.2) | **Yes** (13.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Behind feature flag `geo_package_file_replication`, enabled by default. | +|[Versioned Terraform State](../../terraform_state.md) | **Yes** (13.5) | **Yes** (13.12) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Replication is behind the feature flag `geo_terraform_state_version_replication`, enabled by default. Verification was behind the feature flag `geo_terraform_state_version_verification`, which was removed in 14.0. | +|[External merge request diffs](../../merge_request_diffs.md) | **Yes** (13.5) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Replication is behind the feature flag `geo_merge_request_diff_replication`, enabled by default. Verification was behind the feature flag `geo_merge_request_diff_verification`, removed in 14.7.| +|[Versioned snippets](../../../user/snippets.md#versioned-snippets) | [**Yes** (13.7)](https://gitlab.com/groups/gitlab-org/-/epics/2809) | [**Yes** (14.2)](https://gitlab.com/groups/gitlab-org/-/epics/2810) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Verification was implemented behind the feature flag `geo_snippet_repository_verification` in 13.11, and the feature flag was removed in 14.2. | +|[GitLab Pages](../../pages/index.md) | [**Yes** (14.3)](https://gitlab.com/groups/gitlab-org/-/epics/589) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Behind feature flag `geo_pages_deployment_replication`, enabled by default. Verification was behind the feature flag `geo_pages_deployment_verification`, removed in 14.7. | +|[Project-level Secure files](../../../ci/secure_files/index.md) | **Yes** (15.3) | **Yes** (15.3) | **Yes** (15.3) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | | +| [Incident Metric Images](../../../operations/incident_management/incidents.md#metrics) | **Yes** (15.5) | **Yes** (15.5) | **Yes** (15.5) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Replication/Verification is handled via the Uploads data type. | | +|[Alert Metric Images](../../../operations/incident_management/alerts.md#metrics-tab) | **Yes** (15.5) | **Yes** (15.5) | **Yes** (15.5) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | Replication/Verification is handled via the Uploads data type. | |[Server-side Git hooks](../../server_hooks.md) | [Not planned](https://gitlab.com/groups/gitlab-org/-/epics/1867) | No | Not applicable | Not applicable | Not planned because of current implementation complexity, low customer interest, and availability of alternatives to hooks. | |[Elasticsearch integration](../../../integration/advanced_search/elasticsearch.md) | [Not planned](https://gitlab.com/gitlab-org/gitlab/-/issues/1186) | No | No | No | Not planned because further product discovery is required and Elasticsearch (ES) clusters can be rebuilt. Secondaries use the same ES cluster as the primary. | -|[Dependency Proxy Images](../../../user/packages/dependency_proxy/index.md) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (16.4)](object_storage.md) | | +|[Dependency Proxy Images](../../../user/packages/dependency_proxy/index.md) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (15.7)](https://gitlab.com/groups/gitlab-org/-/epics/8833) | [**Yes** (16.4)<sup>3</sup>](https://gitlab.com/groups/gitlab-org/-/epics/8056) | | |[Vulnerability Export](../../../user/application_security/vulnerability_report/index.md#export-vulnerability-details) | [Not planned](https://gitlab.com/groups/gitlab-org/-/epics/3111) | No | No | No | Not planned because they are ephemeral and sensitive information. They can be regenerated on demand. | |Packages NPM metadata cache | [Not planned](https://gitlab.com/gitlab-org/gitlab/-/issues/408278) | No | No | No | Not planned because it would not notably improve disaster recovery capabilities nor response times at secondary sites. | <sup>1</sup> Migrated to [self-service framework](../../../development/geo/framework.md) in 15.5. See GitLab issue [#337436](https://gitlab.com/gitlab-org/gitlab/-/issues/337436) for more details. -<sup>2</sup> Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. Behind feature flag `geo_project_wiki_repository_replication`, enabled by default. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details. +<sup>2</sup> Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. Behind feature flag `geo_project_wiki_repository_replication`, enabled by default. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details. + +<sup>3</sup> Verification of files stored in object storage was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/8056) in GitLab 16.4 [with a feature flag](../../feature_flags.md) named `geo_object_storage_verification`, enabled by default. -- GitLab