Skip to content
代码片段 群组 项目
未验证 提交 6fcd9f37 编辑于 作者: Marcel Amirault's avatar Marcel Amirault
浏览文件

Fix tables and links in blueprint docs

上级 00ce2c5c
No related branches found
No related tags found
无相关合并请求
...@@ -14,7 +14,7 @@ of GitLab.com and contrasts it with the expected Cells architecture. ...@@ -14,7 +14,7 @@ of GitLab.com and contrasts it with the expected Cells architecture.
## 1. Before Cells - Monolithic architecture ## 1. Before Cells - Monolithic architecture
<img src="diagrams/deployment-before-cells.drawio.png" width="800"> <img src="diagrams/deployment-before-cells.drawio.png" width="800" alt="">
The diagram represents simplified GitLab.com deployment components before the introduction of a Cells architecture. The diagram represents simplified GitLab.com deployment components before the introduction of a Cells architecture.
This diagram intentionally omits some services that are not relevant for the architecture overview (Cloudflare, Consul, PgBouncers, ...). This diagram intentionally omits some services that are not relevant for the architecture overview (Cloudflare, Consul, PgBouncers, ...).
...@@ -41,7 +41,7 @@ The dependent services: ...@@ -41,7 +41,7 @@ The dependent services:
## 2. Development Cells - Adapting application to Cellular architecture ## 2. Development Cells - Adapting application to Cellular architecture
<img src="diagrams/deployment-development-cells.drawio.png" width="800"> <img src="diagrams/deployment-development-cells.drawio.png" width="800" alt="">
The purpose of **Development Cells** is to model a production-like architecture to test and validate the changes introduced. The purpose of **Development Cells** is to model a production-like architecture to test and validate the changes introduced.
This could be achieved with testing Cells on top of the [Reference Architectures](../../../../administration/reference_architectures/index.md). This could be achieved with testing Cells on top of the [Reference Architectures](../../../../administration/reference_architectures/index.md).
...@@ -57,7 +57,7 @@ The differences compared to [Before Cells](#1-before-cells---monolithic-architec ...@@ -57,7 +57,7 @@ The differences compared to [Before Cells](#1-before-cells---monolithic-architec
## 3. Initial Cells deployment - Transforming monolithic architecture to Cells architecture ## 3. Initial Cells deployment - Transforming monolithic architecture to Cells architecture
<img src="diagrams/deployment-initial-cells.drawio.png" width="800"> <img src="diagrams/deployment-initial-cells.drawio.png" width="800" alt="">
The differences compared to [Development Cells](#2-development-cells---adapting-application-to-cellular-architecture) are: The differences compared to [Development Cells](#2-development-cells---adapting-application-to-cellular-architecture) are:
...@@ -83,7 +83,7 @@ The differences compared to [Development Cells](#2-development-cells---adapting- ...@@ -83,7 +83,7 @@ The differences compared to [Development Cells](#2-development-cells---adapting-
## 4. Hybrid Cells deployment - Initial complete Cells architecture ## 4. Hybrid Cells deployment - Initial complete Cells architecture
<img src="diagrams/deployment-hybrid-cells.drawio.png" width="800"> <img src="diagrams/deployment-hybrid-cells.drawio.png" width="800" alt="">
The differences compared to [Initial Cells deployment](#3-initial-cells-deployment---transforming-monolithic-architecture-to-cells-architecture) are: The differences compared to [Initial Cells deployment](#3-initial-cells-deployment---transforming-monolithic-architecture-to-cells-architecture) are:
...@@ -95,7 +95,7 @@ The differences compared to [Initial Cells deployment](#3-initial-cells-deployme ...@@ -95,7 +95,7 @@ The differences compared to [Initial Cells deployment](#3-initial-cells-deployme
## 5. Target Cells - Fully isolated Cells architecture ## 5. Target Cells - Fully isolated Cells architecture
<img src="diagrams/deployment-target-cells.drawio.png" width="800"> <img src="diagrams/deployment-target-cells.drawio.png" width="800" alt="">
The differences compared to [Hybrid Cells deployment](#4-hybrid-cells-deployment---initial-complete-cells-architecture) are: The differences compared to [Hybrid Cells deployment](#4-hybrid-cells-deployment---initial-complete-cells-architecture) are:
...@@ -110,10 +110,10 @@ with scaling the service, its location (cluster-wide or Cell-local), and impact ...@@ -110,10 +110,10 @@ with scaling the service, its location (cluster-wide or Cell-local), and impact
### Cluster-wide services ### Cluster-wide services
| Service | Type | Uses | Description | | Service | Type | Uses | Description |
| ------------------------------ | ------------ | ------------------------------- | --------------------------------------------------------------------------------------------------- | |--------------------------------|--------------|---------------------------------|-------------|
| **Routing Service** | GitLab-built | Cluster-wide Data Provider | A general purpose routing service that can redirect requests from all GitLab SaaS domains to the Cell | | **Routing Service** | GitLab-built | Cluster-wide Data Provider | A general purpose routing service that can redirect requests from all GitLab SaaS domains to the Cell |
| **Cluster-wide Data Provider** | GitLab-built | PostgreSQL, Redis, Event Queue? | Provide user profile and routing information to all clustered services | | **Cluster-wide Data Provider** | GitLab-built | PostgreSQL, Redis, Event Queue? | Provide user profile and routing information to all clustered services |
As per the architecture, the above services are required to be run cluster-wide: As per the architecture, the above services are required to be run cluster-wide:
...@@ -121,36 +121,36 @@ As per the architecture, the above services are required to be run cluster-wide: ...@@ -121,36 +121,36 @@ As per the architecture, the above services are required to be run cluster-wide:
### Cell-local services ### Cell-local services
| Service | Type | Uses | Description | | Service | Type | Uses | Migrate from cluster-wide to Cell | Description |
| ------------------------------ | ------------ | ------------------------------- | --------------------------------------------------------------------------------------------------- | |----------------------------|-----------------|-------------------------------------|-----------------------------------|-------------|
| **Redis Cluster** | Managed service | Disk storage | No problem | Redis is used to hold user sessions, application caches, or Sidekiq queues. Most of that data is only applicable to Cells. | | **Redis Cluster** | Managed service | Disk storage | No problem | Redis is used to hold user sessions, application caches, or Sidekiq queues. Most of that data is only applicable to Cells. |
| **GitLab Runners Manager** | Managed service | API, uses Google Cloud VM Instances | No problem | Significant changes required to API and execution of CI jobs | | **GitLab Runners Manager** | Managed service | API, uses Google Cloud VM Instances | No problem | Significant changes required to API and execution of CI jobs |
As per the architecture, the above services are required to be run Cell-local: As per the architecture, the above services are required to be run Cell-local:
- The consumer data held by the Cell-local services needs to be migratable to another Cell. - The consumer data held by the Cell-local services needs to be migratable to another Cell.
- The compute generated by the service is substatial, and it is strongly desired to reduce impact of [single Cell failure](../goals.md#high-resilience-to-a-single-cell-failure). - The compute generated by the service is substantial, and it is strongly desired to reduce impact of [single Cell failure](../goals.md#high-resilience-to-a-single-cell-failure).
- It is complex to run the service cluster-wide from the Cells architecture perspective. - It is complex to run the service cluster-wide from the Cells architecture perspective.
### Hybrid Services ### Hybrid Services
| Service | | Uses | Migrate from cluster-wide to Cell | Description | | Service | Type | Uses | Migrate from cluster-wide to Cell | Description |
| ------------------- | --------------- | ------------------------------- | ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | |---------------------|-----------------|---------------------------------|-------------------------------------------------------------------------|-------------|
| **GitLab Pages** | GitLab-built | Routing Service, Rails API | No problem | Serving CI generated pages under `.gitlab.io` or custom domains | | **GitLab Pages** | GitLab-built | Routing Service, Rails API | No problem | Serving CI generated pages under `.gitlab.io` or custom domains |
| **GitLab Registry** | GitLab-built | Object Storage, PostgreSQL | Non-trivial data migration in case of split | Service to provide GitLab container registry | | **GitLab Registry** | GitLab-built | Object Storage, PostgreSQL | Non-trivial data migration in case of split | Service to provide GitLab container registry |
| **Gitaly Cluster** | GitLab-built | Disk storage, PostgreSQL | No problem: Built-in migration routines to balance Gitaly nodes | Gitaly holds Git repository data. Many Gitaly clusters can be configured in application. | | **Gitaly Cluster** | GitLab-built | Disk storage, PostgreSQL | No problem: Built-in migration routines to balance Gitaly nodes | Gitaly holds Git repository data. Many Gitaly clusters can be configured in application. |
| **Elasticsearch** | Managed service | Many nodes required by sharding | Time-consuming: Rebuild cluster from scratch | Search across all projects | | **Elasticsearch** | Managed service | Many nodes required by sharding | Time-consuming: Rebuild cluster from scratch | Search across all projects |
| **Object Storage** | Managed service | | Not straightforward: Rather hard to selectively migrate between buckets | Holds all user and CI uploaded files that is served by GitLab | | **Object Storage** | Managed service | | Not straightforward: Rather hard to selectively migrate between buckets | Holds all user and CI uploaded files that is served by GitLab |
As per the architecture, the above services are allowed to be run either cluster-wide or Cell-local: As per the architecture, the above services are allowed to be run either cluster-wide or Cell-local:
- The ability to run hybrid services cluster-wide might reduce the amount of work to migrate data between Cells due to some services being shared. - The ability to run hybrid services cluster-wide might reduce the amount of work to migrate data between Cells due to some services being shared.
- The hybrid services that are run cluster-wide might negatively impact Cell availability and resiliency due to increased impact caused by [single Cell failure](../goals.md#high-resilience-to-a-single-cell-failure). - The hybrid services that are run cluster-wide might negatively impact Cell availability and resiliency due to increased impact caused by [single Cell failure](../goals.md#high-resilience-to-a-single-cell-failure).
| Service | Type | Uses | Description | | Service | Type | Uses | Migrate from cluster-wide to Cell | Description |
| ------------------------------ | ------------ | ------------------------------- | --------------------------------------------------------------------------------------------------- | |--------------------|-----------------|---------------------------------|-------------------------------------------------------------------------|-------------|
| **Elasticsearch** | Managed service | Many nodes requires by sharding | Time-consuming: Rebuild cluster from scratch | Search across all projects | | **Elasticsearch** | Managed service | Many nodes requires by sharding | Time-consuming: Rebuild cluster from scratch | Search across all projects |
| **Object Storage** | Managed service | | Not straightforward: Rather hard to selectively migrate between buckets | Holds all user and CI uploaded files that is served by GitLab | | **Object Storage** | Managed service | | Not straightforward: Rather hard to selectively migrate between buckets | Holds all user and CI uploaded files that is served by GitLab |
As per the architecture, the above services are allowed to be run either cluster-wide or Cell-local: As per the architecture, the above services are allowed to be run either cluster-wide or Cell-local:
......
...@@ -292,7 +292,7 @@ Some of those benefits could also be seen as downsides and maybe not the right c ...@@ -292,7 +292,7 @@ Some of those benefits could also be seen as downsides and maybe not the right c
##### High level proposal ##### High level proposal
<img src="diagrams/sharding_proposal_2023-08.drawio.png" height="600"> <img src="diagrams/sharding_proposal_2023-08.drawio.png" height="600" alt="">
1. Zoekt nodes are started with 3 additional arguments: its own address, shard name, and GitLab URL. 1. Zoekt nodes are started with 3 additional arguments: its own address, shard name, and GitLab URL.
1. We'd like to keep shard name separate so that one will be able to migrate a shard to a different address. 1. We'd like to keep shard name separate so that one will be able to migrate a shard to a different address.
......
...@@ -64,7 +64,7 @@ Much of the proposed architecture already exists and is in operation for GitLab. ...@@ -64,7 +64,7 @@ Much of the proposed architecture already exists and is in operation for GitLab.
The following diagram outlines the architecture for GitLab Observability Backend and how clients, including the GitLab UI, will interact with it. The following diagram outlines the architecture for GitLab Observability Backend and how clients, including the GitLab UI, will interact with it.
<img src="./arch.png"> <img src="./arch.png" alt="">
### Key Components ### Key Components
......
...@@ -18,7 +18,7 @@ Read more about what an Organization is in [Organization](index.md). ...@@ -18,7 +18,7 @@ Read more about what an Organization is in [Organization](index.md).
## What? ## What?
<img src="diagrams/organization-isolation.drawio.png" width="800"> <img src="diagrams/organization-isolation.drawio.png" width="800" alt="">
All Cell-local data and functionality in GitLab (all data except the few All Cell-local data and functionality in GitLab (all data except the few
things that need to exist on all Cells in the cluster) must be isolated. things that need to exist on all Cells in the cluster) must be isolated.
...@@ -32,7 +32,7 @@ A few examples of things that would be disallowed by Organization Isolation are: ...@@ -32,7 +32,7 @@ A few examples of things that would be disallowed by Organization Isolation are:
## Why? ## Why?
<img src="diagrams/organization-isolation-broken.drawio.png" width="800"> <img src="diagrams/organization-isolation-broken.drawio.png" width="800" alt="">
[GitLab Cells](../cells/index.md) depend on using the Organization as the sharding key, which will allow us to shard data between different Cells. [GitLab Cells](../cells/index.md) depend on using the Organization as the sharding key, which will allow us to shard data between different Cells.
Initially, when we start rolling out Organizations, we will be working with a single Cell `Cell 1`. Initially, when we start rolling out Organizations, we will be working with a single Cell `Cell 1`.
......
...@@ -338,30 +338,30 @@ scope. ...@@ -338,30 +338,30 @@ scope.
### Stage 1 - Deprecations ### Stage 1 - Deprecations
| Component | Milestone | Changes | | Component | Milestone | Changes |
|------------------|----------:|---------| |----------------------------------|----------:|---------|
| GitLab Rails app | `15.6` | Deprecate `POST /api/v4/runners` endpoint for `17.0`. This hinges on a [proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/373774) to allow deprecating REST API endpoints for security reasons. | | GitLab Rails app | `15.6` | Deprecate `POST /api/v4/runners` endpoint for `17.0`. This hinges on a [proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/373774) to allow deprecating REST API endpoints for security reasons. |
| GitLab Runner | `15.6` | Add deprecation notice for `register` command for `17.0`. | | GitLab Runner | `15.6` | Add deprecation notice for `register` command for `17.0`. |
| GitLab Runner Helm Chart | `15.6` | Add deprecation notice for `runnerRegistrationToken` command for `17.0`. | | GitLab Runner Helm Chart | `15.6` | Add deprecation notice for `runnerRegistrationToken` command for `17.0`. |
| GitLab Runner Operator | `15.6` | Add deprecation notice for `runner-registration-token` command for `17.0`. | | GitLab Runner Operator | `15.6` | Add deprecation notice for `runner-registration-token` command for `17.0`. |
| GitLab Runner / GitLab Rails app | `15.7` | Add deprecation notice for registration token reset for `17.0`. | | GitLab Runner / GitLab Rails app | `15.7` | Add deprecation notice for registration token reset for `17.0`. |
### Stage 2 - Prepare `gitlab-runner` for `system_id` ### Stage 2 - Prepare `gitlab-runner` for `system_id`
| Component | Milestone | Changes | | Component | Milestone | Changes |
|------------------|----------:|---------| |---------------|----------:|---------|
| GitLab Runner | `15.7` | Ensure a sidecar TOML file exists with a `system_id` value.<br/>Log new system ID values with `INFO` level as they get assigned. | | GitLab Runner | `15.7` | Ensure a sidecar TOML file exists with a `system_id` value.<br/>Log new system ID values with `INFO` level as they get assigned. |
| GitLab Runner | `15.9` | Log unique system ID in the build logs. | | GitLab Runner | `15.9` | Log unique system ID in the build logs. |
| GitLab Runner | `15.9` | Label Prometheus metrics with unique system ID. | | GitLab Runner | `15.9` | Label Prometheus metrics with unique system ID. |
| GitLab Runner | `15.8` | Prepare `register` command to fail if runner server-side configuration options are passed together with a new `glrt-` token. | | GitLab Runner | `15.8` | Prepare `register` command to fail if runner server-side configuration options are passed together with a new `glrt-` token. |
### Stage 2a - Prepare GitLab Runner Helm Chart and GitLab Runner Operator ### Stage 2a - Prepare GitLab Runner Helm Chart and GitLab Runner Operator
| Component | Milestone | Issue | Changes | | Component | Milestone | Changes |
|------------------|----------:|-------|---------| |--------------------------|----------:|---------|
| GitLab Runner Helm Chart | `%15.10` | Update the Runner Helm Chart to support registration with the authentication token. | | GitLab Runner Helm Chart | `%15.10` | Update the Runner Helm Chart to support registration with the authentication token. |
| GitLab Runner Operator | `%15.10` | Update the Runner Operator to support registration with the authentication token. | | GitLab Runner Operator | `%15.10` | Update the Runner Operator to support registration with the authentication token. |
| GitLab Runner Helm Chart | `%16.2` | Add `systemID` to Runner Helm Chart. | | GitLab Runner Helm Chart | `%16.2` | Add `systemID` to Runner Helm Chart. |
### Stage 3 - Database changes ### Stage 3 - Database changes
......
0% 加载中 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册