diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index a1c02870217322f89f0b76ecf725aa3039fc3882..25dacc9c43779862f9e1182aaed12f4f834c426d 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w description: Control the job concurrency in GitLab CI/CD --- -# Resource Group **(FREE)** +# Resource group **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7. @@ -13,7 +13,7 @@ By default, pipelines in GitLab CI/CD run in parallel. The parallelization is an the feedback loop in merge requests, however, there are some situations that you may want to limit the concurrency on deployment jobs to run them one by one. -Resource Group allows you to strategically control +Use resource groups to strategically control the concurrency of the jobs for optimizing your continuous deployments workflow with safety. ## Add a resource group @@ -139,9 +139,59 @@ Depending on the process mode of the resource group: - `deploy-1`, `deploy-2`, and `deploy-3` do not run in parallel. - `deploy-3` runs first, `deploy-2` runs second and `deploy-1` runs last. -## Pipeline-level concurrency control with Cross-Project/Parent-Child pipelines +## Pipeline-level concurrency control with cross-project/parent-child pipelines -See the how to [control the pipeline concurrency in cross-project pipelines](../yaml/index.md#pipeline-level-concurrency-control-with-cross-projectparent-child-pipelines). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39057) in GitLab 13.9. + +You can define `resource_group` for downstream pipelines that are sensitive to concurrent +executions. The [`trigger` keyword](../yaml/index.md#trigger) can trigger downstream pipelines and the +[`resource_group` keyword](../yaml/index.md#resource_group) can co-exist with it. `resource_group` is useful to control the +concurrency of deployment pipelines, while other jobs can continue to run concurrently. + +The following example has two pipeline configurations in a project. When a pipeline starts running, +non-sensitive jobs are executed first and aren't affected by concurrent executions in other +pipelines. However, GitLab ensures that there are no other deployment pipelines running before +triggering a deployment (child) pipeline. If other deployment pipelines are running, GitLab waits +until those pipelines finish before running another one. + +```yaml +# .gitlab-ci.yml (parent pipeline) + +build: + stage: build + script: echo "Building..." + +test: + stage: test + script: echo "Testing..." + +deploy: + stage: deploy + trigger: + include: deploy.gitlab-ci.yml + strategy: depend + resource_group: AWS-production +``` + +```yaml +# deploy.gitlab-ci.yml (child pipeline) + +stages: + - provision + - deploy + +provision: + stage: provision + script: echo "Provisioning..." + +deployment: + stage: deploy + script: echo "Deploying..." +``` + +You must define [`strategy: depend`](../yaml/index.md#triggerstrategy) +with the `trigger` keyword. This ensures that the lock isn't released until the downstream pipeline +finishes. ## API diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 745d2776ead80cf5aa431fbdbd70413edcec32e0..01e61679c0d139510840aa741f8c9ff0935e1fd8 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -3941,92 +3941,37 @@ In this example, a new pipeline causes a running pipeline to be: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7. -Sometimes running multiple jobs at the same time in an environment -can lead to errors during the deployment. +Use `resource_group` to create a [resource group](../resource_groups/index.md) that +ensures a job is mutually exclusive across different pipelines for the same project. -To avoid these errors, use the `resource_group` attribute to make sure that -the runner doesn't run certain jobs concurrently. Resource groups behave similar -to semaphores in other programming languages. +For example, if multiple jobs that belong to the same resource group are queued simultaneously, +only one of the jobs starts. The other jobs wait until the `resource_group` is free. -When the `resource_group` keyword is defined for a job in the `.gitlab-ci.yml` file, -job executions are mutually exclusive across different pipelines for the same project. -If multiple jobs belonging to the same resource group are enqueued simultaneously, -only one of the jobs is picked by the runner. The other jobs wait until the -`resource_group` is free. - -For example: - -```yaml -deploy-to-production: - script: deploy - resource_group: production -``` - -In this case, two `deploy-to-production` jobs in two separate pipelines can never run at the same time. As a result, -you can ensure that concurrent deployments never happen to the production environment. +Resource groups behave similar to semaphores in other programming languages. You can define multiple resource groups per environment. For example, -when deploying to physical devices, you may have multiple physical devices. Each device -can be deployed to, but there can be only one deployment per device at any given time. +when deploying to physical devices, you might have multiple physical devices. Each device +can be deployed to, but only one deployment can occur per device at any given time. -The `resource_group` value can only contain letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces. -It can't start or end with `/`. - -For more information, see [Resource Group documentation](../resource_groups/index.md). - -#### Pipeline-level concurrency control with Cross-Project/Parent-Child pipelines - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39057) in GitLab 13.9. +**Keyword type**: Job keyword. You can use it only as part of a job. -You can define `resource_group` for downstream pipelines that are sensitive to concurrent -executions. The [`trigger` keyword](#trigger) can trigger downstream pipelines. The -[`resource_group` keyword](#resource_group) can co-exist with it. This is useful to control the -concurrency for deployment pipelines, while running non-sensitive jobs concurrently. +**Possible inputs**: Only letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces. +It can't start or end with `/`. -The following example has two pipeline configurations in a project. When a pipeline starts running, -non-sensitive jobs are executed first and aren't affected by concurrent executions in other -pipelines. However, GitLab ensures that there are no other deployment pipelines running before -triggering a deployment (child) pipeline. If other deployment pipelines are running, GitLab waits -until those pipelines finish before running another one. +**Example of `resource_group`**: ```yaml -# .gitlab-ci.yml (parent pipeline) - -build: - stage: build - script: echo "Building..." - -test: - stage: test - script: echo "Testing..." - -deploy: - stage: deploy - trigger: - include: deploy.gitlab-ci.yml - strategy: depend - resource_group: AWS-production +deploy-to-production: + script: deploy + resource_group: production ``` -```yaml -# deploy.gitlab-ci.yml (child pipeline) - -stages: - - provision - - deploy - -provision: - stage: provision - script: echo "Provisioning..." +In this example, two `deploy-to-production` jobs in two separate pipelines can never run at the same time. As a result, +you can ensure that concurrent deployments never happen to the production environment. -deployment: - stage: deploy - script: echo "Deploying..." -``` +**Related topics**: -You must define [`strategy: depend`](#triggerstrategy) -with the `trigger` keyword. This ensures that the lock isn't released until the downstream pipeline -finishes. +- [Pipeline-level concurrency control with cross-project/parent-child pipelines](../resource_groups/index.md#pipeline-level-concurrency-control-with-cross-projectparent-child-pipelines). ### `release`