diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index df91b562a3ce8b3e7da5785f4a87fc125b996274..57cbf387115cbecf33cdba91530e0a34842e95fc 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -134,7 +134,7 @@ job: ## Inherit global configuration, but override specific settings per job You can override cache settings without overwriting the global cache by using -[anchors](../yaml/yaml_specific_features.md#anchors). For example, if you want to override the +[anchors](../yaml/yaml_optimization.md#anchors). For example, if you want to override the `policy` for one job: ```yaml diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 2788f2f3bae00ab66a477adf330c4f369b80e86d..7c1845775426fcd637c73903fe72ab8e20171dcb 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -177,7 +177,7 @@ file: You can use hidden jobs that start with `.` as templates for reusable configuration with: - The [`extends` keyword](../yaml/index.md#extends). -- [YAML anchors](../yaml/yaml_specific_features.md#anchors). +- [YAML anchors](../yaml/yaml_optimization.md#anchors). ## Control the inheritance of default keywords and global variables diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index c0d3b17604b8605d5c4743629613ab819bc933ac..0f92ae5ca49ec19468b09219c163e39d68760804 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -294,7 +294,7 @@ You can use the `$` character for both variables and paths. For example, if the > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322992) in GitLab 14.3. -Use [`!reference` tags](../yaml/yaml_specific_features.md#reference-tags) to reuse rules in different +Use [`!reference` tags](../yaml/yaml_optimization.md#reference-tags) to reuse rules in different jobs. You can combine `!reference` rules with regular job-defined rules: ```yaml diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index b84f38055acf7ad13bc35d9467359e32ba4b6944..5be016aff404e8a52ee9a88a7a4f1a32d83c2707 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -82,8 +82,8 @@ where: - Configuration imported with [`include`](../yaml/index.md#include) is copied into the view. - Jobs that use [`extends`](../yaml/index.md#extends) display with the - [extended configuration merged into the job](../yaml/index.md#merge-details). -- YAML anchors are [replaced with the linked configuration](../yaml/yaml_specific_features.md#anchors). + [extended configuration merged into the job](../yaml/yaml_optimization.md#merge-details). +- YAML anchors are [replaced with the linked configuration](../yaml/yaml_optimization.md#anchors). ## Commit changes to CI configuration diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 2d0a24ea60be094d5c5d94c680cc8a198ab16f36..8ad102c4b7c337f7e20d23f66e9e4e9b0064e3cc 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -624,7 +624,7 @@ All jobs except [trigger jobs](#trigger) require a `script` keyword. - Single line commands. - Long commands [split over multiple lines](script.md#split-long-commands). -- [YAML anchors](yaml_specific_features.md#yaml-anchors-for-scripts). +- [YAML anchors](yaml_optimization.md#yaml-anchors-for-scripts). **Example of `script`:** @@ -662,7 +662,7 @@ Use `before_script` to define an array of commands that should run before each j - Single line commands. - Long commands [split over multiple lines](script.md#split-long-commands). -- [YAML anchors](yaml_specific_features.md#yaml-anchors-for-scripts). +- [YAML anchors](yaml_optimization.md#yaml-anchors-for-scripts). **Example of `before_script`:** @@ -700,7 +700,7 @@ Use `after_script` to define an array of commands that run after each job, inclu - Single line commands. - Long commands [split over multiple lines](script.md#split-long-commands). -- [YAML anchors](yaml_specific_features.md#yaml-anchors-for-scripts). +- [YAML anchors](yaml_optimization.md#yaml-anchors-for-scripts). **Example of `after_script`:** @@ -861,16 +861,17 @@ job2: ### `extends` -Use `extends` to reuse configuration sections. It's an alternative to [YAML anchors](yaml_specific_features.md#anchors) -and is a little more flexible and readable. You can use `extends` to reuse configuration -from [included configuration files](#use-extends-and-include-together). +Use `extends` to reuse configuration sections. It's an alternative to [YAML anchors](yaml_optimization.md#anchors) +and is a little more flexible and readable. -In the following example, the `rspec` job uses the configuration from the `.tests` template job. -GitLab: +**Keyword type**: Job keyword. You can use it only as part of a job. -- Performs a reverse deep merge based on the keys. -- Merges the `.tests` content with the `rspec` job. -- Doesn't merge the values of the keys. +**Possible inputs:** + +- The name of another job in the pipeline. +- A list (array) of names of other jobs in the pipeline. + +**Example of `extends`:** ```yaml .tests: @@ -888,6 +889,13 @@ rspec: - $RSPEC ``` +In this example, the `rspec` job uses the configuration from the `.tests` template job. +When creating the pipeline, GitLab: + +- Performs a reverse deep merge based on the keys. +- Merges the `.tests` content with the `rspec` job. +- Doesn't merge the values of the keys. + The result is this `rspec` job: ```yaml @@ -901,182 +909,18 @@ rspec: - $RSPEC ``` -`.tests` in this example is a [hidden job](../jobs/index.md#hide-jobs), but it's -possible to extend configuration from regular jobs as well. - -`extends` supports multi-level inheritance. You should avoid using more than three levels, -but you can use as many as eleven. The following example has two levels of inheritance: - -```yaml -.tests: - rules: - - if: $CI_PIPELINE_SOURCE == "push" - -.rspec: - extends: .tests - script: rake rspec - -rspec 1: - variables: - RSPEC_SUITE: '1' - extends: .rspec - -rspec 2: - variables: - RSPEC_SUITE: '2' - extends: .rspec - -spinach: - extends: .tests - script: rake spinach -``` - -In GitLab 12.0 and later, it's also possible to use multiple parents for -`extends`. - -#### Merge details - -You can use `extends` to merge hashes but not arrays. -The algorithm used for merge is "closest scope wins," so -keys from the last member always override anything defined on other -levels. For example: - -```yaml -.only-important: - variables: - URL: "http://my-url.internal" - IMPORTANT_VAR: "the details" - rules: - - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH - - if: $CI_COMMIT_BRANCH == "stable" - tags: - - production - script: - - echo "Hello world!" - -.in-docker: - variables: - URL: "http://docker-url.internal" - tags: - - docker - image: alpine - -rspec: - variables: - GITLAB: "is-awesome" - extends: - - .only-important - - .in-docker - script: - - rake rspec -``` - -The result is this `rspec` job: - -```yaml -rspec: - variables: - URL: "http://docker-url.internal" - IMPORTANT_VAR: "the details" - GITLAB: "is-awesome" - rules: - - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH - - if: $CI_COMMIT_BRANCH == "stable" - tags: - - docker - image: alpine - script: - - rake rspec -``` - -In this example: - -- The `variables` sections merge, but `URL: "http://docker-url.internal"` overwrites `URL: "http://my-url.internal"`. -- `tags: ['docker']` overwrites `tags: ['production']`. -- `script` does not merge, but `script: ['rake rspec']` overwrites - `script: ['echo "Hello world!"']`. You can use [YAML anchors](yaml_specific_features.md#anchors) to merge arrays. - -##### Exclude a key from `extends` - -To exclude a key from the extended content, you must assign it to `null`, for example: - -```yaml -.base: - script: test - variables: - VAR1: base var 1 - -test1: - extends: .base - variables: - VAR1: test1 var 1 - VAR2: test2 var 2 - -test2: - extends: .base - variables: - VAR2: test2 var 2 - -test3: - extends: .base - variables: {} - -test4: - extends: .base - variables: null -``` - -Merged configuration: - -```yaml -test1: - script: test - variables: - VAR1: test1 var 1 - VAR2: test2 var 2 - -test2: - script: test - variables: - VAR1: base var 1 - VAR2: test2 var 2 - -test3: - script: test - variables: - VAR1: base var 1 - -test4: - script: test - variables: null -``` - -#### Use `extends` and `include` together - -To reuse configuration from different configuration files, -combine `extends` and [`include`](#include). - -In the following example, a `script` is defined in the `included.yml` file. -Then, in the `.gitlab-ci.yml` file, `extends` refers -to the contents of the `script`: - -- `included.yml`: - - ```yaml - .template: - script: - - echo Hello! - ``` +**Additional details:** -- `.gitlab-ci.yml`: +- In GitLab 12.0 and later, you can use multiple parents for `extends`. +- The `extends` keyword supports up to eleven levels of inheritance, but you should + avoid using more than three levels. +- In the example above, `.tests` is a [hidden job](../jobs/index.md#hide-jobs), + but you can extend configuration from regular jobs as well. - ```yaml - include: included.yml +**Related topics:** - useTemplate: - image: alpine - extends: .template - ``` +- [Reuse configuration sections by using `extends`](yaml_optimization.md#use-extends-to-reuse-configuration-sections). +- Use `extends` to reuse configuration from [included configuration files](yaml_optimization.md#use-extends-and-include-together). ### `rules` @@ -1116,7 +960,7 @@ The job is not added to the pipeline: - If no rules match. - If a rule matches and has `when: never`. -You can use [`!reference` tags](yaml_specific_features.md#reference-tags) to [reuse `rules` configuration](../jobs/job_control.md#reuse-rules-in-different-jobs) +You can use [`!reference` tags](yaml_optimization.md#reference-tags) to [reuse `rules` configuration](../jobs/job_control.md#reuse-rules-in-different-jobs) in different jobs. #### `rules:if` @@ -2162,7 +2006,7 @@ Also in the example, `GIT_STRATEGY` is set to `none`. If the the runner won't try to check out the code after the branch is deleted. The example also overwrites global variables. If your `stop` `environment` job depends -on global variables, use [anchor variables](yaml_specific_features.md#yaml-anchors-for-variables) when you set the `GIT_STRATEGY` +on global variables, use [anchor variables](yaml_optimization.md#yaml-anchors-for-variables) when you set the `GIT_STRATEGY` to change the job without overriding the global variables. The `stop_review_app` job is **required** to have the following keywords defined: @@ -4186,7 +4030,7 @@ deploy_review_job: **Related topics**: -- You can use [YAML anchors for variables](yaml_specific_features.md#yaml-anchors-for-variables). +- You can use [YAML anchors for variables](yaml_optimization.md#yaml-anchors-for-variables). - [Predefined variables](../variables/predefined_variables.md) are variables the runner automatically creates and makes available in the job. - You can [configure runner behavior with variables](../runners/configure_runners.md#configure-runner-behavior-with-variables). diff --git a/doc/ci/yaml/yaml_specific_features.md b/doc/ci/yaml/yaml_optimization.md similarity index 61% rename from doc/ci/yaml/yaml_specific_features.md rename to doc/ci/yaml/yaml_optimization.md index cc49d3e9ceba3129b384319483da186fc069da0d..503ba9bbd802e95ac030ba3d3ca27f3e395a52c6 100644 --- a/doc/ci/yaml/yaml_specific_features.md +++ b/doc/ci/yaml/yaml_optimization.md @@ -5,18 +5,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# YAML-specific features +# Optimize GitLab CI/CD configuration files **(FREE)** -In your `.gitlab-ci.yml` file, you can use YAML-specific features like anchors (`&`), aliases (`*`), -and map merging (`<<`). Use these features to reduce the complexity -of the code in the `.gitlab-ci.yml` file. +You can reduce complexity and duplicated configuration in your GitLab CI/CD configuration +files by using: -Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/). - -In most cases, the [`extends` keyword](index.md#extends) is more user friendly and you should -use it when possible. - -You can use YAML anchors to merge YAML arrays. +- YAML-specific features like [anchors (`&`)](#anchors), aliases (`*`), and map merging (`<<`). + Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/). +- The [`extends` keyword](#use-extends-to-reuse-configuration-sections), + which is more flexible and readable. We recommend you use `extends` where possible. ## Anchors @@ -27,10 +24,12 @@ Use anchors to duplicate or inherit properties. Use anchors with [hidden jobs](. to provide templates for your jobs. When there are duplicate keys, GitLab performs a reverse deep merge based on the keys. +You can use YAML anchors to merge YAML arrays. + You can't use YAML anchors across multiple files when using the [`include`](index.md#include) keyword. Anchors are only valid in the file they were defined in. To reuse configuration from different YAML files, use [`!reference` tags](#reference-tags) or the -[`extends` keyword](index.md#extends). +[`extends` keyword](#use-extends-to-reuse-configuration-sections). The following example uses anchors and map merging. It creates two jobs, `test1` and `test2`, that inherit the `.job_template` configuration, each @@ -214,6 +213,183 @@ job_no_git_strategy: script: echo $SAMPLE_VARIABLE ``` +## Use `extends` to reuse configuration sections + +You can use the [`extends` keyword](index.md#extends) to reuse configuration in +multiple jobs. It is similar to [YAML anchors](#anchors), but simpler and you can +[use `extends` with `includes`](#use-extends-and-include-together). + +`extends` supports multi-level inheritance. You should avoid using more than three levels, +but you can use as many as eleven. The following example has two levels of inheritance: + +```yaml +.tests: + rules: + - if: $CI_PIPELINE_SOURCE == "push" + +.rspec: + extends: .tests + script: rake rspec + +rspec 1: + variables: + RSPEC_SUITE: '1' + extends: .rspec + +rspec 2: + variables: + RSPEC_SUITE: '2' + extends: .rspec + +spinach: + extends: .tests + script: rake spinach +``` + +### Exclude a key from `extends` + +To exclude a key from the extended content, you must assign it to `null`, for example: + +```yaml +.base: + script: test + variables: + VAR1: base var 1 + +test1: + extends: .base + variables: + VAR1: test1 var 1 + VAR2: test2 var 2 + +test2: + extends: .base + variables: + VAR2: test2 var 2 + +test3: + extends: .base + variables: {} + +test4: + extends: .base + variables: null +``` + +Merged configuration: + +```yaml +test1: + script: test + variables: + VAR1: test1 var 1 + VAR2: test2 var 2 + +test2: + script: test + variables: + VAR1: base var 1 + VAR2: test2 var 2 + +test3: + script: test + variables: + VAR1: base var 1 + +test4: + script: test + variables: null +``` + +### Use `extends` and `include` together + +To reuse configuration from different configuration files, +combine `extends` and [`include`](index.md#include). + +In the following example, a `script` is defined in the `included.yml` file. +Then, in the `.gitlab-ci.yml` file, `extends` refers +to the contents of the `script`: + +- `included.yml`: + + ```yaml + .template: + script: + - echo Hello! + ``` + +- `.gitlab-ci.yml`: + + ```yaml + include: included.yml + + useTemplate: + image: alpine + extends: .template + ``` + +### Merge details + +You can use `extends` to merge hashes but not arrays. +The algorithm used for merge is "closest scope wins," so +keys from the last member always override anything defined on other +levels. For example: + +```yaml +.only-important: + variables: + URL: "http://my-url.internal" + IMPORTANT_VAR: "the details" + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + - if: $CI_COMMIT_BRANCH == "stable" + tags: + - production + script: + - echo "Hello world!" + +.in-docker: + variables: + URL: "http://docker-url.internal" + tags: + - docker + image: alpine + +rspec: + variables: + GITLAB: "is-awesome" + extends: + - .only-important + - .in-docker + script: + - rake rspec +``` + +The result is this `rspec` job: + +```yaml +rspec: + variables: + URL: "http://docker-url.internal" + IMPORTANT_VAR: "the details" + GITLAB: "is-awesome" + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + - if: $CI_COMMIT_BRANCH == "stable" + tags: + - docker + image: alpine + script: + - rake rspec +``` + +In this example: + +- The `variables` sections merge, but `URL: "http://docker-url.internal"` overwrites `URL: "http://my-url.internal"`. +- `tags: ['docker']` overwrites `tags: ['production']`. +- `script` does not merge, but `script: ['rake rspec']` overwrites + `script: ['echo "Hello world!"']`. You can use [YAML anchors](yaml_optimization.md#anchors) to merge arrays. + ## `!reference` tags > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/266173) in GitLab 13.9. diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index 802d3838fe17b14ca5cd856b913f65b761bf065d..98f63c7cbf0336415b5d0e0ae47ef071e4950e8e 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -698,7 +698,7 @@ then included in individual jobs via [`extends`](../ci/yaml/index.md#extends). The `rules` definitions are composed of `if:` conditions and `changes:` patterns, which are also defined in [`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml) -and included in `rules` definitions via [YAML anchors](../ci/yaml/yaml_specific_features.md#anchors) +and included in `rules` definitions via [YAML anchors](../ci/yaml/yaml_optimization.md#anchors) #### `if:` conditions