Merge branch 'ci-components-directory-structure' into 'master'

Revisit directory structure of components project

See merge request https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106677



Merged-by: Grzegorz Bizon <grzegorz@gitlab.com>
Reviewed-by: Kamil Trzciński <ayufan@ayufan.eu>
Reviewed-by: Grzegorz Bizon <grzegorz@gitlab.com>
Reviewed-by: Fabio Pitino <fpitino@gitlab.com>
Co-authored-by: Fabio Pitino <fpitino@gitlab.com>

doc/architecture/blueprints/ci_pipeline_components/index.md

@@ -143,30 +143,35 @@ For example: `gitlab-org/dast@1.0`.
 
 ### The component path
 
-A component path must contain at least the metadata YAML and optionally a related `README.md` documentation file.
+A component path must contain at least the component YAML and optionally a related `README.md` documentation file.
 
 The component path can be:
 
-- A path to a project: `gitlab-org/dast`. In this case the 2 files are defined in the root directory of the repository.
-- A path to a project subdirectory: `gitlab-org/dast/api-scan`. In this case the 2 files are defined in the `api-scan` directory.
-- A path to a local directory: `/path/to/component`. This path must contain the metadata YAML that defines the component.
+- A path to a project: `gitlab-org/dast`. The default component is processed.
+- A path to an explicit component: `gitlab-org/dast/api-scan`. In this case the explicit `api-scan` component is processed.
+- A path to a local directory: `/path/to/component`. This path must contain the component YAML that defines the component.
   The path must start with `/` to indicate a full path in the repository.
 
-The metadata YAML file follows the filename convention `gitlab-<component-type>.yml` where component type is one of:
+The component YAML file follows the filename convention `<type>.yml` where component type is one of:
 
 | Component type | Context |
 | -------------- | ------- |
 | `template`     | For components used under `include:` keyword |
 | `step`         | For components used under `steps:` keyword  |
-| `workflow`     | For components used under `trigger:` keyword |
 
 Based on the context where the component is used we fetch the correct YAML file.
-For example, if we are including a component `gitlab-org/dast@1.0` we expect a YAML file named `gitlab-template.yml` in the
-top level directory of `gitlab-org/dast` repository.
+For example:
 
-A `gitlab-<component-type>.yml` file:
+- if we are including a component `gitlab-org/dast@1.0` we expect a YAML file named `template.yml` in the
+  root directory of `gitlab-org/dast` repository.
+- if we are including a component `gitlab-org/dast/api-scan@1.0` we expect a YAML file named `template.yml` inside a
+  directory `api-scan` of `gitlab-org/dast` repository.
+- if we are using a step component `gitlab-org/dast/api-scan@1.0` we expect a YAML file named `step.yml` inside a
+  directory `api-scan` of `gitlab-org/dast` repository.
 
-- Must have a **name** to be referenced to and **description** for extra details.
+A component YAML file:
+
+- Must have a **name** to be referenced to.
 - Must specify its **type** in the filename, which defines how it can be used (raw configuration to be `include`d, child pipeline workflow, job step).
 - Must define its **content** based on the type.
 - Must specify **input parameters** that it accepts. Components should depend on input parameters for dynamic values and not environment variables.
@@ -187,8 +192,8 @@ spec:
 content: { ... }
 ```
 
-Components that are released in the catalog must have a `README.md` file in the same directory as the
-metadata YAML file. The `README.md` represents the documentation for the specific component, hence it's recommended
+Components that are released in the catalog must have a `README.md` file at the root directory of the repository.
+The `README.md` represents the documentation for the specific component, hence it's recommended
 even when not releasing versions in the catalog.
 
 ### The component version
@@ -230,30 +235,28 @@ The following directory structure would support 1 component per project:
 
 ```plaintext
 .
-├── gitlab-<type>.yml
+├── template.yml
 ├── README.md
 └── .gitlab-ci.yml
 ```
 
 The `.gitlab-ci.yml` is recommended for the project to ensure changes are verified accordingly.
 
-The component is now identified by the path `myorg/rails-rspec`. In other words, this means that
-the `gitlab-<type>.yml` and `README.md` are located in the root directory of the repository.
+The component is now identified by the path `myorg/rails-rspec` and we expect a `template.yml` file
+and `README.md` located in the root directory of the repository.
 
 The following directory structure would support multiple components per project:
 
 ```plaintext
 .
 ├── .gitlab-ci.yml
+├── README.md
 ├── unit/
-│   ├── gitlab-workflow.yml
-│   └── README.md
+│   └── template.yml
 ├── integration/
-│   ├── gitlab-workflow.yml
-│   └── README.md
+│   └── template.yml
 └── feature/
-    ├── gitlab-workflow.yml
-    └── README.md
+    └── template.yml
 ```
 
 In this example we are defining multiple test profiles that are executed with RSpec.
@@ -266,18 +269,20 @@ This directory structure could also support both strategies:
 
 ```plaintext
 .
-├── gitlab-template.yml # myorg/rails-rspec
+├── template.yml       # myorg/rails-rspec
 ├── README.md
+├── LICENSE
 ├── .gitlab-ci.yml
 ├── unit/
-│   ├── gitlab-workflow.yml # myorg/rails-rspec/unit
-│   └── README.md
+│   └── template.yml   # myorg/rails-rspec/unit
 ├── integration/
-│   ├── gitlab-workflow.yml # myorg/rails-rspec/integration
-│   └── README.md
-└── feature/
-    ├── gitlab-workflow.yml # myorg/rails-rspec/feature
-    └── README.md
+│   └── template.yml   # myorg/rails-rspec/integration
+├── feature/
+│   └── template.yml   # myorg/rails-rspec/feature
+└── report/
+    ├── step.yml       # myorg/rails-rspec/report
+    ├── Dockerfile
+    └── ... other files
 ```
 
 With the above structure we could have a top-level component that can be used as the
@@ -285,7 +290,7 @@ default component. For example, `myorg/rails-rspec` could run all the test profi
 However, more specific test profiles could be used separately (for example `myorg/rails-rspec/integration`).
 
 NOTE:
-Any nesting more than 1 level is initially not permitted.
+Nesting of components is not permitted.
 This limitation encourages cohesion at project level and keeps complexity low.
 
 ## Input parameters `spec:inputs:` parameters