diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 07ba3d8f91632cea3313b51a7c4fafd94efdc569..31cb6bc9946ba95b42ca0c2f7201a9ef7a98b902 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -14,9 +14,13 @@ repository into your project and keep your commits separate. ## Configure the `.gitmodules` file When you use Git submodules, your project should have a file named `.gitmodules`. -You might need to modify it to work in a GitLab CI/CD job. +You have multiple options to configure it to work in a GitLab CI/CD job. -For example, your `.gitmodules` configuration might look like the following if: +### Using absolute URLs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3198) in GitLab Runner 15.11. + +For example, your generated `.gitmodules` configuration might look like the following if: - Your project is located at `https://gitlab.com/secret-group/my-project`. - Your project depends on `https://gitlab.com/group/project`, which you want @@ -26,19 +30,43 @@ For example, your `.gitmodules` configuration might look like the following if: ```ini [submodule "project"] path = project - url = ../../group/project.git + url = git@gitlab.com:secret-group/project.git +``` + +In this case, use the [`GIT_SUBMODULE_FORCE_HTTPS`](runners/configure_runners.md#rewrite-submodule-urls-to-https) variable +to instruct GitLab Runner to convert the URL to HTTPS before it clones the submodules. + +Alternatively, if you also use HTTPS locally, you can configure an HTTPS URL: + +```ini +[submodule "project"] + path = project + url = https://gitlab.com/secret-group/project.git ``` -When your submodule is on the same GitLab server, you should use relative URLs in -your `.gitmodules` file. Then you can clone with HTTPS in all your CI/CD jobs. You -can also use SSH for all your local checkouts. +You do not need to configure additional variables in this case, but you need to use a +[personal access token](../user/profile/personal_access_tokens.md) to clone it locally. + +### Using relative URLs + +WARNING: +If you use relative URLs, submodules may resolve incorrectly in forking workflows. +Use absolute URLs instead if you expect your project to have forks. + +When your submodule is on the same GitLab server, you can also use relative URLs in +your `.gitmodules` file: + +```ini +[submodule "project"] + path = project + url = ../../project.git +``` The above configuration instructs Git to automatically deduce the URL to -use when cloning sources. Git uses the same configuration for both HTTPS and SSH. -GitLab CI/CD uses HTTPS for cloning your sources, and you can continue to use SSH -to clone locally. +use when cloning sources. You can clone with HTTPS in all your CI/CD jobs, and you +can continue to use SSH to clone locally. -For submodules not located on the same GitLab server, use the full URL: +For submodules not located on the same GitLab server, always use the full URL: ```ini [submodule "project-x"] @@ -50,8 +78,6 @@ For submodules not located on the same GitLab server, use the full URL: To make submodules work correctly in CI/CD jobs: -1. Make sure you use [relative URLs](#configure-the-gitmodules-file) - for submodules located in the same GitLab server. 1. You can set the `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` to tell the runner to [fetch your submodules before the job](runners/configure_runners.md#git-submodule-strategy): @@ -60,6 +86,9 @@ To make submodules work correctly in CI/CD jobs: GIT_SUBMODULE_STRATEGY: recursive ``` +1. For submodules located on the same GitLab server and configured with a Git or SSH URL, make sure + you set the [`GIT_SUBMODULE_FORCE_HTTPS`](runners/configure_runners.md#rewrite-submodule-urls-to-https) variable. + 1. Use `GIT_SUBMODULE_DEPTH` to configure the cloning depth of submodules independently of the [`GIT_DEPTH`](runners/configure_runners.md#shallow-cloning) variable: ```yaml diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index e32ad4ec7de8838f6c00f9895d4abaa257197cd0..cf4b95f511d0b49dd23dd3320ea8b67ec9294c5d 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -334,6 +334,7 @@ globally or for individual jobs: - [`GIT_CLEAN_FLAGS`](#git-clean-flags) - [`GIT_FETCH_EXTRA_FLAGS`](#git-fetch-extra-flags) - [`GIT_SUBMODULE_UPDATE_FLAGS`](#git-submodule-update-flags) +- [`GIT_SUBMODULE_FORCE_HTTPS`](#rewrite-submodule-urls-to-https) - [`GIT_DEPTH`](#shallow-cloning) (shallow cloning) - [`GIT_SUBMODULE_DEPTH`](#git-submodule-depth) - [`GIT_CLONE_PATH`](#custom-build-directories) (custom build directories) @@ -590,6 +591,23 @@ You should be aware of the implications for the security, stability, and reprodu your builds when using the `--remote` flag. In most cases, it is better to explicitly track submodule commits as designed, and update them using an auto-remediation/dependency bot. +### Rewrite submodule URLs to HTTPS + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3198) in GitLab Runner 15.11. + +Use the `GIT_SUBMODULE_FORCE_HTTPS` variable to force a rewrite of all Git and SSH submodule URLs to HTTPS. +This allows you to clone submodules on the same GitLab instance that use absolute URLs, even if they were +configured with a Git or SSH protocol. + +```yaml +variables: + GIT_SUBMODULE_STRATEGY: recursive + GIT_SUBMODULE_FORCE_HTTPS: "true" +``` + +When enabled, GitLab Runner uses a [CI/CD job token](../jobs/ci_job_token.md) to clone the submodules with +the permissions of the user executing the job, and does not require SSH credentials. + ### Shallow cloning > Introduced in GitLab 8.9 as an experimental feature.