diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index 1c74d4b8d39d04e9beea8281961cc595182894f8..31fa8f4dfe2843848023db150c973c6d44062bdc 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -7,212 +7,193 @@ type: index, howto # Migrating from Jenkins **(FREE ALL)** -A lot of GitLab users have successfully migrated to GitLab CI/CD from Jenkins. -We've collected several resources here that you might find informative if you're just getting started. -Think of this page as a "GitLab CI/CD for Jenkins Users" guide. +If you're migrating from Jenkins to GitLab CI/CD, you should be able +to create CI/CD pipelines that do everything you need. + +You can start by watching the [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF5Y) +video for examples of: + +- Converting a Jenkins pipeline into a GitLab CI/CD pipeline. +- Using Auto DevOps to test your code automatically. + +## Get started The following list of recommended steps was created after observing organizations -that were able to quickly complete this migration: - -1. Start by reading the GitLab CI/CD [Quick Start Guide](../quick_start/index.md) and [important product differences](#important-product-differences). -1. Learn the importance of [managing the organizational transition](#manage-organizational-transition). -1. [Add runners](../runners/index.md) to your GitLab instance. -1. Educate and enable your developers to independently perform the following steps in their projects: - 1. Review the [Quick Start Guide](../quick_start/index.md) and [Pipeline Configuration Reference](../yaml/index.md). - 1. Use the [Jenkins Wrapper](#jenkinsfile-wrapper) to temporarily maintain fragile Jenkins jobs. - 1. Migrate the build and CI jobs and configure them to show results directly in your merge requests. They can use [Auto DevOps](../../topics/autodevops/index.md) as a starting point, and [customize](../../topics/autodevops/customize.md) or [decompose](../../topics/autodevops/customize.md#use-individual-components-of-auto-devops) the configuration as needed. - 1. Add [Review Apps](../review_apps/index.md). - 1. Migrate the deployment jobs using [cloud deployment templates](../cloud_deployment/index.md), adding [environments](../environments/index.md), and [deploy boards](../../user/project/deploy_boards.md). - 1. Work to unwrap any jobs still running with the use of the Jenkins wrapper. -1. Take stock of any common CI/CD job definitions then create and share [templates](#templates) for them. -1. Check the [pipeline efficiency documentation](../pipelines/pipeline_efficiency.md) - to learn how to make your GitLab CI/CD pipelines faster and more efficient. - -Watch the [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF5Y) video for examples of how to: - -- Convert a Jenkins pipeline into a GitLab CI/CD pipeline. -- Use Auto DevOps to test your code automatically. - -Otherwise, read on for important information that helps you get the ball rolling. Welcome -to GitLab! +that were able to quickly complete this migration. + +Engineers that plan to migrate projects to GitLab CI/CD should: + +- Read about some [key GitLab CI/CD features](#key-gitlab-cicd-features). +- Follow tutorials to create: + - [Your first GitLab pipeline](../quick_start/index.md). + - [A more complex pipeline](../quick_start/tutorial.md) that builds, tests, + and deploys a static site. +- Review the [`.gitlab-ci.yml` keyword reference](../yaml/index.md). +- Ensure [runners](../runners/index.md) are available, either by using shared GitLab.com runners + or installing new runners. +- Migrate build and CI jobs and configure them to show results directly in merge requests. + You can use [Auto DevOps](../../topics/autodevops/index.md) as a starting point, + and [customize](../../topics/autodevops/customize.md) or [decompose](../../topics/autodevops/customize.md#use-individual-components-of-auto-devops) + the configuration as needed. +- Migrate deployment jobs by using [cloud deployment templates](../cloud_deployment/index.md), + [environments](../environments/index.md), and the [GitLab agent for Kubernetes](../../user/clusters/agent/index.md). +- Check if any CI/CD configuration can be reused across different projects, then create + and share [templates](#templates). +- Check the [pipeline efficiency documentation](../pipelines/pipeline_efficiency.md) + to learn how to make your GitLab CI/CD pipelines faster and more efficient. If you have questions that are not answered here, the [GitLab community forum](https://forum.gitlab.com/) can be a great resource. -## Manage organizational transition +### Manage organizational changes An important part of transitioning from Jenkins to GitLab is the cultural and organizational -changes that come with the move, and successfully managing them. A few -things we have found that help this are: +changes that come with the move, and successfully managing them. -- Setting and communicating a clear vision of what your migration goals are helps +A few things that organizations have reported as helping: + +- Set and communicate a clear vision of what your migration goals are, which helps your users understand why the effort is worth it. The value is clear when the work is done, but people need to be aware while it's in progress too. -- Sponsorship and alignment from the relevant leadership team helps with the point above. -- Spending time educating your users on what's different and sharing this document - with them helps ensure you are successful. -- Finding ways to sequence or delay parts of the migration can help a lot, but you - don't want to leave things in a non-migrated (or partially-migrated) state for too - long. To gain all the benefits of GitLab, moving your existing Jenkins setup over - as-is, including any current problems, isn't enough. You need to take advantage - of the improvements that GitLab offers, and this requires (eventually) updating - your implementation as part of the transition. - -## JenkinsFile Wrapper - -We are building a [JenkinsFile Wrapper](https://gitlab.com/gitlab-org/jfr-container-builder/) which -you can use to run a complete Jenkins instance inside of a GitLab job, including plugins. This can help ease the process -of transition, by letting you delay the migration of less urgent pipelines for a period of time. - -If you are interested in helping GitLab test the wrapper, join our [public testing issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215675) for instructions and to provide your feedback. - -NOTE: -If you have a paid GitLab subscription, the JenkinsFile Wrapper is not packaged with GitLab and falls outside of the scope of support. For more information, see the [Statement of Support](https://about.gitlab.com/support/statement-of-support/). - -## Important product differences - -Some high level differences between the products worth mentioning are: - -- With GitLab you don't need a root `pipeline` keyword to wrap everything. -- The way pipelines are triggered and [trigger other pipelines](../yaml/index.md#trigger) - is different than Jenkins. GitLab pipelines can be triggered: - - - on push - - on [schedule](../pipelines/schedules.md) - - from the [GitLab UI](../pipelines/index.md#run-a-pipeline-manually) - - by [API call](../triggers/index.md) - - by [webhook](../triggers/index.md#use-a-webhook) - - by [ChatOps](../chatops/index.md) - -- You can control which jobs run in which cases, depending on how they are triggered, - with the [`rules` syntax](../yaml/index.md#rules). -- GitLab [pipeline scheduling concepts](../pipelines/schedules.md) are also different from Jenkins. -- You can reuse pipeline configurations using the [`include` keyword](../yaml/index.md#include) - and [templates](#templates). Your templates can be kept in a central repository (with different - permissions), and then any project can use them. This central project could also - contain scripts or other reusable code. -- You can also use the [`extends` keyword](../yaml/index.md#extends) to reuse configuration - in a single pipeline configuration. -- All jobs in a single stage always run in parallel, and all stages run in sequence. - Certain jobs might break this sequencing as needed with our [directed acyclic graph](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47063) - feature. +- Sponsorship and alignment from the relevant leadership teams helps with the point above. +- Spend time educating your users on what's different, and share this guide + with them. +- Finding ways to sequence or delay parts of the migration can help a lot. Importantly though, + try not to leave things in a non-migrated (or partially-migrated) state for too + long. +- To gain all the benefits of GitLab, moving your existing Jenkins setup over + as-is, including any current problems, isn't enough. Take advantage of the improvements + that GitLab CI/CD offers, and update your implementation as part of the transition. + +### Key GitLab CI/CD features + +GitLab CI/CD key features might be different or not exist in Jenkins. For example, +in GitLab: + +- Pipelines can be triggered with: + - A Git push + - A [Schedule](../pipelines/schedules.md) + - The [GitLab UI](../pipelines/index.md#run-a-pipeline-manually) + - An [API call](../triggers/index.md) + - A [webhook](../triggers/index.md#use-a-webhook) +- You can control which jobs run in which cases with the [`rules` syntax](../yaml/index.md#rules). +- You can reuse pipeline configurations: + - Use the [`extends` keyword](../yaml/index.md#extends) to reuse configuration + in a single pipeline configuration. + - Use the [`include` keyword](../yaml/index.md#include) to reuse configuration across + multiple pipelines and projects. +- Jobs are grouped into stages, and jobs in the same stage can run at the same time. + Stages run in sequence. Jobs can be configured to run outside of the stage ordering with the + [`needs` keyword](../yaml/index.md#needs). - The [`parallel`](../yaml/index.md#parallel) keyword can automatically parallelize tasks, - like tests that support parallelization. -- Usually all jobs in a single stage run in parallel, and all stages run in sequence. - Different [pipeline architectures](../pipelines/pipeline_architectures.md) allow you to change this behavior. -- The new [`rules` syntax](../yaml/index.md#rules) is the recommended method of - controlling when different jobs run. It is more powerful than the `only/except` syntax. -- One important difference is that jobs run independently of each other and have a - fresh environment in each job. Passing artifacts between jobs is controlled using the - [`artifacts`](../yaml/index.md#artifacts) and [`dependencies`](../yaml/index.md#dependencies) - keywords. When finished, use the planned [Workspaces](https://gitlab.com/gitlab-org/gitlab/-/issues/29265) - feature to persist a common workspace between serial jobs. -- The `.gitlab-ci.yml` file is checked in to the root of your repository, much like a Jenkinsfile, but - is in the YAML format (see [complete reference](../yaml/index.md)) instead of a Groovy DSL. It's most - analogous to the declarative Jenkinsfile format. -- Manual approvals or gates can be set up as [`when:manual` jobs](../jobs/job_control.md#create-a-job-that-must-be-run-manually). These can - also leverage [`protected environments`](../jobs/job_control.md#run-a-job-after-a-delay) - to control who is able to approve them. -- GitLab comes with a [container registry](../../user/packages/container_registry/index.md), so you can use - container images to set up your build environment. For example, set up one pipeline that builds your build environment - itself and publish that to the container registry. Then, have your pipelines use this instead of each building their - own environment, which is slower and may be less consistent. We have extensive documentation on [how to use the Container Registry](../../user/packages/container_registry/index.md). -- A central utilities repository can be a great place to put assorted scheduled jobs - or other manual jobs that function like utilities. Jenkins installations tend to - have a few of these. - -## Agents vs. runners - -Both Jenkins agents and GitLab runners are the hosts that run jobs. To convert the -Jenkins agent, uninstall it and then [install and register the runner](../runners/index.md). -Runners do not require much overhead, so you can size them similarly to the Jenkins -agents you were using. - -Some important differences in the way runners work in comparison to agents are: - -- Runners can be set up as [shared across an instance, be added at the group level, or set up at the project level](../runners/runners_scope.md). - They self-select jobs from the scopes you've defined automatically. -- You can also [use tags](../runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) for finer control, and - associate runners with specific jobs. For example, you can use a tag for jobs that + especially tests that support parallelization. +- Jobs run independently of each other and have a fresh environment for each job. + Passing artifacts between jobs is controlled using the [`artifacts`](../yaml/index.md#artifacts) + and [`dependencies`](../yaml/index.md#dependencies) keywords. +- The `.gitlab-ci.yml` configuration file exists in your Git repository, like a `Jenkinsfile`, + but is [a YAML file](#yaml-configuration-file), not Groovy. +- GitLab comes with a [container registry](../../user/packages/container_registry/index.md). + You can build and store custom container images to run your jobs in. + +## Runners + +Like Jenkins agents, GitLab runners are the hosts that run jobs. If you are using GitLab.com, +you can use the [shared runner fleet](../runners/index.md) to run jobs without provisioning +your own runners. + +To convert a Jenkins agent for use with GitLab CI/CD, uninstall the agent and then +[install and register a runner](../runners/index.md). Runners do not require much overhead, +so you might be able to use similar provisioning as the Jenkins agents you were using. + +Some key details about runners: + +- Runners can be [configured](../runners/runners_scope.md) to be shared across an instance, + a group, or dedicated to a single project. +- You can use the [`tags` keyword](../runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) + for finer control, and associate runners with specific jobs. For example, you can use a tag for jobs that require dedicated, more powerful, or specific hardware. - GitLab has [autoscaling for runners](https://docs.gitlab.com/runner/configuration/autoscale.html). Use autoscaling to provision runners only when needed and scale down when not needed, similar to ephemeral agents in Jenkins. -If you are using `gitlab.com`, you can take advantage of our [shared runner fleet](../runners/index.md) -to run jobs without provisioning your own runners. We are investigating making them -[available for self-managed instances](https://gitlab.com/groups/gitlab-org/-/epics/835) -as well. - -## Groovy vs. YAML +## YAML configuration file -Jenkins Pipelines are based on [Groovy](https://groovy-lang.org/), so the pipeline specification is written as code. -GitLab works a bit differently, using the more highly structured [YAML](https://yaml.org/) format. -The scripting elements are in `script` blocks separate from the pipeline specification itself. +GitLab pipeline configuration files use the [YAML](https://yaml.org/) format instead of +the [Groovy](https://groovy-lang.org/) format that Jenkins uses. -Using YAML is a strength of GitLab, in that it helps keep the learning curve much simpler to get up and running. -It also avoids some of the problem of unconstrained complexity which can make your Jenkinsfile hard to understand -and manage. - -We do of course still value DRY (don't repeat yourself) principles. We want to ensure that -behaviors of your jobs can be codified once and applied as needed. You can use the `extends` syntax to -[reuse configuration in your jobs](../yaml/index.md#extends), and `include` can -be used to [reuse pipeline configurations](../yaml/index.md#include) in pipelines -in different projects: +Using YAML is a strength of GitLab CI/CD, as it is a simple format to understand +and start using. For example, a small configuration file with two jobs and some +shared configuration in a hidden job: ```yaml -.in-docker: +.test-config: tags: - - docker - image: alpine + - docker-runners + stage: test -rspec: +test-job: extends: - - .in-docker + - .docker-config script: - - rake rspec + - bundle exec rake rspec + +lint-job: + extends: + - .docker-config + script: + - yarn run prettier ``` -## Artifact publishing +In this example: + +- The commands to run in jobs are added with the [`script` keyword](../yaml/index.md#script). +- The [`extends` keyword](../yaml/index.md#extends) reduces duplication in the configuration + by adding the same `tags` and `stage` configuration defined in `.test-config` to both jobs. + +### Artifacts + +In GitLab, any job can use the [`artifacts` keyword](../yaml/index.md#artifacts) +to define a set of [artifacts](../jobs/job_artifacts.md) to be stored when a job completes. +Artifacts are files that can be used in later jobs, for example for testing or deployment. -Artifacts may work a bit differently than you've used them with Jenkins. In GitLab, any job can define -a set of artifacts to be saved by using the `artifacts` keyword. This can be configured to point to a file -or set of files that can then be persisted from job to job. Read more on our detailed -[artifacts documentation](../jobs/job_artifacts.md): +For example: ```yaml pdf: script: xelatex mycv.tex artifacts: paths: - - ./mycv.pdf - - ./output/ + - mycv.pdf + - output/ expire_in: 1 week ``` -Additionally, we have package management features like built-in container and package registries that you -can leverage. You can see the complete list of packaging features in the -[Packages and registries](../../user/packages/index.md) documentation. +In this example: -## Integrated features +- The `mycv.pdf` file and all the files in `output/` are stored and could be used + in later jobs. +- To save resources, the artifacts expire and are deleted after one week. -You may have used plugins to get things like code quality, unit tests, and security scanning working in Jenkins. -GitLab takes advantage of our connected ecosystem to automatically pull these kinds of results into -your merge requests, pipeline details pages, and other locations. You may find that you actually don't -need to configure anything to have these appear. +### Scanning features + +You might have used plugins for things like code quality, security, or static application scanning +in Jenkins. Tools like these are already available in GitLab and can be used in your +pipeline. + +GitLab features including [code quality](../testing/code_quality.md), [security scanning](../../user/application_security/index.md), +[SAST](../../user/application_security/sast/index.md), and many others generate reports +when they complete. These reports can be displayed in merge requests and pipeline details pages. ### Templates -For advanced CI/CD teams, project templates can enable the reuse of pipeline configurations, -as well as encourage inner sourcing. +For organizations with many CI/CD pipelines, you can use project templates to configure +custom CI/CD configuration templates and reuse them across projects. -In self-managed GitLab instances, you can build an [Instance Template Repository](../../administration/settings/instance_template_repository.md). -Development teams across the whole organization can select templates from a dropdown list. -A group maintainer or a group owner is able to set a group to use as the source for the -[custom project templates](../../administration/custom_project_templates.md). This can -be used by all projects in the group. An instance administrator can set a group as -the source for [instance project templates](../../user/group/custom_project_templates.md), -which can be used by projects in that instance. +Group maintainers can configure a group to use as the source for [custom project templates](../../administration/custom_project_templates.md). +These templates can be used by all projects in the group. + +An instance administrator can set a group as the source for [instance project templates](../../user/group/custom_project_templates.md), +which can be used by all projects in that instance. ## Convert a declarative Jenkinsfile @@ -360,5 +341,13 @@ my_job: ## Additional resources -For help making your pipelines faster and more efficient, see the -[pipeline efficiency documentation](../pipelines/pipeline_efficiency.md). +- You can use the [JenkinsFile Wrapper](https://gitlab.com/gitlab-org/jfr-container-builder/) + to run a complete Jenkins instance inside of a GitLab CI/CD job, including plugins. Use this tool to + help ease the transition to GitLab CI/CD, by delaying the migration of less urgent pipelines. + + NOTE: + The JenkinsFile Wrapper is not packaged with GitLab and falls outside of the scope of support. + For more information, see the [Statement of Support](https://about.gitlab.com/support/statement-of-support/). +- If your tooling outputs packages that you want to make accessible, you can store them + in a [package registry](../../user/packages/index.md). +- Use [review Apps](../review_apps/index.md) to preview changes before merging them. diff --git a/doc/ci/quick_start/tutorial.md b/doc/ci/quick_start/tutorial.md index 2265d628ed805e0f386f1705f499bb4a53bfd3f2..412396155908821068c89ee797e17a58ef413ab9 100644 --- a/doc/ci/quick_start/tutorial.md +++ b/doc/ci/quick_start/tutorial.md @@ -8,10 +8,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w This tutorial walks you through configuring a progressively more complex CI/CD pipeline through small, iterative steps. The pipeline is always fully functional, -but it gains more functionality with each step. +but it gains more functionality with each step. The goal is to build, test, and deploy +a documentation site. -When you finish this tutorial, you will have a new project on GitLab.com and a working documentation site on -[Docusaurus](https://docusaurus.io/). +When you finish this tutorial, you will have a new project on GitLab.com and a working documentation site +using [Docusaurus](https://docusaurus.io/). To complete this tutorial, you will: diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 395873d3107ed4c1c45c7bca69a81cc9c6836641..91a3d71642d878b108f8854df7a181a2712b22aa 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -21,7 +21,7 @@ type: howto, reference WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. [An epic exists](https://gitlab.com/groups/gitlab-org/-/epics/2493) -to add this functionality to the [agent](../index.md). +to add this functionality to the [agent](../clusters/agent/index.md). FLAG: On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`.