diff --git a/doc/user/application_security/dependency_scanning/troubleshooting_dependency_scanning.md b/doc/user/application_security/dependency_scanning/troubleshooting_dependency_scanning.md index 29f09b8e204f28765218d8b8cbb08e49702274bd..4edbc03935bd165bd288577fa4f389fbd7fa7b2b 100644 --- a/doc/user/application_security/dependency_scanning/troubleshooting_dependency_scanning.md +++ b/doc/user/application_security/dependency_scanning/troubleshooting_dependency_scanning.md @@ -15,7 +15,7 @@ When working with dependency scanning, you might encounter the following issues. ## Debug-level logging Debug-level logging can help when troubleshooting. For details, see -[debug-level logging](../index.md#debug-level-logging). +[debug-level logging](../../application_security/troubleshooting_application_security.md#debug-level-logging). ### Working around missing support for certain languages or package managers @@ -79,7 +79,7 @@ scanning job might be triggered even if the scanner doesn't support your project ## Error: `dependency_scanning is used for configuration only, and its script should not be executed` -For information, see the [GitLab Secure troubleshooting section](../index.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). +For information, see the [GitLab Secure troubleshooting section](../../application_security/troubleshooting_application_security.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). ## Import multiple certificates for Java-based projects diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index f5f25c97cfdb363f6223bed04acad03b8a5a54ed..10f339e6da440a94b700736c683ed234996df174 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -579,221 +579,3 @@ Additional details about the differences between the two solutions are outlined | **Ability to apply one standard to multiple projects** | The same compliance framework label can be applied to multiple projects inside a group. | The same security policy project can be used for multiple projects across GitLab with no requirement of being located in the same group. | Feedback is welcome on our vision for [unifying the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312) - -## Troubleshooting - -### Logging level - -The verbosity of logs output by GitLab analyzers is determined by the `SECURE_LOG_LEVEL` environment -variable. Messages of this logging level or higher are output. - -From highest to lowest severity, the logging levels are: - -- `fatal` -- `error` -- `warn` -- `info` (default) -- `debug` - -#### Debug-level logging - -WARNING: -Debug logging can be a serious security risk. The output may contain the content of -environment variables and other secrets available to the job. The output is uploaded -to the GitLab server and is visible in job logs. - -To enable debug-level logging, add the following to your `.gitlab-ci.yml` file: - -```yaml -variables: - SECURE_LOG_LEVEL: "debug" -``` - -This indicates to all GitLab analyzers that they are to output **all** messages. For more details, -see [logging level](#logging-level). - -<!-- NOTE: The below subsection(`### Secure job failing with exit code 1`) documentation URL is referred in the [/gitlab-org/security-products/analyzers/command](https://gitlab.com/gitlab-org/security-products/analyzers/command/-/blob/main/command.go#L19) repository. If this section/subsection changes, ensure to update the corresponding URL in the mentioned repository. ---> - -### Secure job failing with exit code 1 - -If a Secure job is failing and it's unclear why: - -1. Enable [debug-level logging](#debug-level-logging). -1. Run the job. -1. Examine the job's output. -1. Remove the `debug` log level to return to the default `info` value. - -### Outdated security reports - -When a security report generated for a merge request becomes outdated, the merge request shows a -warning message in the security widget and prompts you to take an appropriate action. - -This can happen in two scenarios: - -- Your [source branch is behind the target branch](#source-branch-is-behind-the-target-branch). -- The [target branch security report is out of date](#target-branch-security-report-is-out-of-date). - -#### Source branch is behind the target branch - -A security report can be out of date when the most recent common ancestor commit between the -target branch and the source branch is not the most recent commit on the target branch. - -To fix this issue, rebase or merge to incorporate the changes from the target branch. - - - -#### Target branch security report is out of date - -This can happen for many reasons, including failed jobs or new advisories. When the merge request -shows that a security report is out of date, you must run a new pipeline on the target branch. -Select **new pipeline** to run a new pipeline. - - - -### Getting warning messages `… report.json: no matching files` - -WARNING: -Debug logging can be a serious security risk. The output may contain the content of -environment variables and other secrets available to the job. The output is uploaded -to the GitLab server and visible in job logs. - -This message is often followed by the [error `No files to upload`](../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload), -and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Check -the entire job log for such messages. If you don't find these messages, retry the failed job after -setting `SECURE_LOG_LEVEL: "debug"` as a [custom CI/CD variable](../../ci/variables/index.md#for-a-project). -This provides extra information to investigate further. - -### Getting error message `sast job: config key may not be used with 'rules': only/except` - -When [including](../../ci/yaml/index.md#includetemplate) a `.gitlab-ci.yml` template -like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), -the following error may occur, depending on your GitLab CI/CD configuration: - -```plaintext -Unable to create pipeline - - jobs:sast config key may not be used with `rules`: only/except -``` - -This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs) -with [the deprecated `only` or `except` syntax.](../../ci/yaml/index.md#only--except) -To fix this issue, you must either: - -- [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules). -- (Temporarily) [Pin your templates to the deprecated versions](#pin-your-templates-to-the-deprecated-versions) - -For more information, see [Overriding SAST jobs](sast/index.md#overriding-sast-jobs). - -#### Transitioning your `only/except` syntax to `rules` - -When overriding the template to control job execution, previous instances of -[`only` or `except`](../../ci/yaml/index.md#only--except) are no longer compatible -and must be transitioned to [the `rules` syntax](../../ci/yaml/index.md#rules). - -If your override is aimed at limiting jobs to only run on `main`, the previous syntax -would look similar to: - -```yaml -include: - - template: Security/SAST.gitlab-ci.yml - -# Ensure that the scanning is only executed on main or merge requests -spotbugs-sast: - only: - refs: - - main - - merge_requests -``` - -To transition the above configuration to the new `rules` syntax, the override -would be written as follows: - -```yaml -include: - - template: Security/SAST.gitlab-ci.yml - -# Ensure that the scanning is only executed on main or merge requests -spotbugs-sast: - rules: - - if: $CI_COMMIT_BRANCH == "main" - - if: $CI_MERGE_REQUEST_ID -``` - -If your override is aimed at limiting jobs to only run on branches, not tags, -it would look similar to: - -```yaml -include: - - template: Security/SAST.gitlab-ci.yml - -# Ensure that the scanning is not executed on tags -spotbugs-sast: - except: - - tags -``` - -To transition to the new `rules` syntax, the override would be rewritten as: - -```yaml -include: - - template: Security/SAST.gitlab-ci.yml - -# Ensure that the scanning is not executed on tags -spotbugs-sast: - rules: - - if: $CI_COMMIT_TAG == null -``` - -For more information, see [`rules`](../../ci/yaml/index.md#rules). - -#### Pin your templates to the deprecated versions - -To ensure the latest support, we **strongly** recommend that you migrate to [`rules`](../../ci/yaml/index.md#rules). - -If you're unable to immediately update your CI configuration, there are several workarounds that -involve pinning to the previous template versions, for example: - - ```yaml - include: - remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/12-10-stable-ee/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml' - ``` - -Additionally, we provide a dedicated project containing the versioned legacy templates. -This can be used for offline setups or for anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). - -Instructions are available in the [legacy template project](https://gitlab.com/gitlab-org/auto-devops-v12-10). - -#### Vulnerabilities are found, but the job succeeds. How can you have a pipeline fail instead? - -In these circumstances, that the job succeeds is the default behavior. The job's status indicates -success or failure of the analyzer itself. Analyzer results are displayed in the -[job logs](../../ci/jobs/index.md#expand-and-collapse-job-log-sections), -[merge request widget](#merge-request), or -[security dashboard](security_dashboard/index.md). - -### Error: job `is used for configuration only, and its script should not be executed` - -[Changes made in GitLab 13.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41260) -to the `Security/Dependency-Scanning.gitlab-ci.yml` and `Security/SAST.gitlab-ci.yml` -templates mean that if you enable the `sast` or `dependency_scanning` jobs by setting the `rules` attribute, -they fail with the error `(job) is used for configuration only, and its script should not be executed`. - -The `sast` or `dependency_scanning` stanzas can be used to make changes to all SAST or Dependency Scanning, -such as changing `variables` or the `stage`, but they cannot be used to define shared `rules`. - -There [is an issue open to improve extendability](https://gitlab.com/gitlab-org/gitlab/-/issues/218444). -You can upvote the issue to help with prioritization, and -[contributions are welcomed](https://about.gitlab.com/community/contribute/). - -### Empty Vulnerability Report, Dependency List, License list pages - -If the pipeline has manual steps with a job that has the `allow_failure: false` option, and this job is not finished, -GitLab can't populate listed pages with the data from security reports. -In this case, [the Vulnerability Report](vulnerability_report/index.md), [the Dependency List](dependency_list/index.md), -and [the License list](../compliance/license_list.md) pages are empty. -These security pages can be populated by running the jobs from the manual step of the pipeline. - -There is [an issue open to handle this scenario](https://gitlab.com/gitlab-org/gitlab/-/issues/346843). -You can upvote the issue to help with prioritization, and -[contributions are welcomed](https://about.gitlab.com/community/contribute/). diff --git a/doc/user/application_security/sast/troubleshooting.md b/doc/user/application_security/sast/troubleshooting.md index 9416eaf1b82ce6e7e57bffbabdab71318e63b550..2b7b86c13fd7a02488521cc103e1f158c0b477fd 100644 --- a/doc/user/application_security/sast/troubleshooting.md +++ b/doc/user/application_security/sast/troubleshooting.md @@ -13,7 +13,7 @@ DETAILS: ## Debug-level logging Debug-level logging can help when troubleshooting. For details, see -[debug-level logging](../index.md#debug-level-logging). +[debug-level logging](../../application_security/troubleshooting_application_security.md#debug-level-logging). ## Pipeline errors related to changes in the GitLab-managed CI/CD template @@ -58,7 +58,7 @@ For information on this, see the [general Application Security troubleshooting s ## Error: `sast is used for configuration only, and its script should not be executed` -For information on this, see the [GitLab Secure troubleshooting section](../index.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). +For information on this, see the [GitLab Secure troubleshooting section](../../application_security/troubleshooting_application_security.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). ## SAST jobs are running unexpectedly diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index f010c5611307412975f9330a3422b9816ca8014b..4dfd9c0233dbb3b51ea23cb94d16a539f4044a32 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -634,7 +634,7 @@ This feature is separate from Secret Detection scanning, which checks your Git r ### Debug-level logging Debug-level logging can help when troubleshooting. For details, see -[debug-level logging](../index.md#debug-level-logging). +[debug-level logging](../../application_security/troubleshooting_application_security.md#debug-level-logging). ### Warning: `gl-secret-detection-report.json: no matching files` @@ -652,7 +652,7 @@ For example, you could have a pipeline triggered from a merge request containing clone is not deep enough to contain all of the relevant commits. To verify the current value, see [pipeline configuration](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone). -To confirm this as the cause of the error, enable [debug-level logging](../index.md#debug-level-logging), +To confirm this as the cause of the error, enable [debug-level logging](../../application_security/troubleshooting_application_security.md#debug-level-logging), then rerun the pipeline. The logs should look similar to the following example. The text "object not found" is a symptom of this error. diff --git a/doc/user/application_security/troubleshooting_application_security.md b/doc/user/application_security/troubleshooting_application_security.md new file mode 100644 index 0000000000000000000000000000000000000000..55252f064c2e2b35e95722cf9982e654bbe96eda --- /dev/null +++ b/doc/user/application_security/troubleshooting_application_security.md @@ -0,0 +1,191 @@ +--- +stage: Secure +group: Static Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Troubleshooting application security + +DETAILS: +**Tier:** Ultimate +**Offering:** SaaS, Self-managed + +When working with application security features, you might encounter the following issues. + +## Logging level + +The verbosity of logs output by GitLab analyzers is determined by the `SECURE_LOG_LEVEL` environment variable. Messages of this logging level or higher are output. + +From highest to lowest severity, the logging levels are: + +- `fatal` +- `error` +- `warn` +- `info` (default) +- `debug` + +### Debug-level logging + +WARNING: +Debug logging can be a serious security risk. The output may contain the content of environment variables and other secrets available to the job. The output is uploaded to the GitLab server and is visible in job logs. + +To enable debug-level logging, add the following to your `.gitlab-ci.yml` file: + +```yaml +variables: + SECURE_LOG_LEVEL: "debug" +``` + +This indicates to all GitLab analyzers that they are to output **all** messages. For more details, see [logging level](#logging-level). + +<!-- NOTE: The below subsection(`### Secure job failing with exit code 1`) documentation URL is referred in the [/gitlab-org/security-products/analyzers/command](https://gitlab.com/gitlab-org/security-products/analyzers/command/-/blob/main/command.go#L19) repository. If this section/subsection changes, ensure to update the corresponding URL in the mentioned repository. +--> + +## Secure job failing with exit code 1 + +If a Secure job is failing and it's unclear why: + +1. Enable [debug-level logging](#debug-level-logging). +1. Run the job. +1. Examine the job's output. +1. Remove the `debug` log level to return to the default `info` value. + +## Outdated security reports + +When a security report generated for a merge request becomes outdated, the merge request shows a warning message in the security widget and prompts you to take an appropriate action. + +This can happen in two scenarios: + +- Your [source branch is behind the target branch](#source-branch-is-behind-the-target-branch). +- The [target branch security report is out of date](#target-branch-security-report-is-out-of-date). + +### Source branch is behind the target branch + +A security report can be out of date when the most recent common ancestor commit between the target branch and the source branch is not the most recent commit on the target branch. + +To fix this issue, rebase or merge to incorporate the changes from the target branch. + + + +### Target branch security report is out of date + +This can happen for many reasons, including failed jobs or new advisories. When the merge request shows that a security report is out of date, you must run a new pipeline on the target branch. Select **new pipeline** to run a new pipeline. + + + +## Getting warning messages `… report.json: no matching files` + +WARNING: +Debug logging can be a serious security risk. The output may contain the content of environment variables and other secrets available to the job. The output is uploaded to the GitLab server and visible in job logs. + +This message is often followed by the [error `No files to upload`](../../ci/jobs/job_artifacts_troubleshooting.md#error-message-no-files-to-upload), and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Check the entire job log for such messages. If you don't find these messages, retry the failed job after setting `SECURE_LOG_LEVEL: "debug"` as a [custom CI/CD variable](../../ci/variables/index.md#for-a-project). This provides extra information to investigate further. + +## Getting error message `sast job: config key may not be used with 'rules': only/except` + +When [including](../../ci/yaml/index.md#includetemplate) a `.gitlab-ci.yml` template like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), the following error may occur, depending on your GitLab CI/CD configuration: + +```plaintext +Unable to create pipeline + + jobs:sast config key may not be used with `rules`: only/except +``` + +This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs) with [the deprecated `only` or `except` syntax.](../../ci/yaml/index.md#only--except) + +To fix this issue, you must either: + +- [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules). +- (Temporarily) [Pin your templates to the deprecated versions](#pin-your-templates-to-the-deprecated-versions) + +For more information, see [Overriding SAST jobs](sast/index.md#overriding-sast-jobs). + +### Transitioning your `only/except` syntax to `rules` + +When overriding the template to control job execution, previous instances of [`only` or `except`](../../ci/yaml/index.md#only--except) are no longer compatible and must be transitioned to [the `rules` syntax](../../ci/yaml/index.md#rules). + +If your override is aimed at limiting jobs to only run on `main`, the previous syntax would look similar to: + +```yaml +include: + - template: Security/SAST.gitlab-ci.yml + +# Ensure that the scanning is only executed on main or merge requests +spotbugs-sast: + only: + refs: + - main + - merge_requests +``` + +To transition the above configuration to the new `rules` syntax, the override would be written as follows: + +```yaml +include: + - template: Security/SAST.gitlab-ci.yml + +# Ensure that the scanning is only executed on main or merge requests +spotbugs-sast: + rules: + - if: $CI_COMMIT_BRANCH == "main" + - if: $CI_MERGE_REQUEST_ID +``` + +If your override is aimed at limiting jobs to only run on branches, not tags, it would look similar to: + +```yaml +include: + - template: Security/SAST.gitlab-ci.yml + +# Ensure that the scanning is not executed on tags +spotbugs-sast: + except: + - tags +``` + +To transition to the new `rules` syntax, the override would be rewritten as: + +```yaml +include: + - template: Security/SAST.gitlab-ci.yml + +# Ensure that the scanning is not executed on tags +spotbugs-sast: + rules: + - if: $CI_COMMIT_TAG == null +``` + +For more information, see [`rules`](../../ci/yaml/index.md#rules). + +### Pin your templates to the deprecated versions + +To ensure the latest support, we **strongly** recommend that you migrate to [`rules`](../../ci/yaml/index.md#rules). + +If you're unable to immediately update your CI configuration, there are several workarounds that involve pinning to the previous template versions, for example: + + ```yaml + include: + remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/12-10-stable-ee/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml' + ``` + +Additionally, we provide a dedicated project containing the versioned legacy templates. This can be used for offline setups or for anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). + +Instructions are available in the [legacy template project](https://gitlab.com/gitlab-org/auto-devops-v12-10). + +### Vulnerabilities are found, but the job succeeds. How can you have a pipeline fail instead? + +In these circumstances, that the job succeeds is the default behavior. The job's status indicates success or failure of the analyzer itself. Analyzer results are displayed in the +[job logs](../../ci/jobs/index.md#expand-and-collapse-job-log-sections), [merge request widget](index.md#merge-request), or [security dashboard](security_dashboard/index.md). + +## Error: job `is used for configuration only, and its script should not be executed` + +[Changes made in GitLab 13.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41260) to the `Security/Dependency-Scanning.gitlab-ci.yml` and `Security/SAST.gitlab-ci.yml` templates mean that if you enable the `sast` or `dependency_scanning` jobs by setting the `rules` attribute, they fail with the error `(job) is used for configuration only, and its script should not be executed`. + +The `sast` or `dependency_scanning` stanzas can be used to make changes to all SAST or Dependency Scanning, such as changing `variables` or the `stage`, but they cannot be used to define shared `rules`. + +There [is an issue open to improve extendability](https://gitlab.com/gitlab-org/gitlab/-/issues/218444). You can upvote the issue to help with prioritization, and [contributions are welcomed](https://about.gitlab.com/community/contribute/). + +## Empty Vulnerability Report, Dependency List, License list pages + +If the pipeline has manual steps with a job that has the `allow_failure: false` option, and this job is not finished, GitLab can't populate listed pages with the data from security reports. In this case, [the Vulnerability Report](vulnerability_report/index.md), [the Dependency List](dependency_list/index.md), and [the License list](../compliance/license_list.md) pages are empty. These security pages can be populated by running the jobs from the manual step of the pipeline. + +There is [an issue open to handle this scenario](https://gitlab.com/gitlab-org/gitlab/-/issues/346843). You can upvote the issue to help with prioritization, and [contributions are welcomed](https://about.gitlab.com/community/contribute/).