Add integrated error tracking

doc/operations/error_tracking.md

@@ -34,15 +34,15 @@ For error tracking to work, you need two pieces:
 
 ## Sentry error tracking
 
-[Sentry](https://sentry.io/) is an open source error tracking system. GitLab allows administrators to connect Sentry to GitLab, to allow users to view a list of Sentry errors in GitLab.
+[Sentry](https://sentry.io/) is an open source error tracking system. GitLab allows administrators to connect Sentry to GitLab so users can view a list of Sentry errors in GitLab.
 
 ### Deploying Sentry
 
-You can sign up to the cloud hosted [Sentry](https://sentry.io) or deploy your own [on-premise instance](https://github.com/getsentry/onpremise/).
+You can sign up to the cloud-hosted [Sentry](https://sentry.io) or deploy your own [on-premise instance](https://github.com/getsentry/onpremise/).
 
 ### Enabling Sentry
 
-GitLab provides an easy way to connect Sentry to your project. You need at
+GitLab provides a way to connect Sentry to your project. You need at
 least Maintainer [permissions](../user/permissions.md) to enable the Sentry integration.
 
 1. Sign up to Sentry.io or [deploy your own](#deploying-sentry) Sentry instance.
@@ -94,9 +94,9 @@ Here, you can filter errors by title or by status (one of Ignored , Resolved, or
 
 ## Error Details
 
-From error list, users can navigate to the error details page by selecting the title of any error.
+From error list, users can go to the error details page by selecting the title of any error.
 
-This page has:
+This page includes:
 
 - A link to the Sentry issue.
 - A link to the GitLab commit if the Sentry [release ID/version](https://docs.sentry.io/product/releases/?platform=javascript#configure-sdk) on the Sentry Issue's first release matches a commit SHA in your GitLab hosted project.
@@ -108,26 +108,26 @@ By default, a **Create issue** button is displayed:
 ![Error Details without Issue Link](img/error_details_v12_7.png)
 
 If you create a GitLab issue from the error, the **Create issue** button changes to a **View issue**
-button and a link to the GitLab issue displays within the error detail section.
+button and a link to the GitLab issue displays in the error detail section.
 
 ## Taking Action on errors
 
-You can take action on Sentry Errors from within the GitLab UI. Marking errors ignored or resolved require at least Developer role.
+You can take action on Sentry Errors in the GitLab UI. Marking errors as ignored or resolved requires at least Developer role.
 
 ### Ignoring errors
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39665) in GitLab 12.7.
 
-From within the [Error Details](#error-details) page you can ignore a Sentry error by selecting the **Ignore** button near the top of the page.
+In the [Error Details](#error-details) page you can ignore a Sentry error by selecting **Ignore** near the top of the page.
 
-Ignoring an error prevents it from appearing in the [Error Tracking List](#error-tracking-list), and silences notifications that were set up within Sentry.
+Ignoring an error prevents it from appearing in the [Error Tracking List](#error-tracking-list), and silences notifications that were set up in Sentry.
 
 ### Resolving errors
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39825) in GitLab 12.7.
 
-From within the [Error Details](#error-details) page you can resolve a Sentry error by
-selecting the **Resolve** button near the top of the page.
+In the [Error Details](#error-details) page you can resolve a Sentry error by
+selecting **Resolve** near the top of the page.
 
 Marking an error as resolved indicates that the error has stopped firing events. If a GitLab issue is linked to the error, then the issue closes.
 
@@ -142,51 +142,231 @@ If another event occurs, the error reverts to unresolved.
 NOTE:
 Available only on GitLab.com. This feature is currently in [Open Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#open-beta).
 
-Integrated error tracking is a lightweight alternative to Sentry backend.
-You still use Sentry SDK with your application. But you don't need to deploy Sentry
-or set up for cloud-hosted Sentry. Instead, you use GitLab as a backend for it.
+### Known limitations
 
-Sentry backend automatically assigns a Data Source Name (DSN) for every project you create.
-GitLab does the same. You should be able to find a DSN for your project in the GitLab error tracking
-settings. By using a GitLab-provided DSN, your application connects to GitLab to report an error.
-Those errors are stored in the GitLab database and rendered by the GitLab UI, in the same way as
-Sentry integration.
+Only basic support is provided with `capture_exception` as the holding method.
+Additional features requests (see this [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/340178)) will be added on a case-by-case basis.
 
-In GitLab 15.6 and later, the integrated error tracking
-uses a new backend based on the ClickHouse database that enables better scalability.
-Integrated error tracking remains limited in comparison to the Sentry backend, as only a small subset of the
-Sentry API is implemented.
+### Debugging issues
 
-Changing the GitLab error UI to use the GitLab Observability UI is tracked in epic [19](https://gitlab.com/groups/gitlab-org/opstrace/-/epics/32).
+The majority of languages are supported by Sentry expose `debug` option as part of initialization.
+It is also possible to output JSON before it is sent to the API.
+See the [Go example](#go) below for a suggested solution.
 
-### Project settings
+### Enabling error tracking
 
-You can find the feature configuration at **Settings > Monitor > Error Tracking**.
+Regardless of the programming language, you need to enable error tracking for your project. This doc assumes you already have a project for which you want to enable error tracking.
+This example uses the `gitlab.com` instance.
 
-#### How to enable
+To enable error tracking, follow these steps:
 
-1. Select **GitLab** as the error tracking backend for your project:
+1. In your project, go to **Settings >Monitor**. Expand the **Error Tracking** tab:
 
-   ![Error Tracking Settings](img/error_tracking_setting_v14_3.png)
+   ![MonitorTabPreEnable](img/Monitor_tab-pre-enable.png)
 
-1. Select **Save changes**. After page reload you should see a text field with the DSN string. Copy it.
+1. Enable Error Tracking with GitLab as backend:
 
-   ![Error Tracking Settings DSN](img/error_tracking_setting_dsn_v14_4.png)
+   ![MonitorTabPostEnable](img/Monitor_tab-post-enable.png)
 
-1. Take the DSN from the previous step and configure your Sentry SDK with it. Errors are now
-   reported to the GitLab collector and are visible in the [GitLab UI](#error-tracking-list).
+1. Select the `Save Changes` button.
 
-#### Managing DSN
+1. Copy the DSN string. You will need this later.
 
-When you enable the feature you receive a DSN. It includes a hash used for authentication. This hash
-is a client key. GitLab uses client keys to authenticate error tracking requests from your
-application to the GitLab backend.
+### Listing captured errors
 
-In some situations, you may want to create a new client key and remove an existing one.
-You can do so by managing client keys with the [error tracking API](../api/error_tracking.md).
+Once your application has emitted errors to the Error Tracking API through the Sentry SDK, they should be available under **Monitor > Error Tracking** tab/section.
 
-#### Limitations
+![MonitorListErrors](img/Monitor-list_errors.png)
 
-The Integrated Error Tracking feature was built and tested with Sentry SDK for Ruby on Rails.
-Support for other languages and frameworks is not guaranteed. For up-to-date information, see the
-[compatibility issue](https://gitlab.com/gitlab-org/gitlab/-/issues/340178).
+For more details, refer to the [GitLab error tracking documentation](https://gitlab.com/help/operations/error_tracking#error-tracking-list).
+
+This process assumes the GDK feature flag `integrated_error_tracking` is enabled. If you are running GDK locally and you do not see the option for error tracking, you can enable it by running the following commands:
+
+```linux
+cd <PATH_TO_GDK>
+gdk rails console
+Feature.enable(:integrated_error_tracking)
+```
+
+### Emitting Errors
+
+#### Supported Sentry types
+
+According to the [data model](https://develop.sentry.dev/sdk/envelopes/#data-model), the available item types are:
+
+- [Event](https://develop.sentry.dev/sdk/event-payloads/)
+- [Transactions](https://develop.sentry.dev/sdk/event-payloads/transaction/)
+- Attachment
+- [Session](https://develop.sentry.dev/sdk/sessions/)
+- [Sessions](https://develop.sentry.dev/sdk/sessions/)
+- [User feedback](https://develop.sentry.dev/sdk/envelopes/#user-feedback) (also known as user report)
+- [Client report](https://develop.sentry.dev/sdk/client-reports/)
+
+Items of various types can be sent to the error tracking app, using either the Store endpoint, the envelope endpoint, or both. The following table lists all event types available through Sentry SDK. It also explains which endpoint can be used for ingestion and whether it is supported by GitLab Observability Backend.
+
+Event item types can contain various interfaces, such as exception, message, stack trace, and template. You can read more about the core data interfaces in [Sentry documentation](https://develop.sentry.dev/sdk/event-payloads/#core-interfaces).
+
+| Item type | Interface  | Can be sent through the Store endpoint | Can be sent through the Envelope endpoint | Currently Supported |
+|---------------|--------------|----------------|----------------|---------------------|
+| event         | exception    |  **{check-circle}** Yes      |  **{check-circle}** Yes       | **{check-circle}** Yes           |
+| event         | message      |  **{check-circle}** Yes       |  **{check-circle}** Yes       | **{check-circle}** Yes          |
+| event         | stack trace   |  **{check-circle}** Yes       |  **{check-circle}** Yes       | **{check-circle}** Yes          |
+| event         | template     |  **{check-circle}** Yes       |  **{check-circle}** Yes       | **{dotted-circle}** No          |
+| transaction   |     NA       |  **{dotted-circle}** No      |  **{check-circle}** Yes       | **{dotted-circle}** No           |
+| attachment    |     NA       |  **{dotted-circle}** No      |  **{check-circle}** Yes       | **{dotted-circle}** No           |
+| session       |     NA       |  **{dotted-circle}** No      |  **{check-circle}** Yes       | **{dotted-circle}** No           |
+| sessions      |     NA       |  **{dotted-circle}** No      |  **{check-circle}** Yes       | **{dotted-circle}** No           |
+| user_report   |     NA       |  **{dotted-circle}** No      |  **{check-circle}** Yes       | **{dotted-circle}** No           |
+| client_report |     NA       |  **{dotted-circle}** No      |  **{check-circle}** Yes       | **{dotted-circle}** No           |
+
+\* NA = Not Applicable
+
+#### Supported languages
+
+Each language shows a basic example of how to capture exceptions with the respective SDK.
+For more in-depth documentation, see [documentation for Sentry SDK](https://docs.sentry.io/). You can also find information for additional programming languages.
+
+Only a subset of languages is supported.
+
+The following table lists them:
+
+| Sentry SDK  | Supported?  |
+| ----------- | ----------- |
+| Ruby        | Yes |
+| Go          | Yes |
+| JavaScript  | Yes |
+| Java        | Yes |
+| Python      | Yes |
+| PHP         | Yes |
+| .NET        | Not tested |
+| Android     | Not tested |
+| Apple       | Not tested |
+| Perl        | Not tested |
+
+A more up-to-date version of [this matrix can be found in this doc](https://gitlab.com/gitlab-org/opstrace/opstrace/-/issues/1737).
+
+#### Go
+
+1. `chdir` into folder `docs/guides/user/error_tracking_examples/go/`
+1. Install the dependencies with the following command:
+
+   ```shell
+   go mod tidy
+   ```
+
+1. Run the following command:
+
+   ```shell
+   export SENTRY_DSN="<DSN string>"
+   go run main.go <DSN string>
+   ```
+
+After you've run this program, there should be an error visible in the Error tracking tab from `Listing captured errors` section of this document.
+
+#### Ruby
+
+1. `chdir` into folder `docs/guides/user/error_tracking_examples/ruby/`
+1. Install the dependencies with the following command:
+
+   ```shell
+   gem install bundler
+   bundle install
+   ```
+
+1. Execute the example with the following command:
+
+   ```shell
+   export SENTRY_DSN="<DSN string>"
+   ruby app.rb
+   ```
+
+After you've run this program, there should be an error visible in the Error tracking tab from `Listing captured errors` section of this document.
+
+#### PHP
+
+1. `chdir` into folder `docs/guides/user/error_tracking_examples/php/`
+
+1. Build and run the Docker container with the following commands:
+
+```shell
+export SENTRY_DSN="<DSN string>"
+docker build -t sentry-php .
+docker run -e SENTRY_DSN --rm sentry-php
+```
+
+After you've run this program, there should be an error visible in the Error tracking tab from `Listing captured errors` section of this document.
+
+#### Python
+
+1. `chdir` into folder `docs/guides/user/error_tracking_examples/python/`
+
+1. Install the dependencies with the following commands:
+
+   ```shell
+   virtualenv env
+   source env/bin/activate
+   pip install -r requirements.txt
+   ```
+
+1. Run the following commands:
+
+```shell
+export SENTRY_DSN="<DSN string>"
+python send_exception.py
+```
+
+After you've run this program, there should be an error visible in the Error tracking tab from `Listing captured errors` section of this document.
+
+#### Java
+
+1. `chdir` into folder `docs/guides/user/error_tracking_examples/python/`
+
+1. Run the following command:
+
+```shell
+export SENTRY_DSN="<DSN string>"
+./gradlew run
+```
+
+#### Node.js
+
+1. `chdir` into folder `docs/guides/user/error_tracking_examples/nodejs/`
+
+1. Install the dependencies with the following command:
+
+   ```shell
+   npm install --save @sentry/node @sentry/tracing
+   ```
+
+1. Run the following command:
+
+   ```shell
+   export SENTRY_DSN="<DSN string>"
+   node ./test.js
+   ```
+
+After you've run this program, there should be an error visible in the Error tracking tab from `Listing captured errors` section of this document.
+
+### Rotating Sentry DSN
+
+The Sentry DSN (client key) is a secret and it should not be exposed to the public. If it's leaked, you can rotate the Sentry DSN with the following steps:
+
+1. [Create an access token](../user/profile/personal_access_tokens.md#create-a-personal-access-token) by clicking your profile picture in GitLab.com. Then choose Preferences,then Access Token. Make sure you add the API scope.
+1. Using the [error tracking API](../api/error_tracking.md), create a new Sentry DSN with the following command:
+
+   ```shell
+   curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --header "Content-Type: application/json" \
+     "https://gitlab.example.com/api/v4/projects/<your_project_number>/error_tracking/client_keys"
+   ```
+
+1. Get the available client keys (Sentry DSNs). Ensure that the newly created Sentry DSN is in place. Then note down the key ID of the old client key:
+
+   ```shell
+   curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/<your_project_number>/error_tracking/client_keys"
+   ```
+
+1. Delete the old client key with the following command:
+
+   ```shell
+   curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/<your_project_number>/error_tracking/client_keys/<key_id>"
+   ```