From 4ba995448ecd920a510a32e044896df0ba41030a Mon Sep 17 00:00:00 2001 From: Evan Read <eread@gitlab.com> Date: Tue, 5 Dec 2023 00:13:03 +0000 Subject: [PATCH] Unify Gitaly timeout information --- doc/administration/gitaly/configure_gitaly.md | 47 ------------- .../settings/gitaly_timeouts.md | 70 ++++++++++++++++--- .../project/repository/monorepos/index.md | 2 +- 3 files changed, 62 insertions(+), 57 deletions(-) diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md index 891416a7debb..535e35cc0127 100644 --- a/doc/administration/gitaly/configure_gitaly.md +++ b/doc/administration/gitaly/configure_gitaly.md @@ -1542,50 +1542,3 @@ go_cloud_url = "s3://gitaly-backups?region=minio&endpoint=my.minio.local:8080&di ``` ::EndTabs - -## Configure negotiation timeouts - -> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/5574) in GitLab 16.5. - -Gitaly supports configurable negotiation timeouts. - -Negotiation timeouts can be configured for the `git-upload-pack(1)` and `git-upload-archive(1)` -operations, which are invoked by a Gitaly node when you execute the `git fetch` and -`git archive --remote` commands respectively. You might need to increase the negotiation timeout: - -- For particularly large repositories. -- When performing these commands in parallel. - -These timeouts affect only the [negotiation phase](https://git-scm.com/docs/pack-protocol/2.2.3#_packfile_negotiation) of -remote Git operations, not the entire transfer. - -Valid values for timeouts follow the format of [`ParseDuration`](https://pkg.go.dev/time#ParseDuration) in Go. - -How you configure negotiation timeouts depends on the type of installation you have: - -::Tabs - -:::TabTitle Linux package (Omnibus) - -Edit `/etc/gitlab/gitlab.rb`: - -```ruby -gitaly['configuration'] = { - timeout: { - upload_pack_negotiation: '10m', # 10 minutes - upload_archive_negotiation: '20m', # 20 minutes - } -} -``` - -:::TabTitle Self-compiled (source) - -Edit `/home/git/gitaly/config.toml`: - -```toml -[timeout] -upload_pack_negotiation = "10m" -upload_archive_negotiation = "20m" -``` - -::EndTabs diff --git a/doc/administration/settings/gitaly_timeouts.md b/doc/administration/settings/gitaly_timeouts.md index b60cf141cf62..93db0467d701 100644 --- a/doc/administration/settings/gitaly_timeouts.md +++ b/doc/administration/settings/gitaly_timeouts.md @@ -6,23 +6,75 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Gitaly timeouts **(FREE SELF)** -[Gitaly](../gitaly/index.md) timeouts are configurable. The timeouts can be -configured to make sure that long-running Gitaly calls don't needlessly take up resources. +[Gitaly](../gitaly/index.md) provides two types of configurable timeouts: -To access Gitaly timeout settings: +- Call timeouts, configured by using the GitLab UI. +- Negotiation timeouts, configured by using Gitaly configuration files. + +## Configure the call timeouts + +Configure the following call timeouts to make sure that long-running Gitaly calls don't needlessly take up resources. To +configure the call timeouts: 1. On the left sidebar, at the bottom, select **Admin Area**. 1. Select **Settings > Preferences**. 1. Expand the **Gitaly timeouts** section. +1. Set each timeout as required. -## Available timeouts +### Available call timeouts -The following timeouts are available. +Different call timeouts are available for different Gitaly operations. | Timeout | Default | Description | |:--------|:-----------|:------------| -| Default | 55 seconds | Timeout for most Gitaly calls (not enforced for `git` `fetch` and `push` operations, or Sidekiq jobs). For example, checking if a repository exists on disk. Makes sure that Gitaly calls made within a web request cannot exceed the entire request timeout. It should be shorter than the [worker timeout](../operations/puma.md#change-the-worker-timeout) that can be configured for [Puma](../../install/requirements.md#puma-settings). If a Gitaly call timeout exceeds the worker timeout, the remaining time from the worker timeout is used to avoid having to terminate the worker. | -| Fast | 10 seconds | Timeout for fast Gitaly operations used within requests, sometimes multiple times. For example, checking if a repository exists on disk. If fast operations exceed this threshold, there may be a problem with a storage shard. Failing fast can help maintain the stability of the GitLab instance. | -| Medium | 30 seconds | Timeout for Gitaly operations that should be fast (possibly within requests) but preferably not used multiple times within a request. For example, loading blobs. Timeout that should be set between Default and Fast. | +| Default | 55 seconds | Timeout for most Gitaly calls (not enforced for `git` `fetch` and `push` operations, or Sidekiq jobs). For example, checking if a repository exists on disk. Makes sure that Gitaly calls made in a web request cannot exceed the entire request timeout. It should be shorter than the [worker timeout](../operations/puma.md#change-the-worker-timeout) that can be configured for [Puma](../../install/requirements.md#puma-settings). If a Gitaly call timeout exceeds the worker timeout, the remaining time from the worker timeout is used to avoid having to terminate the worker. | +| Fast | 10 seconds | Timeout for fast Gitaly operations used in requests, sometimes multiple times. For example, checking if a repository exists on disk. If fast operations exceed this threshold, there may be a problem with a storage shard. Failing fast can help maintain the stability of the GitLab instance. | +| Medium | 30 seconds | Timeout for Gitaly operations that should be fast (possibly in requests) but preferably not used multiple times in a request. For example, loading blobs. Timeout that should be set between Default and Fast. | + +## Configure the negotiation timeouts + +> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/5574) in GitLab 16.5. + +You might need to increase the negotiation timeout: + +- For particularly large repositories. +- When performing these commands in parallel. + +You can configure negotiation timeouts for: + +- `git-upload-pack(1)`, which is invoked by a Gitaly node when you execute `git fetch`. +- `git-upload-archive(1)`, which is invoked by a Gitaly node when you execute `git archive --remote`. + +To configure these timeouts: + +::Tabs + +:::TabTitle Linux package (Omnibus) + +Edit `/etc/gitlab/gitlab.rb`: + +```ruby +gitaly['configuration'] = { + timeout: { + upload_pack_negotiation: '10m', # 10 minutes + upload_archive_negotiation: '20m', # 20 minutes + } +} +``` + +:::TabTitle Self-compiled (source) + +Edit `/home/git/gitaly/config.toml`: + +```toml +[timeout] +upload_pack_negotiation = "10m" +upload_archive_negotiation = "20m" +``` + +::EndTabs + +For the values, use the format of [`ParseDuration`](https://pkg.go.dev/time#ParseDuration) in Go. -You can also [configure negotiation timeouts](../gitaly/configure_gitaly.md#configure-negotiation-timeouts). +These timeouts affect only the [negotiation phase](https://git-scm.com/docs/pack-protocol/2.2.3#_packfile_negotiation) of +remote Git operations, not the entire transfer. diff --git a/doc/user/project/repository/monorepos/index.md b/doc/user/project/repository/monorepos/index.md index 144f46cd7d52..b73757a952e5 100644 --- a/doc/user/project/repository/monorepos/index.md +++ b/doc/user/project/repository/monorepos/index.md @@ -189,7 +189,7 @@ You might experience a `fatal: the remote end hung up unexpectedly` error when a - The same large repository in parallel. You can attempt to mitigate this issue by increasing the default negotiation timeout values. For more information, see -[Configure negotiation timeouts](../../../../administration/gitaly/configure_gitaly.md#configure-negotiation-timeouts). +[Configure the negotiation timeouts](../../../../administration/settings/gitaly_timeouts.md#configure-the-negotiation-timeouts). ## Optimize your repository -- GitLab