diff --git a/doc/architecture/blueprints/clickhouse_usage/index.md b/doc/architecture/blueprints/clickhouse_usage/index.md index 6623ebbaa7f999b4e1688b999d739a72389f7c91..099f228c76b8970b6e2a8c92599fba1d1caac722 100644 --- a/doc/architecture/blueprints/clickhouse_usage/index.md +++ b/doc/architecture/blueprints/clickhouse_usage/index.md @@ -47,7 +47,7 @@ In FY24 Q2 we began working to integrate ClickHouse with GitLab.com to support m #### FY25 H1 (current) -After we have formulated best practices of managing ClickHouse ourselves for GitLab.com, we will begin to offer supported recommendations for self-managed instances that want to run ClickHouse themselves. During this phase we will allow users to "Bring your own ClickHouse" similar to our [approach for Elasticsearch](../../../integration/advanced_search/elasticsearch.md#install-elasticsearch). For the features that require ClickHouse for optimal usage (Value Streams Dashboard, [Product Analytics](https://gitlab.com/groups/gitlab-org/-/epics/8921)), this will be the initial go-to-market action. Notably, the Observability team has made the decision to support self-managed users via GitLab Cloud Connector instead of following this approach. +After we have formulated best practices of managing ClickHouse ourselves for GitLab.com, we will begin to offer supported recommendations for self-managed instances that want to run ClickHouse themselves. During this phase we will allow users to "Bring your own ClickHouse" similar to our [approach for Elasticsearch](../../../integration/advanced_search/elasticsearch.md#install-elasticsearch-or-aws-opensearch-cluster). For the features that require ClickHouse for optimal usage (Value Streams Dashboard, [Product Analytics](https://gitlab.com/groups/gitlab-org/-/epics/8921)), this will be the initial go-to-market action. Notably, the Observability team has made the decision to support self-managed users via GitLab Cloud Connector instead of following this approach. #### Long-term diff --git a/doc/architecture/blueprints/gitlab_rag/elasticsearch.md b/doc/architecture/blueprints/gitlab_rag/elasticsearch.md index e1cceda9c03d829e83aee55a930c8a7cc86d21cb..3d721fd35d42c6428738e7f1f1ec1f62ea6b3078 100644 --- a/doc/architecture/blueprints/gitlab_rag/elasticsearch.md +++ b/doc/architecture/blueprints/gitlab_rag/elasticsearch.md @@ -89,7 +89,7 @@ Elasticsearch currently supports [Reciprocal rank fusion (RRF)](https://www.elas Elasticsearch is available on GitLab.com and can be integrated on Dedicated and Self-Managed instances. To use as a vector store only: -- [Install Elasticsearch version `8.12`](../../../integration/advanced_search/elasticsearch.md#install-elasticsearch) or upgrade to at least version `8.12`. +- [Install Elasticsearch version `8.12`](../../../integration/advanced_search/elasticsearch.md#install-elasticsearch-or-aws-opensearch-cluster) or upgrade to at least version `8.12`. - Add URL, Username and Password on the Advanced Search settings page: `admin/application_settings/advanced_search` After the integration is configured, instance admins don't need to do further work to use it as a vector store since the GitLab Elasticsearch framework handles setting mappings, settings and indexing data. diff --git a/doc/development/adding_service_component.md b/doc/development/adding_service_component.md index 938a1db290c30d677bbe1709b48b0e1bc6668e59..6d2bf90317d501441ab79efda2410d4d97251ac2 100644 --- a/doc/development/adding_service_component.md +++ b/doc/development/adding_service_component.md @@ -48,7 +48,7 @@ In addition, it needs to cover the following: The first iteration should be to add the ability to connect and use the service as an externally installed component. Often this involves providing settings in GitLab to connect to the service, or allow connections from it. And then shipping documentation on how to install and configure the service with GitLab. -[Elasticsearch](../integration/advanced_search/elasticsearch.md#install-elasticsearch) is an example of a service that has been integrated this way. Many of the other services, including internal projects like Gitaly, started off as separately installed alternatives. +[Elasticsearch](../integration/advanced_search/elasticsearch.md#install-elasticsearch-or-aws-opensearch-cluster) is an example of a service that has been integrated this way. Many of the other services, including internal projects like Gitaly, started off as separately installed alternatives. **For services that depend on the existing GitLab codebase:** diff --git a/doc/integration/advanced_search/elasticsearch.md b/doc/integration/advanced_search/elasticsearch.md index 556a04a4f5a6df6be2204b7ae2f49674e96f0f9f..91a4315b645c3bb0d6152d6db4c79d565f245b3b 100644 --- a/doc/integration/advanced_search/elasticsearch.md +++ b/doc/integration/advanced_search/elasticsearch.md @@ -50,27 +50,174 @@ Elasticsearch requires additional resources to those documented in the Memory, CPU, and storage resource amounts vary depending on the amount of data you index into the Elasticsearch cluster. Heavily used Elasticsearch clusters may require more resources. The [`estimate_cluster_size`](#gitlab-advanced-search-rake-tasks) Rake task uses the total repository size to estimate the advanced search storage requirements. -## Install Elasticsearch +## Install Elasticsearch or AWS OpenSearch cluster -Elasticsearch is *not* included in the Linux package or when you self-compile your installation. -You must [install it separately](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/install-elasticsearch.html "Elasticsearch 7.x installation documentation") and ensure you select your version. Detailed information on how to install Elasticsearch is out of the scope of this page. +Elasticsearch and AWS OpenSearch are **not** included in the Linux package or when you perform a direct package installation. Detailed information on how to install Elasticsearch is out of the scope of this page. -You can install Elasticsearch yourself, or use a cloud hosted offering such as [Elasticsearch Service](https://www.elastic.co/elasticsearch/service) (available on AWS, GCP, or Azure) or the [Amazon OpenSearch](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/gsg.html) +You can install a search cluster yourself, or use a cloud hosted offering such as [Elasticsearch Service](https://www.elastic.co/elasticsearch/service) (available on AWS, GCP, or Azure) or the [Amazon OpenSearch](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/gsg.html) service. -You should install Elasticsearch on a separate server. Running Elasticsearch on the same server as GitLab is not recommended and can cause a degradation in GitLab instance performance. +You should install the search cluster on a separate server. Running the search cluster on the same server as GitLab is not recommended and can cause a degradation in GitLab instance performance. -For a single node Elasticsearch cluster, the functional cluster health status is always yellow due to the allocation of the primary shard. Elasticsearch cannot assign replica shards to the same node as primary shards. +For a single node search cluster, the functional cluster health status is always yellow due to the allocation of the primary shard. The cluster cannot assign replica shards to the same node as primary shards. The search index updates after you: - Add data to the database or repository. -- [Enable Elasticsearch](#enable-advanced-search) in the Admin Area. +- [Enable advanced search](#enable-advanced-search) in the Admin Area. NOTE: Before you use a new Elasticsearch cluster in production, see the [Elasticsearch documentation on important settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html). +### Elasticsearch access control configuration + +Elasticsearch offers role based access control to secure the cluster. To access and perform operations in the +Elasticsearch cluster, the `Username` configured in the Admin UI must have role(s) assigned that grant the following +privileges. The `Username` makes requests from GitLab to the search cluster. + +For more information, +see [Elasticsearch role based access control](https://www.elastic.co/guide/en/elasticsearch/reference/current/authorization.html#roles) +and [Elasticsearch security privileges](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html). + +```json +{ + "cluster": ["monitor"], + "indices": [ + { + "names": ["gitlab-*"], + "privileges": [ + "create_index", + "delete_index", + "view_index_metadata", + "read", + "manage", + "write" + ] + } + ] +} +``` + +### AWS OpenSearch service configuration + +AWS OpenSearch offers multiple methods of access control which are supported by GitLab: + +- [Domain level access policy](#domain-level-access-policy-configuration) +- Fine-grained access control + - [With IAM ARN as master user](#connecting-with-an-iam-user) + - [With master user](#connecting-with-a-master-user-in-the-internal-database) + +For more details on fine-grained access control see +[recommended configurations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html#fgac-recommendations) + +#### Domain level access policy configuration + +Configure the AWS OpenSearch domain access policy to allow `es:ESHttp*` actions. You can customize +the following example configuration to limit principals or resources: + +NOTE: +All `es:ESHttp` actions are required by GitLab. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": [ + "*" + ] + }, + "Action": [ + "es:ESHttp*" + ], + "Resource": "arn:aws:es:REGION:AWS_ACCOUNT_ID:domain/DOMAIN_NAME/*" + } + ] +} +``` + +For more information, +see [Identity and Access Management in Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html). + +##### Service linked role configuration + +The GitLab Rails and Sidekiq nodes require permissions to communicate with the search cluster. + +Create an IAM role with the following options and attach the role to the GitLab Rails and Sidekiq nodes: + +- Trusted entity type: `AWS Service` for `EC2` service +- Permission policy: `AmazonOpenSearchServiceFullAccess` + +##### Connecting with a domain level access policy only + +When using a domain level access policy, you must check the box **Use AWS OpenSearch Service with IAM credentials** and +fill in **AWS region** while leaving **AWS Access Key** and **AWS Secret Access Key** blank in the advanced search settings. + +NOTE: +Domain level access policy can be used standalone or in addition to fine-grained access control policies. + +#### Fine-grained access control configuration + +To access and perform operations in the AWS OpenSearch cluster, the user in **Username** must have role(s) assigned that +grant the following privileges. This user makes requests from GitLab to the search cluster. + +For more information, +see [OpenSearch access control permissions](https://opensearch.org/docs/latest/security/access-control/permissions/) +and [Creating roles](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html#fgac-roles). + +NOTE: +The index pattern `*` requires a few permissions for Advanced search to work. + +```json +{ + "cluster_permissions": [ + "cluster_composite_ops", + "cluster_monitor" + ], + "index_permissions": [ + { + "index_patterns": [ + "gitlab*" + ], + "allowed_actions": [ + "data_access", + "manage_aliases", + "search", + "create_index", + "delete", + "manage" + ] + }, + { + "index_patterns": [ + "*" + ], + "allowed_actions": [ + "indices:admin/aliases/get", + "indices:monitor/stats" + ] + } + ] +} +``` + +##### Connecting with a master user in the internal database + +When using fine-grained access control with a user in the internal database, you should use HTTP basic +authentication to connect to AWS OpenSearch. You can provide the master username and password as part of the +AWS OpenSearch URL or in the **Username** and **Password** text boxes in the advanced search settings. See +[Tutorial: Internal user database and HTTP basic authentication](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac-walkthrough-basic.html) +for details. + +##### Connecting with an IAM user + +When using fine-grained access control with IAM credentials, you must check the box **Use AWS OpenSearch Service with +IAM credentials** in the **AWS OpenSearch IAM credentials** section in the advanced search settings. +Provide the **AWS region**, **AWS Access Key**, and **AWS Secret Access Key**. + ## Upgrade to a new Elasticsearch major version > - Support for Elasticsearch 6.8 [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/350275) in GitLab 15.0. @@ -282,113 +429,6 @@ Sidekiq performance. Return them to their default values if you see increased `s in your Sidekiq logs. For more information, see [issue 322147](https://gitlab.com/gitlab-org/gitlab/-/issues/322147). -### Access requirements - -#### Elasticsearch with role privileges - -To access and perform operations in Elasticsearch, GitLab requires a role with the following privileges: - -```json -{ - "cluster": ["monitor"], - "indices": [ - { - "names": ["gitlab-*"], - "privileges": [ - "create_index", - "delete_index", - "view_index_metadata", - "read", - "manage", - "write" - ] - } - ] -} -``` - -For more information, see [Elasticsearch security privileges](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html). - -#### AWS OpenSearch Service with fine-grained access control - -To use the self-managed AWS OpenSearch Service with GitLab using fine-grained access control, try one of the -[recommended configurations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html#fgac-recommendations). - -Configure your instance's domain access policies to allow `es:ESHttp*` actions. You can customize -the following example configuration to limit principals or resources: - -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { - "AWS": [ - "*" - ] - }, - "Action": [ - "es:ESHttp*" - ], - "Resource": "domain-arn/*" - } - ] -} -``` - -For more information, see [Identity and Access Management in Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html). - -##### Connecting with a master user in the internal database - -When using fine-grained access control with a user in the internal database, you should use HTTP basic -authentication to connect to OpenSearch. You can provide the master username and password as part of the -OpenSearch URL or in the **Username** and **Password** text boxes in the advanced search settings. See -[Tutorial: Internal user database and HTTP basic authentication](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac-walkthrough-basic.html) for details. - -##### Connecting with an IAM user - -When using fine-grained access control with IAM credentials, you can provide the credentials in the **AWS OpenSearch IAM credentials** section in the advanced search settings. - -##### Permissions for fine-grained access control - -The following permissions are required for advanced search. See [Creating roles](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html#fgac-roles) for details. - -```json -{ - "cluster_permissions": [ - "cluster_composite_ops", - "cluster_monitor" - ], - "index_permissions": [ - { - "index_patterns": [ - "gitlab*" - ], - "allowed_actions": [ - "data_access", - "manage_aliases", - "search", - "create_index", - "delete", - "manage" - ] - }, - { - "index_patterns": [ - "*" - ], - "allowed_actions": [ - "indices:admin/aliases/get", - "indices:monitor/stats" - ] - } - ] -} -``` - -The index pattern `*` requires a few permissions for advanced search to work. - ### Limit the amount of namespace and project data to index When you select the **Limit the amount of namespace and project data to index** diff --git a/doc/integration/advanced_search/elasticsearch_troubleshooting.md b/doc/integration/advanced_search/elasticsearch_troubleshooting.md index e5730db8a33384296cd78efd01fb46eefc0fa928..d4bd68133a7b19bb4e9c4de0c1efadf955b6ac26 100644 --- a/doc/integration/advanced_search/elasticsearch_troubleshooting.md +++ b/doc/integration/advanced_search/elasticsearch_troubleshooting.md @@ -143,6 +143,49 @@ There are a couple of ways to achieve that: ::Gitlab::CurrentSettings.search_using_elasticsearch?(scope: Project.find_by_full_path("/my-namespace/my-project")) ``` +## Troubleshooting access + +### User: anonymous is not authorized to perform: es:ESHttpGet + +When using a domain level access policy with AWS OpenSearch or Elasticsearch, the AWS role is not assigned to the +correct GitLab nodes. The GitLab Rails and Sidekiq nodes require permission to communicate with the search cluster. + +```plaintext +User: anonymous is not authorized to perform: es:ESHttpGet because no resource-based policy allows the es:ESHttpGet +action +``` + +To fix this, ensure the AWS role is assigned to the correct GitLab nodes. + +### Credential should be scoped to a valid region + +When using AWS authorization with Advanced search, the region must be valid. + +### No permissions for [indices:data/write/bulk] + +When using fine-grained access control with an IAM role or a role created using AWS OpenSearch Dashboards, you might +encounter the following error: + +```json +{ + "error": { + "root_cause": [ + { + "type": "security_exception", + "reason": "no permissions for [indices:data/write/bulk] and User [name=arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE, backend_roles=[arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE], requestedTenant=null]" + } + ], + "type": "security_exception", + "reason": "no permissions for [indices:data/write/bulk] and User [name=arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE, backend_roles=[arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE], requestedTenant=null]" + }, + "status": 403 +} +``` + +To fix this, you need +to [map the roles to users](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html#fgac-mapping) +in the AWS OpenSearch Dashboards. + ## Troubleshooting indexing Troubleshooting indexing issues can be tricky. It can pretty quickly go to either GitLab @@ -512,16 +555,6 @@ however, searches only surface results that can be viewed by the user. Advanced search honors all permission checks in the application by filtering out projects that a user does not have access to at search time. -### Role mapping when using fine-grained access control with AWS Elasticsearch or OpenSearch - -When using fine-grained access control with an IAM role or a role created using OpenSearch Dashboards, you might encounter the following error: - -```plaintext -{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for [indices:data/write/bulk] and User [name=arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE, backend_roles=[arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE], requestedTenant=null]"}],"type":"security_exception","reason":"no permissions for [indices:data/write/bulk] and User [name=arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE, backend_roles=[arn:aws:iam::xxx:role/INSERT_ROLE_NAME_HERE], requestedTenant=null]"},"status":403} -``` - -To fix this, you need to [map the roles to users](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html#fgac-mapping) in Kibana. - ## Elasticsearch workers overload Sidekiq In some cases, Elasticsearch cannot connect to GitLab anymore because: diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 72ab4bd30fe0b48073850e10b4c51a405e238d8c..5eec688797369ae56a3a746cfe10e11b10d0a4d0 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -109,7 +109,7 @@ DETAILS: **Tier:** Premium, Ultimate **Offering:** Self-managed -Follow the [install instruction](../integration/advanced_search/elasticsearch.md#install-elasticsearch). +Follow the [install instruction](../integration/advanced_search/elasticsearch.md#install-elasticsearch-or-aws-opensearch-cluster). ### 9. Start application diff --git a/doc/update/upgrading_from_ce_to_ee.md b/doc/update/upgrading_from_ce_to_ee.md index e7cdb5de9c701b44a0c2d75b3dd4b5a5c9af66bd..2afb4b466f9c8f482dee5afcbbcf76f4013ef4be 100644 --- a/doc/update/upgrading_from_ce_to_ee.md +++ b/doc/update/upgrading_from_ce_to_ee.md @@ -88,7 +88,7 @@ DETAILS: **Tier:** Premium, Ultimate **Offering:** Self-managed -Follow the [install instruction](../integration/advanced_search/elasticsearch.md#install-elasticsearch). +Follow the [install instruction](../integration/advanced_search/elasticsearch.md#install-elasticsearch-or-aws-opensearch-cluster). ### 5. Start application