diff --git a/doc/administration/self_hosted_models/configure_duo_features.md b/doc/administration/self_hosted_models/configure_duo_features.md index 291444baa0ac28827f7e5203fe2555c0d28d69d6..50f19082166102616e72c510590c86e311c1e8ae 100644 --- a/doc/administration/self_hosted_models/configure_duo_features.md +++ b/doc/administration/self_hosted_models/configure_duo_features.md @@ -19,7 +19,7 @@ DETAILS: To configure your GitLab instance to access the available self-hosted models in your infrastructure: -1. Use a [locally hosted or GitLab.com AI gateway](index.md#choose-a-configuration-type). +1. [Confirm that a fully self-hosted model configuration is appropriate for your use case](index.md#decide-on-your-configuration-type). 1. Configure your GitLab instance. 1. Configure the self-hosted model. 1. Configure the GitLab Duo features to use your self-hosted model. diff --git a/doc/administration/self_hosted_models/index.md b/doc/administration/self_hosted_models/index.md index 6e84966fe9f08896991c6ea787e583ab636281c6..3f17dad495d77c5923ba409d1d22b6d12581587b 100644 --- a/doc/administration/self_hosted_models/index.md +++ b/doc/administration/self_hosted_models/index.md @@ -19,15 +19,15 @@ DETAILS: To maintain full control over your data privacy, security, and the deployment of large language models (LLMs) in your own infrastructure, use GitLab Duo Self-Hosted Models. -By deploying self-hosted models, you can manage the entire lifecycle of requests made to LLM backends for GitLab Duo features, ensuring that all requests stay within your enterprise network and avoiding external dependencies. +By deploying self-hosted models, you can manage the entire lifecycle of requests made to LLM backends for GitLab Duo features, ensuring that all requests stay in your enterprise network, and avoiding external dependencies. ## Why use self-hosted models With self-hosted models, you can: - Choose any GitLab-approved LLM. -- Retain full control over data by keeping all request/response logs within your domain, ensuring complete privacy and security with no external API calls. -- Isolate the GitLab instance, AI gateway, and models within your own environment. +- Retain full control over data by keeping all request/response logs in your domain, ensuring complete privacy and security with no external API calls. +- Isolate the GitLab instance, AI gateway, and models in your own environment. - Select specific GitLab Duo features tailored to your users. - Eliminate reliance on the shared GitLab AI gateway. @@ -39,36 +39,49 @@ Before setting up a self-hosted model infrastructure, you must have: - A [supported model](supported_models_and_hardware_requirements.md) (either cloud-based or on-premises). - A [supported serving platform](supported_llm_serving_platforms.md) (either cloud-based or on-premises). -- A locally hosted or GitLab.com AI gateway. +- A [locally hosted AI gateway](../../install/install_ai_gateway.md). - [Ultimate with GitLab Duo Enterprise](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?toggle=gitlab-duo-pro). -## Choose a configuration type +## Decide on your configuration type -There are two configuration options for self-managed customers: +The configuration for self-hosted models is different to the default configuration +that uses GitLab external AI vendors. -- GitLab.com AI gateway with default GitLab external vendor LLMs -- Self-hosted AI gateway and LLMs +NOTE: +Both of the following configuration types are for self-managed GitLab instances. -Before setting up a self-hosted model infrastructure, you must decide which -configuration type to implement. +### Self-hosted AI gateway and LLMs -### GitLab.com AI gateway with default GitLab external vendor LLMs +In a fully self-hosted configuration, you deploy your own AI gateway and LLMs in your infrastructure, without relying on external public services. This gives you full control over your data and security. -This is the default Enterprise offering and is not fully self-hosted. In this configuration, you connect your self-managed GitLab instance to the GitLab-hosted AI gateway, which integrates with external vendor LLM providers (such as Google Vertex or Anthropic). +If you have an offline environment with physical barriers or security policies that prevent or limit internet access, and comprehensive LLM controls, you can use self-hosted models. -These LLMs communicate through the [GitLab Cloud Connector](../../development/cloud_connector/index.md), offering a ready-to-use AI solution without the need for on-premise infrastructure. +For licensing, you must have a GitLab Ultimate subscription and GitLab Duo Enterprise. Offline Enterprise licenses are available for those customers with fully isolated offline environments. To get access to your purchased subscription, request a license through the [Customers Portal](../../subscriptions/customers_portal.md). -For licensing, you must have a GitLab Premium or Ultimate subscription and [GitLab Duo Enterprise](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?type=free-trial). +For more information, see the +[self-hosted AI gateway configuration diagram](configuration_types.md#self-hosted-ai-gateway). -For more information, see the [GitLab.com AI gateway configuration diagram](configuration_types.md#gitlabcom-ai-gateway). +To set up a self-hosted infrastructure, see +[Set up a self-hosted infrastructure](#set-up-a-self-hosted-infrastructure). -### Self-hosted AI gateway and LLMs +### GitLab.com AI gateway with default GitLab external vendor LLMs + +If you do not meet the use case criteria for self-hosted models, you can use the +GitLab.com AI gateway with default GitLab external vendor LLMs. + +The GitLab.com AI gateway is the default Enterprise offering and is not self-hosted. In this configuration, +you connect your self-managed GitLab instance to the GitLab-hosted AI gateway, which +integrates with external vendor LLM providers (such as Google Vertex or Anthropic). + +These LLMs communicate through the [GitLab Cloud Connector](../../development/cloud_connector/index.md), +offering a ready-to-use AI solution without the need for on-premise infrastructure. -In this configuration, you deploy your own AI gateway and LLMs within your infrastructure, without relying on external public services. This gives you full control over your data and security. +For licensing, you must have a GitLab Ultimate subscription, and either [GitLab Duo Pro](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?type=free-trial) or [GitLab Duo Enterprise](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?type=free-trial). To get access to your purchased subscription, request a license through the [Customers Portal](../../subscriptions/customers_portal.md) -For licensing, you must have a valid GitLab license. You can request a license through the [Customers Portal](../../subscriptions/customers_portal.md). +For more information, see the +[GitLab.com AI gateway configuration diagram](configuration_types.md#gitlabcom-ai-gateway). -For more information, see the [self-hosted AI gateway configuration diagram](configuration_types.md#self-hosted-ai-gateway). +To set up this infrastructure, see [how to configure GitLab Duo on a self-managed instance](../../user/gitlab_duo/setup.md). ## Set up a self-hosted infrastructure @@ -87,7 +100,7 @@ To set up a fully isolated self-hosted model infrastructure: See the [Configure GitLab Duo features documentation](configure_duo_features.md) for instructions on how to customize your environment to effectively meet your operational needs. 1. **Enable logging** - You can find configuration details for enabling logging within your environment. For help in using logs to track and manage your system's performance effectively, see the [logging documentation](logging.md). + You can find configuration details for enabling logging in your environment. For help in using logs to track and manage your system's performance effectively, see the [logging documentation](logging.md). ## Related topics