Skip to content
代码片段 群组 项目
未验证 提交 cdb0b74c 编辑于 作者: Susie Bitters's avatar Susie Bitters 提交者: GitLab
浏览文件

Apply 1 suggestion(s) to 1 file(s)

上级 068181d1
No related branches found
No related tags found
无相关合并请求
...@@ -19,7 +19,7 @@ DETAILS: ...@@ -19,7 +19,7 @@ DETAILS:
To configure your GitLab instance to access the available self-hosted models in your infrastructure: To configure your GitLab instance to access the available self-hosted models in your infrastructure:
1. Use a [locally hosted or GitLab.com AI gateway](index.md#choose-a-configuration-type). 1. [Confirm that a fully self-hosted model configuration is appropriate for your use case](index.md#decide-on-your-configuration-type).
1. Configure your GitLab instance. 1. Configure your GitLab instance.
1. Configure the self-hosted model. 1. Configure the self-hosted model.
1. Configure the GitLab Duo features to use your self-hosted model. 1. Configure the GitLab Duo features to use your self-hosted model.
......
...@@ -19,15 +19,15 @@ DETAILS: ...@@ -19,15 +19,15 @@ DETAILS:
To maintain full control over your data privacy, security, and the deployment of large language models (LLMs) in your own infrastructure, use GitLab Duo Self-Hosted Models. To maintain full control over your data privacy, security, and the deployment of large language models (LLMs) in your own infrastructure, use GitLab Duo Self-Hosted Models.
By deploying self-hosted models, you can manage the entire lifecycle of requests made to LLM backends for GitLab Duo features, ensuring that all requests stay within your enterprise network and avoiding external dependencies. By deploying self-hosted models, you can manage the entire lifecycle of requests made to LLM backends for GitLab Duo features, ensuring that all requests stay in your enterprise network, and avoiding external dependencies.
## Why use self-hosted models ## Why use self-hosted models
With self-hosted models, you can: With self-hosted models, you can:
- Choose any GitLab-approved LLM. - Choose any GitLab-approved LLM.
- Retain full control over data by keeping all request/response logs within your domain, ensuring complete privacy and security with no external API calls. - Retain full control over data by keeping all request/response logs in your domain, ensuring complete privacy and security with no external API calls.
- Isolate the GitLab instance, AI gateway, and models within your own environment. - Isolate the GitLab instance, AI gateway, and models in your own environment.
- Select specific GitLab Duo features tailored to your users. - Select specific GitLab Duo features tailored to your users.
- Eliminate reliance on the shared GitLab AI gateway. - Eliminate reliance on the shared GitLab AI gateway.
...@@ -39,36 +39,49 @@ Before setting up a self-hosted model infrastructure, you must have: ...@@ -39,36 +39,49 @@ Before setting up a self-hosted model infrastructure, you must have:
- A [supported model](supported_models_and_hardware_requirements.md) (either cloud-based or on-premises). - A [supported model](supported_models_and_hardware_requirements.md) (either cloud-based or on-premises).
- A [supported serving platform](supported_llm_serving_platforms.md) (either cloud-based or on-premises). - A [supported serving platform](supported_llm_serving_platforms.md) (either cloud-based or on-premises).
- A locally hosted or GitLab.com AI gateway. - A [locally hosted AI gateway](../../install/install_ai_gateway.md).
- [Ultimate with GitLab Duo Enterprise](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?toggle=gitlab-duo-pro). - [Ultimate with GitLab Duo Enterprise](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?toggle=gitlab-duo-pro).
## Choose a configuration type ## Decide on your configuration type
There are two configuration options for self-managed customers: The configuration for self-hosted models is different to the default configuration
that uses GitLab external AI vendors.
- GitLab.com AI gateway with default GitLab external vendor LLMs NOTE:
- Self-hosted AI gateway and LLMs Both of the following configuration types are for self-managed GitLab instances.
Before setting up a self-hosted model infrastructure, you must decide which ### Self-hosted AI gateway and LLMs
configuration type to implement.
### GitLab.com AI gateway with default GitLab external vendor LLMs In a fully self-hosted configuration, you deploy your own AI gateway and LLMs in your infrastructure, without relying on external public services. This gives you full control over your data and security.
This is the default Enterprise offering and is not fully self-hosted. In this configuration, you connect your self-managed GitLab instance to the GitLab-hosted AI gateway, which integrates with external vendor LLM providers (such as Google Vertex or Anthropic). If you have an offline environment with physical barriers or security policies that prevent or limit internet access, and comprehensive LLM controls, you can use self-hosted models.
These LLMs communicate through the [GitLab Cloud Connector](../../development/cloud_connector/index.md), offering a ready-to-use AI solution without the need for on-premise infrastructure. For licensing, you must have a GitLab Ultimate subscription and GitLab Duo Enterprise. Offline Enterprise licenses are available for those customers with fully isolated offline environments. To get access to your purchased subscription, request a license through the [Customers Portal](../../subscriptions/customers_portal.md).
For licensing, you must have a GitLab Premium or Ultimate subscription and [GitLab Duo Enterprise](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?type=free-trial). For more information, see the
[self-hosted AI gateway configuration diagram](configuration_types.md#self-hosted-ai-gateway).
For more information, see the [GitLab.com AI gateway configuration diagram](configuration_types.md#gitlabcom-ai-gateway). To set up a self-hosted infrastructure, see
[Set up a self-hosted infrastructure](#set-up-a-self-hosted-infrastructure).
### Self-hosted AI gateway and LLMs ### GitLab.com AI gateway with default GitLab external vendor LLMs
If you do not meet the use case criteria for self-hosted models, you can use the
GitLab.com AI gateway with default GitLab external vendor LLMs.
The GitLab.com AI gateway is the default Enterprise offering and is not self-hosted. In this configuration,
you connect your self-managed GitLab instance to the GitLab-hosted AI gateway, which
integrates with external vendor LLM providers (such as Google Vertex or Anthropic).
These LLMs communicate through the [GitLab Cloud Connector](../../development/cloud_connector/index.md),
offering a ready-to-use AI solution without the need for on-premise infrastructure.
In this configuration, you deploy your own AI gateway and LLMs within your infrastructure, without relying on external public services. This gives you full control over your data and security. For licensing, you must have a GitLab Ultimate subscription, and either [GitLab Duo Pro](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?type=free-trial) or [GitLab Duo Enterprise](https://about.gitlab.com/solutions/gitlab-duo-pro/sales/?type=free-trial). To get access to your purchased subscription, request a license through the [Customers Portal](../../subscriptions/customers_portal.md)
For licensing, you must have a valid GitLab license. You can request a license through the [Customers Portal](../../subscriptions/customers_portal.md). For more information, see the
[GitLab.com AI gateway configuration diagram](configuration_types.md#gitlabcom-ai-gateway).
For more information, see the [self-hosted AI gateway configuration diagram](configuration_types.md#self-hosted-ai-gateway). To set up this infrastructure, see [how to configure GitLab Duo on a self-managed instance](../../user/gitlab_duo/setup.md).
## Set up a self-hosted infrastructure ## Set up a self-hosted infrastructure
...@@ -87,7 +100,7 @@ To set up a fully isolated self-hosted model infrastructure: ...@@ -87,7 +100,7 @@ To set up a fully isolated self-hosted model infrastructure:
See the [Configure GitLab Duo features documentation](configure_duo_features.md) for instructions on how to customize your environment to effectively meet your operational needs. See the [Configure GitLab Duo features documentation](configure_duo_features.md) for instructions on how to customize your environment to effectively meet your operational needs.
1. **Enable logging** 1. **Enable logging**
You can find configuration details for enabling logging within your environment. For help in using logs to track and manage your system's performance effectively, see the [logging documentation](logging.md). You can find configuration details for enabling logging in your environment. For help in using logs to track and manage your system's performance effectively, see the [logging documentation](logging.md).
## Related topics ## Related topics
......
0% 加载中 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册