Skip to content
代码片段 群组 项目
未验证 提交 bd451a65 编辑于 作者: Jon Glassman's avatar Jon Glassman 提交者： GitLab
浏览文件

Merge branch 'migrate-aigw-setup-docs' into 'master' 

Migrate AI Gateway documentation

See merge request https://gitlab.com/gitlab-org/gitlab/-/merge_requests/168628



Merged-by: default avatarJon Glassman <jglassman@gitlab.com>
Approved-by: default avatarManoj M J <mmj@gitlab.com>
Approved-by: default avatarJon Glassman <jglassman@gitlab.com>
Reviewed-by: default avatarManoj M J <mmj@gitlab.com>
Co-authored-by: default avatarMohamed Hamda <mhamda@gitlab.com>
上级 a4bd9555 13e9597a
No related branches found
No related tags found
无相关合并请求
隐藏空白变更内容
行内 左右并排
doc/administration/self_hosted_models/install_infrastructure.md
+ 2
− 143
浏览文件 @ bd451a65
...@@ -91,145 +91,9 @@ Prerequisites: ...@@ -91,145 +91,9 @@ Prerequisites:
sudo gitlab-ctl reconfigure sudo gitlab-ctl reconfigure
``` ```
## Install the GitLab AI Gateway ## GitLab AI Gateway
### Install by using Docker [Install the GitLab AI Gateway](../../install/install_ai_gateway.md).
Prerequisites:
- Install a Docker container engine, such as [Docker](https://docs.docker.com/engine/install/#server).
- Use a valid hostname accessible within your network. Do not use `localhost`.
The GitLab AI Gateway Docker image contains all necessary code and dependencies
in a single container.
#### Find the AI Gateway release
Find the GitLab official Docker image at:
- [AI Gateway Docker image on Container Registry](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/container_registry/).
- [AI Gateway Docker image on DockerHub](https://hub.docker.com/repository/docker/gitlab/model-gateway/tags).
- [Release process for self-hosted AI Gateway](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/docs/release.md).
Use the image tag that corresponds to your GitLab version. For example, if the
GitLab version is `v17.5.0`, use `self-hosted-v17.5.0-ee` tag.
#### Start a container from the image
For Docker images with version `self-hosted-17.4.0-ee` and later, run the following:
```shell
docker run -p 5052:5052 \
-e AIGW_GITLAB_URL=<your_gitlab_instance> \
-e AIGW_GITLAB_API_URL=https://<your_gitlab_domain>/api/v4/ \
<image>
```
From the container host, accessing `http://localhost:5052/docs`
should open the AI Gateway API documentation.
### Install by using the AI Gateway Helm chart
Prerequisites:
- You must have a:
- Domain you own, that you can add a DNS record to.
- Kubernetes cluster.
- Working installation of `kubectl`.
- Working installation of Helm, version v3.11.0 or later.
For more information, see [Test the GitLab chart on GKE or EKS](https://docs.gitlab.com/charts/quickstart/index.html).
#### Add the AI Gateway Helm repository
Add the AI Gateway Helm repository to Helm’s configuration:
```shell
helm repo add ai-gateway \
https://gitlab.com/api/v4/projects/gitlab-org%2fcharts%2fai-gateway-helm-chart/packages/helm/devel
```
#### Install the AI Gateway
1. Create the `ai-gateway` namespace:
```shell
kubectl create namespace ai-gateway
```
1. Generate the certificate for the domain where you plan to expose the AI Gateway.
1. Create the TLS secret in the previously created namespace:
```shell
kubectl -n ai-gateway create secret tls ai-gateway-tls --cert="<path_to_cert>" --key="<path_to_cert_key>"
```
1. For the AI Gateway to access the API, it must know where the GitLab instance
is located. To do this, set the `gitlab.url` and `gitlab.apiUrl` together with
the `ingress.hosts` and `ingress.tls` values as follows:
```shell
helm repo add ai-gateway \
https://gitlab.com/api/v4/projects/gitlab-org%2fcharts%2fai-gateway-helm-chart/packages/helm/devel
helm repo update
helm upgrade --install ai-gateway \
ai-gateway/ai-gateway \
--version 0.1.1 \
--namespace=ai-gateway \
--set="image.tag=<ai-gateway-image>" \
--set="gitlab.url=https://<your_gitlab_domain>" \
--set="gitlab.apiUrl=https://<your_gitlab_domain>/api/v4/" \
--set "ingress.enabled=true" \
--set "ingress.hosts[0].host=<your_gateway_domain>" \
--set "ingress.hosts[0].paths[0].path=/" \
--set "ingress.hosts[0].paths[0].pathType=ImplementationSpecific" \
--set "ingress.tls[0].secretName=ai-gateway-tls" \
--set "ingress.tls[0].hosts[0]=<your_gateway_domain>" \
--set="ingress.className=nginx" \
--timeout=300s --wait --wait-for-jobs
```
This step can take will take a few seconds in order for all resources to be allocated
and the AI Gateway to start.
Wait for your pods to get up and running:
```shell
kubectl wait pod \
--all \
--for=condition=Ready \
--namespace=ai-gateway \
--timeout=300s
```
When your pods are up and running, you can set up your IP ingresses and DNS records.
#### Configure the GitLab instance
[Configure the GitLab instance](#configure-your-gitlab-instance).
With those steps completed, your Helm chart installation is complete.
## Upgrade the AI Gateway Docker image
To upgrade the AI Gateway, download the newest Docker image tag.
1. Stop the running container:
```shell
sudo docker stop gitlab-aigw
```
1. Remove the existing container:
```shell
sudo docker rm gitlab-aigw
```
1. Pull and [run the new image](#start-a-container-from-the-image).
1. Ensure that the environment variables are all set correctly.
## Enable logging ## Enable logging
...@@ -297,11 +161,6 @@ request. For more information, see the [documentation on finding logs with a cor ...@@ -297,11 +161,6 @@ request. For more information, see the [documentation on finding logs with a cor
Correlation ID is not available in your model provider logs. Correlation ID is not available in your model provider logs.
## Alternative installation methods
For information on alternative ways to install the AI Gateway, see
[issue 463773](https://gitlab.com/gitlab-org/gitlab/-/issues/463773).
## Troubleshooting ## Troubleshooting
First, run the [debugging scripts](troubleshooting.md#use-debugging-scripts) to First, run the [debugging scripts](troubleshooting.md#use-debugging-scripts) to
......
doc/administration/self_hosted_models/troubleshooting.md
+ 2
− 2
浏览文件 @ bd451a65
...@@ -178,7 +178,7 @@ echo $AIGW_CUSTOM_MODELS__ENABLED # must be true ...@@ -178,7 +178,7 @@ echo $AIGW_CUSTOM_MODELS__ENABLED # must be true
``` ```
If the environmental variables are not set up correctly, set them by If the environmental variables are not set up correctly, set them by
[creating a container](install_infrastructure.md#find-the-ai-gateway-release). [creating a container](../../install/install_ai_gateway.md#find-the-ai-gateway-release).
## Check if the model is reachable from AI Gateway ## Check if the model is reachable from AI Gateway
...@@ -193,7 +193,7 @@ To resolve this, contact your network administrator. ...@@ -193,7 +193,7 @@ To resolve this, contact your network administrator.
## The image's platform does not match the host ## The image's platform does not match the host
When [finding the AI Gateway release](install_infrastructure.md#find-the-ai-gateway-release), When [finding the AI Gateway release](../../install/install_ai_gateway.md#find-the-ai-gateway-release),
you might get an error that states `The requested image’s platform (linux/amd64) does not match the detected host`. you might get an error that states `The requested image’s platform (linux/amd64) does not match the detected host`.
To work around this error, add `--platform linux/amd64` to the `docker run` command: To work around this error, add `--platform linux/amd64` to the `docker run` command:
......
doc/install/index.md
+ 1
− 1
浏览文件 @ bd451a65
...@@ -18,6 +18,6 @@ ease of administration (backups, upgrades, and troubleshooting) with the cost of ...@@ -18,6 +18,6 @@ ease of administration (backups, upgrades, and troubleshooting) with the cost of
|--|--|--| |--|--|--|
| [**Installation requirements**](requirements.md)<br>Prerequisites for installation. | [**Installation methods**](install_methods.md)<br>Linux, Helm, Docker, Operator, source, or scripts. | [**Install GitLab on a cloud provider**](cloud_providers.md)<br>AWS, Google Cloud Platform, Azure. | | [**Installation requirements**](requirements.md)<br>Prerequisites for installation. | [**Installation methods**](install_methods.md)<br>Linux, Helm, Docker, Operator, source, or scripts. | [**Install GitLab on a cloud provider**](cloud_providers.md)<br>AWS, Google Cloud Platform, Azure. |
| [**Offline GitLab**](../topics/offline/index.md)<br>Isolated installation. | [**Reference architectures**](../administration/reference_architectures/index.md)<br>Recommended deployments at scale. | [**Upgrade GitLab**](../update/index.md)<br>Latest version instructions. | | [**Offline GitLab**](../topics/offline/index.md)<br>Isolated installation. | [**Reference architectures**](../administration/reference_architectures/index.md)<br>Recommended deployments at scale. | [**Upgrade GitLab**](../update/index.md)<br>Latest version instructions. |
| [**Install GitLab Runner**](https://docs.gitlab.com/runner/install/)<br>Software for CI/CD jobs. | [**Configure GitLab Runner**](https://docs.gitlab.com/runner/configuration/)<br>`Config.toml`, certificates, autoscaling, proxy setup. | | | [**Install GitLab Runner**](https://docs.gitlab.com/runner/install/)<br>Software for CI/CD jobs. | [**Configure GitLab Runner**](https://docs.gitlab.com/runner/configuration/)<br>`Config.toml`, certificates, autoscaling, proxy setup. | [**Install GitLab AI Gateway**](https://docs.gitlab.com/runner/configuration/)<br>Using Docker image, Using Helm chart. |
<!-- markdownlint-enable MD044 --> <!-- markdownlint-enable MD044 -->
doc/install/install_ai_gateway.md 0 → 100644
+ 149
− 0
浏览文件 @ bd451a65
---
stage: AI-Powered
group: AI Framework
description: Set up your self-hosted model GitLab AI Gateway
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# GitLab AI Gateway
The [AI gateway](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/ai_gateway/) is a standalone service that gives access to AI-powered GitLab Duo features.
## Install the GitLab AI Gateway
### Install by using Docker
Prerequisites:
- Install a Docker container engine, such as [Docker](https://docs.docker.com/engine/install/#server).
- Use a valid hostname accessible within your network. Do not use `localhost`.
The GitLab AI Gateway Docker image contains all necessary code and dependencies
in a single container.
#### Find the AI Gateway release
Find the GitLab official Docker image at:
- [AI Gateway Docker image on Container Registry](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/container_registry/).
- [AI Gateway Docker image on DockerHub](https://hub.docker.com/repository/docker/gitlab/model-gateway/tags).
- [Release process for self-hosted AI Gateway](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/docs/release.md).
Use the image tag that corresponds to your GitLab version. For example, if the
GitLab version is `v17.5.0`, use `self-hosted-v17.5.0-ee` tag.
#### Start a container from the image
For Docker images with version `self-hosted-17.4.0-ee` and later, run the following:
```shell
docker run -p 5052:5052 \
-e AIGW_GITLAB_URL=<your_gitlab_instance> \
-e AIGW_GITLAB_API_URL=https://<your_gitlab_domain>/api/v4/ \
<image>
```
From the container host, accessing `http://localhost:5052/docs`
should open the AI Gateway API documentation.
### Install by using the AI Gateway Helm chart
Prerequisites:
- You must have a:
- Domain you own, that you can add a DNS record to.
- Kubernetes cluster.
- Working installation of `kubectl`.
- Working installation of Helm, version v3.11.0 or later.
For more information, see [Test the GitLab chart on GKE or EKS](https://docs.gitlab.com/charts/quickstart/index.html).
#### Add the AI Gateway Helm repository
Add the AI Gateway Helm repository to Helm’s configuration:
```shell
helm repo add ai-gateway \
https://gitlab.com/api/v4/projects/gitlab-org%2fcharts%2fai-gateway-helm-chart/packages/helm/devel
```
#### Install the AI Gateway
1. Create the `ai-gateway` namespace:
```shell
kubectl create namespace ai-gateway
```
1. Generate the certificate for the domain where you plan to expose the AI Gateway.
1. Create the TLS secret in the previously created namespace:
```shell
kubectl -n ai-gateway create secret tls ai-gateway-tls --cert="<path_to_cert>" --key="<path_to_cert_key>"
```
1. For the AI Gateway to access the API, it must know where the GitLab instance
is located. To do this, set the `gitlab.url` and `gitlab.apiUrl` together with
the `ingress.hosts` and `ingress.tls` values as follows:
```shell
helm repo add ai-gateway \
https://gitlab.com/api/v4/projects/gitlab-org%2fcharts%2fai-gateway-helm-chart/packages/helm/devel
helm repo update
helm upgrade --install ai-gateway \
ai-gateway/ai-gateway \
--version 0.1.1 \
--namespace=ai-gateway \
--set="image.tag=<ai-gateway-image>" \
--set="gitlab.url=https://<your_gitlab_domain>" \
--set="gitlab.apiUrl=https://<your_gitlab_domain>/api/v4/" \
--set "ingress.enabled=true" \
--set "ingress.hosts[0].host=<your_gateway_domain>" \
--set "ingress.hosts[0].paths[0].path=/" \
--set "ingress.hosts[0].paths[0].pathType=ImplementationSpecific" \
--set "ingress.tls[0].secretName=ai-gateway-tls" \
--set "ingress.tls[0].hosts[0]=<your_gateway_domain>" \
--set="ingress.className=nginx" \
--timeout=300s --wait --wait-for-jobs
```
This step can take will take a few seconds in order for all resources to be allocated
and the AI Gateway to start.
Wait for your pods to get up and running:
```shell
kubectl wait pod \
--all \
--for=condition=Ready \
--namespace=ai-gateway \
--timeout=300s
```
When your pods are up and running, you can set up your IP ingresses and DNS records.
## Upgrade the AI Gateway Docker image
To upgrade the AI Gateway, download the newest Docker image tag.
1. Stop the running container:
```shell
sudo docker stop gitlab-aigw
```
1. Remove the existing container:
```shell
sudo docker rm gitlab-aigw
```
1. Pull and [run the new image](#start-a-container-from-the-image).
1. Ensure that the environment variables are all set correctly.
## Alternative installation methods
For information on alternative ways to install the AI Gateway, see
[issue 463773](https://gitlab.com/gitlab-org/gitlab/-/issues/463773).
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或