The metadata database enables many new registry features, including
online garbage collection, and increases the efficiency of many registry operations.
See the [Container registry metadata database](container_registry_metadata_database.md) page for details.
## Container registry garbage collection
NOTE:
...
...
@@ -1260,11 +1266,11 @@ blobs start being deleted is anything permanent done.
### Continuous Zero Downtime Garbage Collection **(BETA)**
You can run garbage collection in the background without the need to schedule it or require read-only mode,
if you migrate to the [metadata database (beta)](#use-a-postgresql-database-for-metadata).
if you migrate to the [metadata database (beta)](container_registry_metadata_database.md).
NOTE:
If you would like to try this [beta feature](../../policy/experiment-beta-support.md#beta),
you should review the [known limitations](#known-limitations). If you have any feedback,
you should review the [known limitations](container_registry_metadata_database.md#known-limitations). If you have any feedback,
you can let us know in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/423459).
## Configure GitLab and Registry to run on separate nodes (Linux package installations)
...
...
@@ -1368,42 +1374,6 @@ including all the supported storage backends. To migrate to the GitLab container
you can follow the instructions on this page, and use the same storage backend as the Distribution Registry.
The GitLab container registry should accept the same configuration that you are using for the Distribution Registry.
## Use a PostgreSQL database for metadata **(FREE SELF BETA)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/423459) in GitLab 16.4 as a [Beta feature](../../policy/experiment-beta-support.md) for self-managed GitLab instances.
WARNING:
While the metadata database is already in use on GitLab.com, it is in early beta for self-managed GitLab instances.
By default, the container registry uses object storage to persist metadata
related to container images. This method to store metadata limits how efficiently
the data can be accessed, especially data spanning multiple images, such as when listing tags.
By using a database to store this data, many new features are possible, including
which removes old data automatically with zero downtime.
This database works in conjunction with the object storage already used by the registry, but does not replace object storage.
You must continue to maintain an object storage solution even after migrating to a metadata database.
### Known Limitations
- No support for online migrations.
- Geo Support is not confirmed.
- Registry database migrations must be ran manually when upgrading versions.
### Migration Instructions and Feedback
Instructions on how to migrate to the database may be found in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/423459) for the beta period.
This issue also serves as a place to report issues and to get an overview of the beta status.
### Metadata database feature support
You can migrate existing registries to the metadata database, and use online garbage collection.
Some database-enabled features are only enabled for GitLab.com and automatic database provisioning for
the registry database is not available. Review the feature support table in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/423459#supported-feature-status)
for the status of features related to the container registry database.
## Troubleshooting
Before diving in to the following sections, here's some basic troubleshooting:
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/423459) in GitLab 16.4 as a [Beta feature](../../policy/experiment-beta-support.md) for self-managed GitLab instances.
WARNING:
The metadata database is a [beta feature](../../policy/experiment-beta-support.md#beta).
Carefully review the documentation before enabling the registry database in production!
If you encounter a problem with either the import or operation of the
registry, please add a comment in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/423459).
The metadata database enables many new registry features, including
online garbage collection, and increases the efficiency of many registry operations.
The work on the self-managed release of the registry metadata database feature
is tracked in [epic 5521](https://gitlab.com/groups/gitlab-org/-/epics/5521).
By default, the container registry uses object storage to persist metadata
related to container images. This method to store metadata limits how efficiently
the data can be accessed, especially data spanning multiple images, such as when listing tags.
By using a database to store this data, many new features are possible, including
which removes old data automatically with zero downtime.
This database works in conjunction with the object storage already used by the registry, but does not replace object storage.
You must continue to maintain an object storage solution even after migrating to a metadata database.
## Known Limitations
- No support for online migrations.
- Geo Support is not confirmed.
- Registry database migrations must be ran manually when upgrading versions.
## Metadata database feature support
You can migrate existing registries to the metadata database, and use online garbage collection.
Some database-enabled features are only enabled for GitLab.com and automatic database provisioning for
the registry database is not available. Review the feature support table in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/423459#supported-feature-status)
for the status of features related to the container registry database.
## Enable the metadata database for Linux package installations
Prerequisites:
- GitLab 16.7 or later.
- PostgreSQL database version 12 or later. It must be accessible from the registry node.
Follow the instructions that match your situation:
-[New installation](#new-installations) or enabling the container registry for the first time.
- Migrate existing container images to the metadata database:
-[One-step migration](#one-step-migration). Only recommended for relatively small registries or no requirement to avoid downtime.
-[Three-step migration](#three-step-migration). Recommended for larger container registries.
### Before you start
- After you enable the database, you must continue to use it. The database is
now the source of the registry metadata, disabling it after this point
causes the registry to lose visibility on all images written to it while
the database was active.
- Never run [offline garbage collection](container_registry.md#container-registry-garbage-collection) at any point
after the import step has been completed. That command is not compatible with registries using
the metadata database, and it deletes data.
- Verify you have not automated offline garbage collection.
- You can first [reduce the storage of your registry](../../user/packages/container_registry/reduce_container_registry_storage.md)
to speed up the process.
- Back up [your container registry data](../backup_restore/backup_gitlab.md#container-registry)
if possible.
### New installations
To enable the database:
1. Edit `/etc/gitlab/gitlab.rb` by adding your database connection details, but start with the metadata database **disabled**:
```ruby
registry['database']={
'enabled'=>false,
'host'=>'localhost',
'port'=>5432,
'user'=>'registry-database-user',
'password'=>'registry-database-password',
'dbname'=>'registry-database-name',
'sslmode'=>'require',# See the PostgreSQL documentation for additional information https://www.postgresql.org/docs/current/libpq-ssl.html.
'sslcert'=>'/path/to/cert.pem',
'sslkey'=>'/path/to/private.key',
'sslrootcert'=>'/path/to/ca.pem'
}
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation).
the registry fails with the following error message:
```shell
ERRO[0000] this filesystem is managed by the metadata database, and offline garbage collection is no longer possible, if you are not using the database anymore, remove the file at the lock_path in this log message lock_path=/docker/registry/lockfiles/database-in-use
```
You must either:
- Stop using offline garbage collection.
- If you no longer use the metadata database, delete the indicated lock file at the `lock_path` shown in the error message.
For example, remove the `/docker/registry/lockfiles/database-in-use` file.