diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 6821c8ad3ee731d0158b97acd993af3580e58cea..585b98b6c363ec08343b10c9999bef06a5277a80 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -112,7 +112,7 @@ your repository small: - [Administrator documentation](../../administration/lfs/index.md) for Git LFS at GitLab - Blog post: [Getting Started with Git LFS](https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/) - [Migrate an existing Git repository](lfs/index.md#migrate-an-existing-repository-to-git-lfs) to Git LFS -- [Remove objects](lfs/index.md#removing-objects-from-lfs) from Git LFS +- [Stop tracking a file](lfs/index.md#stop-tracking-a-file-with-git-lfs) with Git LFS - Blog post: [Towards a production-quality open source Git LFS server](https://about.gitlab.com/blog/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) ## Related topics diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md index 9bd7da2244bf255204e8e9252936fd01f41815df..72fb548a8d488a59ec88d0f52b6bf031b8c1d9e4 100644 --- a/doc/topics/git/lfs/index.md +++ b/doc/topics/git/lfs/index.md @@ -7,157 +7,187 @@ description: "Use Git LFS to manage binary assets, like images and video, withou # Git Large File Storage (LFS) -Git Large File Storage (LFS) can manage large binary files efficiently in Git repositories. -It addresses challenges, such as repository performance and capacity limits. -For best performance, keep your repositories as small as possible. -Git LFS creates pointers to the actual file, stored elsewhere. +Git Large File Storage (LFS) helps Git repositories manage large binary files efficiently. +When you add a file to your repository using Git LFS, GitLab: -For GitLab.com repository size limits, see [account and limit settings](../../../administration/settings/account_and_limit_settings.md). +1. Places the file in your project's configured object storage, instead of the Git repository. +1. Adds a pointer in your Git repository, instead of the large file. This small file + tells Git where to find the full version of the file. +1. Queues a job to recalculate your project's statistics, including storage size and + LFS object storage. Your LFS object storage is the sum of the size of all LFS + objects associated with your repository. -Git LFS clients communicate with server over HTTPS, with HTTP Basic authentication. -After the request is authorized, Git LFS client gets instructions on where to fetch or where to push the large file. +Files managed with Git LFS show a **LFS** badge next to the filename: -## Configure your GitLab server for Git LFS + -DETAILS: -**Tier:** Free, Premium, Ultimate -**Offering:** Self-managed, GitLab Dedicated +Your Git repository remains smaller, which helps you adhere to repository size limits. +For more information, see repository size limits +[for self-managed](../../../administration/settings/account_and_limit_settings.md#repository-size-limit) and +[for GitLab.com](../../../user/gitlab_com/index.md#account-and-limit-settings). -To install Git LFS on your self-managed GitLab server, see -[GitLab Git Large File Storage (LFS) Administration](../../../administration/lfs/index.md). +Git LFS clients use HTTP Basic authentication, and communicate with your server +over HTTPS. After you authenticate the request, the Git LFS client receives instructions +on where to fetch (or push) the large file. + +## Understand how Git LFS works with forks -## Enable Git LFS for a project +When you fork a repository, your fork includes the upstream repository's existing LFS objects +that existed at the time of your fork. If you add new LFS objects to your fork, +they belong to only your fork, and not the upstream repository. The total object storage +increases only for your fork. + +When you create a merge request from your fork back to the upstream project, and +your merge request contains a new Git LFS object, GitLab associates the new LFS object +with the _upstream_ project after merge. + +## Known limitations + +- The Git LFS original v1 API is unsupported. +- Even when Git communicates with the repository over SSH, Git LFS objects still use HTTPS. +- Git LFS requests use HTTPS credentials, which means: + - You should use a good Git [credentials store](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). + - If your GitLab server uses HTTP instead, you must + [add the URL to Git configuration manually](troubleshooting.md#getsockopt-connection-refused). +- [Group wikis](../../../user/project/wiki/group.md) do not support Git LFS. + +## Add a file with Git LFS Prerequisites: -- You must have at least the Developer role in the project. +- You have downloaded and installed the appropriate version of the + [CLI extension for Git LFS](https://git-lfs.com) for your operating system. +- You have installed the Git LFS pre-push hook by running `git lfs install` + in the root directory of your repository. -To do this: +To add a large file into your Git repository and track it with Git LFS: -1. On the left sidebar, select **Search or go to** and find your project. -1. Select **Settings > General**. -1. Expand the **Visibility, project features, permissions** section. -1. Turn on the **Git Large File Storage (LFS)** toggle. -1. Select **Save changes**. +1. To track all files of a certain type with Git LFS, rather than a single file, + run this command, replacing `iso` with your desired file type: -## Install the Git LFS client locally + ```shell + git lfs track "*.iso" + ``` -Install the [Git LFS client](https://github.com/git-lfs/git-lfs) appropriate for -your operating system. GitLab requires version 1.0.1 or later of the Git LFS client. + This command creates a `.gitattributes` file with instructions to handle all + ISO files with Git LFS. +1. Add a file of that type (`.iso`) to your repository. +1. Tell Git to track the changes to both the `.gitattributes` file and the `.iso` file: -After Git LFS is installed on the server and client, you can see the **LFS** badge -next to the filename: + ```shell + git add . + ``` - +1. To ensure you've added both files, run `git status`. If the `.gitattributes` file + isn't included in your commit, users who clone your repository don't get the + files they need. +1. Commit both files to your local copy of your repository: -## Known limitations + ```shell + git commit -am "Add an ISO file and .gitattributes" + ``` -- Git LFS v1 original API is not supported, because it was deprecated early in LFS - development. -- Even when Git communicates with the repository over SSH, Git LFS objects still - go through HTTPS. -- Because Git LFS requests require HTTPS credentials, you should use a good Git - [credentials store](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). -- Git LFS always assumes HTTPS so if you have GitLab server on HTTP you must - [add the URL to Git configuration manually](troubleshooting.md#getsockopt-connection-refused). -- [Group wikis](../../../user/project/wiki/group.md) do not support Git LFS. +1. Push your changes back upstream, replacing `main` with the name of your branch: -## How LFS objects affect repository size + ```shell + git push origin main + ``` -When you add an LFS object to a repository, GitLab: + Make sure the files you are changing aren't listed in a `.gitignore` file. + If this file (or all files of this type) is in your `.gitignore` file, Git commits + the change locally, but does not push it to your upstream repository. -1. Creates an LFS object. -1. Associates the LFS object with the repository. -1. Queues a job to recalculate your project's statistics, including storage size and - LFS object storage. Your LFS object storage is the sum of the size of all LFS - objects associated with the repository. +1. Create your merge request. -When your repository is forked, the fork includes LFS objects from the upstream project. -The LFS object storage for the fork, at first, is the same size as the storage used by the upstream -project. -If new LFS objects are added to the fork, the total object storage increases only for the fork. +## Stop tracking a file with Git LFS -If you create a merge request from the fork back to the upstream project, -new LFS objects get associated with the upstream project. +Prerequisites: -## Using Git LFS +- You have downloaded and installed the appropriate version of the + [CLI extension for Git LFS](https://git-lfs.com) for your operating system. +- You have installed the Git LFS pre-push hook by running `git lfs install` + in the root directory of your repository. -Let's take a look at the workflow for checking large files into your Git -repository with Git LFS. For example, if you want to upload a very large file and -check it into your Git repository: +To stop tracking a single file in Git LFS: -```shell -git clone git@gitlab.example.com:group/my-sample-project.git -cd my-sample-project -git lfs install # initialize the Git LFS project -git lfs track "*.iso" # select the file extensions that you want to treat as large files -``` +1. Run the [`git lfs untrack`](https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-untrack.adoc) + command and provide the path to the file: -These commands create a `.gitattributes` file with the file types that you want to track. -In this case, Git LFS is now tracking `*.iso` files. -You can now use these commands to push a large `*.iso` file (and the new `.gitattributes` file) to -your repository: + ```shell + git lfs untrack doc/example.iso + ``` -```shell -cp ~/tmp/debian.iso ./ # copy a large file into the current directory -git add . # add the large file to the project -git commit -am "Add Debian iso and .gitattributes" # commit the file meta data -git push origin main # sync the git repo and large file to the GitLab server -``` +1. Push your changes, create a merge request, and merge the merge request. -**Make sure** you've committed `.gitattributes` to your repository. Otherwise Git -LFS doesn't work properly for those who clone it: +If you delete an object (`example.iso`) tracked by Git LFS, but don't use +the `git lfs untrack` command, `example.iso` shows as `modified` in `git status`. -Cloning the repository works the same as before. Git automatically detects the -LFS-tracked files and clones them over HTTP. If you performed the `git clone` -command with a SSH URL, you have to enter your GitLab credentials for HTTP -authentication. +### Stop tracking all files of a single type -```shell -git clone git@gitlab.example.com:group/my-sample-project.git -``` +Prerequisites: -If you already cloned the repository and want the latest LFS objects -that are on the remote repository: +- You have downloaded and installed the appropriate version of the + [CLI extension for Git LFS](https://git-lfs.com) for your operating system. +- You have installed the Git LFS pre-push hook by running `git lfs install` + in the root directory of your repository. -```shell -git lfs fetch origin main -``` +To stop tracking all files of a particular type in Git LFS: -Make sure your files aren't listed in `.gitignore`, otherwise, they are ignored by Git -and are not pushed to the remote repository. +1. Run the [`git lfs untrack`](https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-untrack.adoc) + command and provide the file type to stop tracking: -### Migrate an existing repository to Git LFS + ```shell + git lfs untrack "*.iso" + ``` -Read the documentation on how to [migrate an existing Git repository with Git LFS](https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-migrate.adoc). +1. Push your changes, create a merge request, and merge the merge request. -### Removing objects from LFS +## Configure Git LFS on your server -To remove objects from LFS: +DETAILS: +**Tier:** Free, Premium, Ultimate +**Offering:** Self-managed, GitLab Dedicated -1. Use [`git filter-repo`](../../../user/project/repository/reducing_the_repo_size_using_git.md) to remove the objects from the repository. -1. Delete the relevant LFS lines for the objects you have removed from your `.gitattributes` file and commit those changes. +Git LFS is enabled by default for both self-managed instances and GitLab.com. +To configure it, such as setting up remote object storage, see +[GitLab Git Large File Storage (LFS) Administration](../../../administration/lfs/index.md). -## LFS objects in project archives +## Enable or disable Git LFS for a project -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15079) support for including Git LFS blobs inside [project source downloads](../../../user/project/repository/index.md) in GitLab 13.5 [with a flag](../../../administration/feature_flags.md) named `include_lfs_blobs_in_archive`. Disabled by default. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46572) in GitLab 13.6. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62539) in GitLab 14.0. Feature flag `include_lfs_blobs_in_archive` removed. +Git LFS is enabled by default for both self-managed instances and GitLab.com. -Prior to GitLab 13.5, [project source downloads](../../../user/project/repository/index.md) would include Git -LFS pointers instead of the actual objects. For example, LFS pointers -look like the following: +Prerequisites: -```markdown -version https://git-lfs.github.com/spec/v1 -oid sha256:3ea5dd307f195f449f0e08234183b82e92c3d5f4cff11c2a6bb014f9e0de12aa -size 177735 +- You must have at least the Developer role in the project. + +To enable or disable Git LFS at the project level: + +1. On the left sidebar, select **Search or go to** and find your project. +1. Select **Settings > General**. +1. Expand the **Visibility, project features, permissions** section. +1. Select the **Git Large File Storage (LFS)** toggle. +1. Select **Save changes**. + +## Clone a repository that uses Git LFS + +When you clone a repository that uses Git LFS, Git detects the LFS-tracked files +and clones them over HTTPS. If you run `git clone` with a SSH URL, like +`user@hostname.com:group/project.git`, must enter your GitLab credentials again for HTTPS +authentication. + +### Update your local clone of LFS objects + +If you have already cloned a repository that uses Git LFS, and want to fetch +new LFS objects from the upstream repository, run: + +```shell +git lfs fetch origin main ``` -In GitLab version 13.5 and later, these pointers are converted to the uploaded -LFS object. +## Migrate an existing repository to Git LFS -Technical details about how this works can be found in the [development documentation for LFS](../../../development/lfs.md#including-lfs-blobs-in-project-archives). +Read the [`git-lfs-migrate` documentation](https://github.com/git-lfs/git-lfs/blob/main/docs/man/git-lfs-migrate.adoc) +on how to migrate an existing Git repository with Git LFS. ## Related topics @@ -166,3 +196,11 @@ Technical details about how this works can be found in the [development document - [Git LFS developer information](../../../development/lfs.md) - [GitLab Git Large File Storage (LFS) Administration](../../../administration/lfs/index.md) for self-managed instances - [Troubleshooting Git LFS](troubleshooting.md) + +## Troubleshooting + +### Reduce repository size after removing large files + +If you need to remove large files from your repository's history, to reduce +the total size of your repository, see +[Reduce repository size](../../../user/project/repository/reducing_the_repo_size_using_git.md). diff --git a/doc/topics/git/lfs/troubleshooting.md b/doc/topics/git/lfs/troubleshooting.md index 156ea876443ece47217068d6272ef94cd12c7327..2213d1b70adda9f0492fb871216f31e83bc1119c 100644 --- a/doc/topics/git/lfs/troubleshooting.md +++ b/doc/topics/git/lfs/troubleshooting.md @@ -139,7 +139,7 @@ It is possible to host LFS objects externally by setting a custom LFS URL with ` You might choose to do this if you are using an appliance like a Nexus Repository to store LFS data. If you choose to use an external LFS store, GitLab can't verify LFS objects. Pushes then fail if you have GitLab LFS support enabled. -To stop push failure, LFS support can be disabled in the [Project settings](index.md#enable-git-lfs-for-a-project), which also disables GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS). +To stop push failure, LFS support can be disabled in the [Project settings](index.md#enable-or-disable-git-lfs-for-a-project), which also disables GitLab LFS value-adds (Verifying LFS objects, UI integration for LFS). ## I/O timeout when pushing LFS objects diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index 9229120b6d11f9f80a953687cbfbe00aea819357..c404ac3bdc7f6458250c06d78cee2cb37ec2c9d2 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -77,7 +77,7 @@ you can skip this step. If you're unsure, re-installing it does no harm: git lfs install ``` -For more information, see [Using Git LFS](../../topics/git/lfs/index.md#using-git-lfs). +For more information, see [Git Large File Storage (LFS)](../../topics/git/lfs/index.md). ### Configure Exclusive File Locks