rake_tasks.md

stage: none
group: unassigned
info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.

Rake tasks for developers

Rake tasks are available for developers and others contributing to GitLab.

Set up database with developer seeds

If your database user does not have advanced privileges, you must create the database manually before running this command.

bundle exec rake setup

The setup task is an alias for gitlab:setup. This tasks calls db:reset to create the database, and calls db:seed_fu to seed the database. db:setup calls db:seed but this does nothing.

Environment variables

MASS_INSERT: Create millions of users (2m), projects (5m) and its relations. It's highly recommended to run the seed with it to catch slow queries while developing. Expect the process to take up to 20 extra minutes.

See also Mass inserting Rails models.

LARGE_PROJECTS: Create large projects (through import) from a predefined set of URLs.

Seeding Data

Seeding issues for all projects or a single project

You can seed issues for all or a given project with the gitlab:seed:issues task:

# All projects
bin/rake gitlab:seed:issues

# A specific project
bin/rake "gitlab:seed:issues[group-path/project-path]"

By default, this seeds an average of 2 issues per week for the last 5 weeks per project.

Seeding issues for Insights charts

DETAILS: Tier: Ultimate Offering: GitLab.com, Self-managed, GitLab Dedicated

You can seed issues specifically for working with the Insights charts with the gitlab:seed:insights:issues task:

# All projects
bin/rake gitlab:seed:insights:issues

# A specific project
bin/rake "gitlab:seed:insights:issues[group-path/project-path]"

By default, this seeds an average of 10 issues per week for the last 52 weeks per project. All issues are also randomly labeled with team, type, severity, and priority.

Seeding groups with subgroups

You can seed groups with subgroups that contain milestones/projects/issues with the gitlab:seed:group_seed task:

bin/rake "gitlab:seed:group_seed[subgroup_depth, username]"

Group are additionally seeded with epics if GitLab instance has epics feature available.

Seeding a runner fleet test environment

Use the gitlab:seed:runner_fleet task to seed a full runner fleet, specifically groups with subgroups and projects that contain runners and pipelines:

bin/rake "gitlab:seed:runner_fleet[username, registration_prefix, runner_count, job_count]"

By default, the Rake task uses the root username to create 40 runners and 400 jobs.

graph TD
    G1[Top level group 1] --> G11
    G2[Top level group 2] --> G21
    G11[Group 1.1] --> G111
    G11[Group 1.1] --> G112
    G111[Group 1.1.1] --> P1111
    G112[Group 1.1.2] --> P1121
    G21[Group 2.1] --> P211

    P1111[Project 1.1.1.1<br><i>70% of jobs, sent to first 5 runners</i>]
    P1121[Project 1.1.2.1<br><i>15% of jobs, sent to first 5 runners</i>]
    P211[Project 2.1.1<br><i>15% of jobs, sent to first 5 runners</i>]

    IR1[Instance runner]
    P1111R1[Shared runner]
    P1111R[Project 1.1.1.1 runners<br>20% total runners]
    P1121R[Project 1.1.2.1 runners<br>49% total runners]
    G111R[Group 1.1.1 runners<br>30% total runners<br><i>remaining jobs</i>]
    G21R[Group 2.1 runners<br>1% total runners]

    P1111 --> P1111R1
    P1111 --> G111R
    P1111 --> IR1
    P1111 --> P1111R
    P1121 --> P1111R1
    P1121 --> IR1
    P1121 --> P1121R
    P211 --> P1111R1
    P211 --> G21R
    P211 --> IR1

    classDef groups fill:#09f6,color:#000000,stroke:#333,stroke-width:3px;
    classDef projects fill:#f96a,color:#000000,stroke:#333,stroke-width:2px;
    class G1,G2,G11,G111,G112,G21 groups
    class P1111,P1121,P211 projects

Seeding custom metrics for the monitoring dashboard

A lot of different types of metrics are supported in the monitoring dashboard.

To import these metrics, you can run:

bundle exec rake 'gitlab:seed:development_metrics[your_project_id]'

Seed a project with vulnerabilities

You can seed a project with security vulnerabilities.

# Seed all projects
bin/rake 'gitlab:seed:vulnerabilities'

# Seed a specific project
bin/rake 'gitlab:seed:vulnerabilities[group-path/project-path]'

Seed a project with environments

You can seed a project with environments.

By default, this creates 10 environments, each with the prefix ENV_. Only project_path is required to run this command.

bundle exec rake "gitlab:seed:project_environments[project_path, seed_count, prefix]"

# Examples
bundle exec rake "gitlab:seed:project_environments[flightjs/Flight]"
bundle exec rake "gitlab:seed:project_environments[flightjs/Flight, 25, FLIGHT_ENV_]"

Seed a group with dependencies

bundle exec rake gitlab:seed:dependencies

Seed CI variables

You can seed a project, group, or instance with CI variables.

By default, each command creates 10 CI variables. Variable names are prepended with its own default prefix (VAR_ for project-level variables, GROUP_VAR_ for group-level variables, and INSTANCE_VAR_ for instance-level variables).

Instance-level variables do not have environment scopes. Project-level and group-level variables use the default "*" environment scope if no environment_scope is supplied. If environment_scope is set to "unique", each variable is created with its own unique environment.

# Seed a project with project-level CI variables
# Only `project_path` is required to run this command.
bundle exec rake "gitlab:seed:ci_variables_project[project_path, seed_count, environment_scope, prefix]"

# Seed a group with group-level CI variables
# Only `group_name` is required to run this command.
bundle exec rake "gitlab:seed:ci_variables_group[group_name, seed_count, environment_scope, prefix]"

# Seed an instance with instance-level CI variables
bundle exec rake "gitlab:seed:ci_variables_instance[seed_count, prefix]"

# Examples
bundle exec rake "gitlab:seed:ci_variables_project[flightjs/Flight]"
bundle exec rake "gitlab:seed:ci_variables_project[flightjs/Flight, 25, staging]"
bundle exec rake "gitlab:seed:ci_variables_project[flightjs/Flight, 25, unique, CI_VAR_]"

bundle exec rake "gitlab:seed:ci_variables_group[group_name]"
bundle exec rake "gitlab:seed:ci_variables_group[group_name, 25, staging]"
bundle exec rake "gitlab:seed:ci_variables_group[group_name, 25, unique, CI_VAR_]"

bundle exec rake "gitlab:seed:ci_variables_instance"
bundle exec rake "gitlab:seed:ci_variables_instance[25, CI_VAR_]"

Seed a project for merge train development

Seeds a project with merge trains configured and 20 merge requests(each with 3 commits). The command:

rake gitlab:seed:merge_trains:project

Automation

If you're very sure that you want to wipe the current database and refill seeds, you can set the FORCE environment variable to yes:

FORCE=yes bundle exec rake setup

This skips the action confirmation/safety check, saving you from answering yes manually.

Discard stdout

Since the script would print a lot of information, it could be slowing down your terminal, and it would generate more than 20G logs if you just redirect it to a file. If we don't care about the output, we could just redirect it to /dev/null:

echo 'yes' | bundle exec rake setup > /dev/null

Because you can't see the questions from stdout, you might just want to echo 'yes' to keep it running. It would still print the errors on stderr so no worries about missing errors.

Extra Project seed options

There are a few environment flags you can pass to change how projects are seeded

• SIZE: defaults to 8, max: 32. Amount of projects to create.
• LARGE_PROJECTS: defaults to false. If set, clones 6 large projects to help with testing.
• FORK: defaults to false. If set to true, forks torvalds/linux five times. Can also be set to an existing project full_path to fork that instead.

Run tests

To run the test you can use the following commands:

• bin/rake spec to run the RSpec suite
• bin/rake spec:unit to run only the unit tests
• bin/rake spec:integration to run only the integration tests
• bin/rake spec:system to run only the system tests

bin/rake spec takes significant time to pass. Instead of running the full test suite locally, you can save a lot of time by running a single test or directory related to your changes. After you submit a merge request, CI runs full test suite for you. Green CI status in the merge request means full test suite is passed.

You can't run rspec . since this tries to run all the _spec.rb files it can find, also the ones in /tmp

You can pass RSpec command line options to the spec:unit, spec:integration, and spec:system tasks. For example, bin/rake "spec:unit[--tag ~geo --dry-run]".

For an RSpec test, to run a single test file you can run:

bin/rspec spec/controllers/commit_controller_spec.rb

To run several tests inside one directory:

• bin/rspec spec/requests/api/ for the RSpec tests if you want to test API only

Run RSpec tests which failed in Merge Request pipeline on your machine

If your Merge Request pipeline failed with RSpec test failures, you can run all the failed tests on your machine with the following Rake task:

bin/rake spec:merge_request_rspec_failure

There are a few caveats for this Rake task:

• You need to be on the same branch on your machine as the source branch of the Merge Request.
• The pipeline must have been completed.
• You may need to wait for the test report to be parsed and retry again.

This Rake task depends on the unit test reports feature, which only gets parsed when it is requested for the first time.

Speed up tests, Rake tasks, and migrations

Spring is a Rails application pre-loader. It speeds up development by keeping your application running in the background so you don't need to boot it every time you run a test, Rake task or migration.

If you want to use it, you must export the ENABLE_SPRING environment variable to 1:

export ENABLE_SPRING=1

Alternatively you can use the following on each spec run,

bundle exec spring rspec some_spec.rb

RuboCop tasks

Generate initial RuboCop TODO list

One way to generate the initial list is to run the Rake task rubocop:todo:generate:

bundle exec rake rubocop:todo:generate

To generate TODO list for specific RuboCop rules, pass them comma-separated as argument to the Rake task:

bundle exec rake 'rubocop:todo:generate[Gitlab/NamespacedClass,Lint/Syntax]'
bundle exec rake rubocop:todo:generate\[Gitlab/NamespacedClass,Lint/Syntax\]

Some shells require brackets to be escaped or quoted.

See Resolving RuboCop exceptions on how to proceed from here.

Run RuboCop in graceful mode

You can run RuboCop in "graceful mode". This means all enabled cop rules are silenced which have "grace period" activated (via Details: grace period).

Run:

bundle exec rake 'rubocop:check:graceful'
bundle exec rake 'rubocop:check:graceful[Gitlab/NamespacedClass]'

Compile Frontend Assets

You shouldn't ever need to compile frontend assets manually in development, but if you ever need to test how the assets get compiled in a production environment you can do so with the following command:

RAILS_ENV=production NODE_ENV=production bundle exec rake gitlab:assets:compile

This compiles and minifies all JavaScript and CSS assets and copy them along with all other frontend assets (images, fonts, etc) into /public/assets where they can be easily inspected.