Skip to content
代码片段 群组 项目
提交 4d8223d2 编辑于 作者: maddievn's avatar maddievn
浏览文件

Adds a script to generate elastic migration files

Creates a migration file, spec file and dictionary file

Changelog: added
上级 40990422
No related branches found
No related tags found
无相关合并请求
...@@ -94,7 +94,7 @@ EE: true ...@@ -94,7 +94,7 @@ EE: true
uses system fonts for all text." uses system fonts for all text."
- Any client-facing change to our REST and GraphQL APIs **must** have a changelog entry. - Any client-facing change to our REST and GraphQL APIs **must** have a changelog entry.
See the [complete list what comprises a GraphQL breaking change](api_graphql_styleguide.md#breaking-changes). See the [complete list what comprises a GraphQL breaking change](api_graphql_styleguide.md#breaking-changes).
- Any change that introduces an [advanced search migration](search/advanced_search_migration_styleguide.md#creating-a-new-advanced-search-migration) - Any change that introduces an [advanced search migration](search/advanced_search_migration_styleguide.md#create-a-new-advanced-search-migration)
**must** have a changelog entry. **must** have a changelog entry.
- A fix for a regression introduced and then fixed in the same release (such as - A fix for a regression introduced and then fixed in the same release (such as
fixing a bug introduced during a monthly release candidate) **should not** fixing a bug introduced during a monthly release candidate) **should not**
......
...@@ -6,13 +6,25 @@ info: To determine the technical writer assigned to the Stage/Group associated w ...@@ -6,13 +6,25 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Advanced search migration style guide # Advanced search migration style guide
## Creating a new advanced search migration ## Create a new advanced search migration
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/234046) in GitLab 13.6.
NOTE: NOTE:
This functionality is only supported for indices created in GitLab 13.0 and later. This functionality is only supported for indices created in GitLab 13.0 and later.
### With a script
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414674) in GitLab 16.3.
Execute `scripts/elastic-migration` and follow the prompts to create:
- A migration file to define the migration: `ee/elastic/migrate/YYYYMMDDHHMMSS_migration_name.rb`
- A spec file to test the migration: `ee/spec/elastic/migrate/YYYYMMDDHHMMSS_migration_name_spec.rb`
- A dictionary file to identify the migration: `ee/elastic/docs/YYYYMMDDHHMMSS_migration_name.yml`
### Manually
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/234046) in GitLab 13.6.
In the [`ee/elastic/migrate/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/elastic/migrate) folder, create a new file with the filename format `YYYYMMDDHHMMSS_migration_name.rb`. This format is the same for Rails database migrations. In the [`ee/elastic/migrate/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/elastic/migrate) folder, create a new file with the filename format `YYYYMMDDHHMMSS_migration_name.rb`. This format is the same for Rails database migrations.
```ruby ```ruby
......
...@@ -527,7 +527,29 @@ With reindex migrations running in the background, there's no need for a manual ...@@ -527,7 +527,29 @@ With reindex migrations running in the background, there's no need for a manual
intervention. This usually happens in situations where new features are added to intervention. This usually happens in situations where new features are added to
advanced search, which means adding or changing the way content is indexed. advanced search, which means adding or changing the way content is indexed.
To confirm that the advanced search migrations ran, you can check with: ### Migration dictionary files
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414674) in GitLab 16.3.
Every migration has a corresponding dictionary file in the `ee/elastic/docs/` folder with the following information:
```yaml
name:
version:
description:
group:
milestone:
introduced_by_url:
obsolete:
marked_obsolete_by_url:
marked_obsolete_in_milestone:
```
You can use this information, for example, to identify when a migration was introduced or was marked as obsolete.
### Check for pending migrations
To check for pending advanced search migrations, run this command:
```shell ```shell
curl "$CLUSTER_URL/gitlab-production-migrations/_search?q=*" | jq . curl "$CLUSTER_URL/gitlab-production-migrations/_search?q=*" | jq .
...@@ -566,7 +588,7 @@ This should return something similar to: ...@@ -566,7 +588,7 @@ This should return something similar to:
} }
``` ```
To debug issues with the migrations you can check the [`elasticsearch.log` file](../../administration/logs/index.md#elasticsearchlog). To debug issues with the migrations, check the [`elasticsearch.log`](../../administration/logs/index.md#elasticsearchlog) file.
### Retry a halted migration ### Retry a halted migration
......
#!/usr/bin/env ruby
# frozen_string_literal: true
# Generate an Elastic migration file, spec and dictionary record with the current timestamp.
require 'yaml'
require 'fileutils'
require 'uri'
require 'readline'
require 'active_support/core_ext/string'
class ElasticMigrationCreator
attr_reader :options
Options = Struct.new(
:name,
:description,
:group,
:introduced_by_url,
:milestone,
:obsolete,
:marked_obsolete_by_url,
:marked_obsolete_in_milestone
)
def initialize
@options = Options.new
end
def execute
options.name = read_name
options.description = read_description
options.group = read_group
options.introduced_by_url = read_introduced_by_url
options.milestone = read_milestone
$stdout.puts "\e[32mcreated\e[0m #{file_path}"
$stdout.puts "\e[32mcreated\e[0m #{spec_file_path}"
$stdout.puts "\e[32mcreated\e[0m #{dictionary_file_path}"
write
$stdout.puts "\n=> Please consult the documentation for Advanced Search Migrations: #{documentation_reference}"
end
private
def read_name
read_variable('name', 'Name of the migration in CamelCase').camelize
end
def read_description
read_variable('description', 'Description of what the migration does')
end
def read_group
read_variable('group', 'The group introducing a feature flag, like: `global search`')
end
def read_milestone
milestone = File.read('VERSION')
milestone.gsub(/^(\d+\.\d+).*$/, '\1').chomp
end
def read_variable(name, description)
$stdout.puts "\n>> #{description}:"
loop do
variable = Readline.readline('?> ', false)&.strip
return variable unless variable.empty?
warn "Error: #{name} is required."
end
end
def read_introduced_by_url
$stdout.puts
$stdout.puts ">> URL of the MR introducing the migration (enter to skip):"
loop do
introduced_by_url = Readline.readline('?> ', false)&.strip
introduced_by_url = nil if introduced_by_url.empty?
return introduced_by_url if introduced_by_url.nil? || introduced_by_url.start_with?('https://')
warn 'Error: URL needs to start with https://'
end
end
def write
# create migration file
FileUtils.mkdir_p(File.dirname(file_path))
File.write(file_path, file_contents)
# create spec
FileUtils.mkdir_p(File.dirname(spec_file_path))
File.write(spec_file_path, spec_contents)
# create dictionary file
FileUtils.mkdir_p(File.dirname(dictionary_file_path))
File.write(dictionary_file_path, dictionary_contents)
end
def timestamp
@timestamp ||= Time.now.strftime('%Y%m%d%H%M%S')
end
def file_name
@file_name ||= "#{timestamp}_#{options.name.dup.underscore}"
end
def file_path
"ee/elastic/migrate/#{file_name}.rb"
end
def spec_file_path
"ee/spec/elastic/migrate/#{file_name}_spec.rb"
end
def dictionary_file_path
"ee/elastic/docs/#{file_name}.yml"
end
def file_contents
"# frozen_string_literal: true
class #{options.name} < Elastic::Migration
end
"
end
def spec_contents
"# frozen_string_literal: true
require 'spec_helper'
require_relative 'migration_shared_examples'
require File.expand_path('#{file_path}')
RSpec.describe #{options.name}, feature_category: :#{options.group.parameterize.underscore} do
let(:version) { #{timestamp} }
end
"
end
def dictionary_contents
dictionary_config_hash.to_yaml
end
def dictionary_config_hash
{
'name' => options.name,
'version' => timestamp,
'description' => options.description,
'group' => "group::#{options.group}",
'milestone' => options.milestone,
'introduced_by_url' => options.introduced_by_url,
'obsolete' => false,
'marked_obsolete_by_url' => nil,
'marked_obsolete_in_milestone' => nil
}
end
def documentation_reference
'https://docs.gitlab.com/ee/development/search/advanced_search_migration_styleguide.html'
end
end
ElasticMigrationCreator.new.execute if $PROGRAM_NAME == __FILE__
# vim: ft=ruby
0% 加载中 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册