diff --git a/doc/development/internal_analytics/metrics/metrics_dictionary.md b/doc/development/internal_analytics/metrics/metrics_dictionary.md index 14556bc9b3cf53af8d4e5b2f265ebd8b1ce37885..f926a1386beb6ae22aba60b4d3674d67ba9c7afb 100644 --- a/doc/development/internal_analytics/metrics/metrics_dictionary.md +++ b/doc/development/internal_analytics/metrics/metrics_dictionary.md @@ -7,7 +7,11 @@ info: Any user with at least the Maintainer role can merge updates to this conte # Metrics Dictionary Guide [Service Ping](../service_ping/index.md) metrics are defined in individual YAML files definitions from which the -[Metrics Dictionary](https://metrics.gitlab.com/) is built. Currently, the metrics dictionary is built automatically once a day. When a change to a metric is made in a YAML file, you can see the change in the dictionary within 24 hours. +[Metrics Dictionary](https://metrics.gitlab.com/) is built. Currently, the metrics dictionary is built automatically once an hour. + +- When a change to a metric is made in a YAML file, you can see the change in the dictionary within 1 hour of the change getting deployed to production. +- When a change to an event is made in a YAML file, you can see the change in the dictionary within 1 hour of the change getting merged to the master branch. + This guide describes the dictionary and how it's implemented. ## Metrics Definition and validation @@ -29,7 +33,7 @@ All metrics are stored in YAML files: WARNING: Only metrics with a metric definition YAML and whose status is not `removed` are added to the Service Ping JSON payload. -Each metric is defined in a separate YAML file consisting of a number of fields: +Each metric is defined in a YAML file consisting of a number of fields: | Field | Required | Additional information | |------------------------------|----------|------------------------| @@ -38,7 +42,7 @@ Each metric is defined in a separate YAML file consisting of a number of fields: | `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. | | `value_type` | yes | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type). | | `status` | yes | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. | -| `time_frame` | yes | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. | +| `time_frame` | yes | `string` or `array`; may be set to `7d`, `28d`, `all`, `none` or an array including any of these values except for `none`. | | `data_source` | yes | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`, `internal_events`. | | `data_category` | yes | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`. | | `instrumentation_class` | no | `string`; used for metrics with `data_source` other than `internal_events`. See [the class that implements the metric](metrics_instrumentation.md). | @@ -59,12 +63,15 @@ The `key_path` of the metric is the location in the JSON Service Ping payload. The `key_path` could be composed from multiple parts separated by `.` and it must be unique. -We recommend to add the metric in one of the top-level keys: +If a metric definition has an array `time_frame`, the `key_path` defined in the YAML file will have a suffix automatically added for each of the included time frames: + +| time_frame | `key_path` suffix| +|------------|------------------| +| `all` | no suffix | +| `7d` | `_weekly` | +| `28d` | `_monthly` | -- `settings`: for settings related metrics. -- `counts_weekly`: for counters that have data for the most recent 7 days. -- `counts_monthly`: for counters that have data for the most recent 28 days. -- `counts`: for counters that have data for all time. +The `key_path`s shown in the [Metrics Dictionary](https://metrics.gitlab.com/) include those suffixes. ### Metric statuses @@ -88,7 +95,7 @@ Metric definitions can have one of the following values for `value_type`: ### Metric `time_frame` -A metric's time frame is calculated based on the `time_frame` field and the `data_source` of the metric. +A metric's time frame is calculated based on the `time_frame` field and the `data_source` of the metric. When `time_frame` is an array, the metric's values are calculated for each of the included time frames. | data_source | time_frame | Description | |------------------------|------------|-------------------------------------------------| diff --git a/scripts/internal_events/cli.rb b/scripts/internal_events/cli.rb index da921d37b03b9296cf11ca42464c35b976814586..148ad22a65563a355375c22ed7f5b14dc1ae58a7 100755 --- a/scripts/internal_events/cli.rb +++ b/scripts/internal_events/cli.rb @@ -15,6 +15,7 @@ require_relative './cli/flows/metric_definer' require_relative './cli/flows/usage_viewer' require_relative './cli/global_state' +require_relative './cli/time_framed_key_path' require_relative './cli/metric' require_relative './cli/event' diff --git a/scripts/internal_events/cli/flows/metric_definer.rb b/scripts/internal_events/cli/flows/metric_definer.rb index 99874078922f2fe4450f80dfac1ef37185ea020a..52c037cbaa426e3a9df8025b5e2744dd2a18c088 100755 --- a/scripts/internal_events/cli/flows/metric_definer.rb +++ b/scripts/internal_events/cli/flows/metric_definer.rb @@ -139,6 +139,7 @@ def prompt_for_metrics per_page: 20, &disabled_format_callback ) + @metrics = reduce_metrics_by_time_frame(@metrics) assign_shared_attrs(:actions, :milestone) do { @@ -221,10 +222,23 @@ def prompt_for_descriptions def file_saved_context_message(attributes) format_prefix " ", <<~TEXT.chomp - Visit #{format_info('https://metrics.gitlab.com')} to find dashboard links for this metric - - Metric trend dashboard: #{format_info(metric_trend_path(attributes['key_path']))} + #{metric_dashboard_links(attributes)} TEXT end + def metric_dashboard_links(attributes) + time_frames = attributes['time_frame'] + unless time_frames.is_a?(Array) + return "- Metric trend dashboard: #{format_info(metric_trend_path(attributes['key_path']))}" + end + + dashboards = time_frames.map do |time_frame| + key_path = TimeFramedKeyPath.build(attributes['key_path'], time_frame) + " - #{format_info(metric_trend_path(key_path))}" + end + ["- Metric trend dashboards:", *dashboards].join("\n") + end + # Check existing event files for attributes to copy over def prompt_for_copying_event_properties shared_values = collect_values_for_shared_event_properties @@ -389,6 +403,19 @@ def selected_events_filter_options end end + def reduce_metrics_by_time_frame(metrics) + # MetricOptions class returns one metric per time_frame value, + # here we merge them into a singular metric including all the time_frame values + return metrics unless metrics.length > 1 + + time_frames = metrics.map do |metric| + metric.time_frame.value + end + + attributes = metrics.first.to_h.merge(time_frame: time_frames) + [Metric.new(**attributes)] + end + # Helper for #prompt_for_event_filters def print_event_filter_header(event, idx, total) cli.say "\n" diff --git a/scripts/internal_events/cli/global_state.rb b/scripts/internal_events/cli/global_state.rb index 2dcd356136450e4bf995205e29fbe5db745fa5d2..f781ecd5781a62cf64741f0a28c0cdaf9e4c4bb7 100644 --- a/scripts/internal_events/cli/global_state.rb +++ b/scripts/internal_events/cli/global_state.rb @@ -12,11 +12,24 @@ def events end def metrics - @metrics ||= load_definitions( - Metric, - InternalEventsCli::NEW_METRIC_FIELDS, - all_metric_paths - ) + @metrics ||= begin + loaded_files = load_definitions( + Metric, + InternalEventsCli::NEW_METRIC_FIELDS, + all_metric_paths + ) + loaded_files.flat_map do |metric| + # copy logic of Gitlab::Usage::MetricDefinition + next metric unless metric.time_frame.is_a?(Array) + + metric.time_frame.map do |time_frame| + current_metric = metric.dup + current_metric.time_frame = time_frame + current_metric.key_path = TimeFramedKeyPath.build(current_metric.key_path, time_frame) + current_metric + end + end + end end def reload_definitions diff --git a/scripts/internal_events/cli/helpers/metric_options.rb b/scripts/internal_events/cli/helpers/metric_options.rb index 37d0ceaecb2d10735bdf576b6027ce547a78860f..06ef2be8c268b545e463fd585996a87b235053b2 100755 --- a/scripts/internal_events/cli/helpers/metric_options.rb +++ b/scripts/internal_events/cli/helpers/metric_options.rb @@ -203,7 +203,7 @@ def identifier # ex) "Monthly/Weekly" def time_frame_phrase - phrase = metrics.map { |metric| metric.time_frame.description }.join('/') + phrase = metrics.map { |metric| metric.time_frame.description.capitalize }.join('/') disabled ? phrase : format_info(phrase) end diff --git a/scripts/internal_events/cli/metric.rb b/scripts/internal_events/cli/metric.rb index 9920fd810dea13a8bfd1068532224e8ccad76c56..0cb9fa4a27d48ca7b894703a39f0d5dc98bc7c81 100755 --- a/scripts/internal_events/cli/metric.rb +++ b/scripts/internal_events/cli/metric.rb @@ -159,12 +159,15 @@ def filters_expected? # Automatically prepended to all new descriptions # ex) Total count of # ex) Weekly/Monthly count of unique + # ex) Count of def description_prefix - [ + description_components = [ time_frame.description, identifier.prefix, *(identifier.plural if identifier.default?) - ].join(' ') + ].compact + + description_components.join(' ').capitalize end # Provides simplified but technically accurate description @@ -172,8 +175,10 @@ def description_prefix def technical_description event_name = actions.first if events.length == 1 && !filtered? event_name ||= 'the selected events' - - "#{time_frame.description} #{identifier.description % event_name}" + [ + time_frame.description, + (identifier.description % event_name).to_s + ].compact.join(' ').capitalize end def bulk_assign(key_value_pairs) @@ -185,21 +190,25 @@ class Metric TimeFrame = Struct.new(:value) do def description case value + when Array + nil # array time_frame metrics have no description prefix when '7d' - 'Weekly' + 'weekly' when '28d' - 'Monthly' + 'monthly' when 'all' - 'Total' + 'total' end end def directory_name + return "counts_all" if value.is_a? Array + "counts_#{value}" end def key_path - description&.downcase if value != 'all' + description&.downcase if %w[7d 28d].include?(value) end end diff --git a/scripts/internal_events/cli/time_framed_key_path.rb b/scripts/internal_events/cli/time_framed_key_path.rb new file mode 100644 index 0000000000000000000000000000000000000000..5925ebe251da64bad351cb540bbbfbbafd8a37ed --- /dev/null +++ b/scripts/internal_events/cli/time_framed_key_path.rb @@ -0,0 +1,17 @@ +# frozen_string_literal: true + +# Helpers for shared & state across all CLI flows +module InternalEventsCli + class TimeFramedKeyPath + METRIC_TIME_FRAME_SUFFIX = { + '7d' => '_weekly', + '28d' => '_monthly', + 'all' => '' + }.freeze + + def self.build(base_key_path, time_frame) + # copy logic of Gitlab::Usage::MetricDefinition + "#{base_key_path}#{METRIC_TIME_FRAME_SUFFIX[time_frame]}" + end + end +end diff --git a/spec/fixtures/scripts/internal_events/event_definer_examples.yml b/spec/fixtures/scripts/internal_events/event_definer_examples.yml index 07bdd5edfe4a3e19e861aa5fd4e0d8c3c8e7a52d..b8ac06d6a20f1c7378115def19e9ec4f9d44749c 100644 --- a/spec/fixtures/scripts/internal_events/event_definer_examples.yml +++ b/spec/fixtures/scripts/internal_events/event_definer_examples.yml @@ -121,10 +121,8 @@ files: - path: config/events/random_name.yml content: spec/fixtures/scripts/internal_events/events/keyboard_smashed_event.yml - - path: config/metrics/counts_28d/count_distinct_user_id_from_random_name_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_28d.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_random_name_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_7d.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_random_name.yml + content: spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric.yml - description: Creates an event after helping the user figure out next steps inputs: diff --git a/spec/fixtures/scripts/internal_events/metric_definer_examples.yml b/spec/fixtures/scripts/internal_events/metric_definer_examples.yml index 6f2d10cca3a5de2290d33f6dabb9c42b09205722..655ab205f19aac680c4f1d060339c8946fd9f94f 100644 --- a/spec/fixtures/scripts/internal_events/metric_definer_examples.yml +++ b/spec/fixtures/scripts/internal_events/metric_definer_examples.yml @@ -10,17 +10,13 @@ - "\n" # Select: config/events/internal_events_cli_used.yml - "\n" # Select: Weekly count of unique users - "who defined an internal event using the CLI\n" # Input description - - "\n" # Submit weekly description for monthly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_internal_events_cli_used_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_internal_events_cli_used_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml - description: Create a weekly/monthly metric for a multiple events, but select only one event inputs: @@ -35,17 +31,13 @@ - "\n" # Submit selections - "\n" # Select: Weekly count of unique projects - "who defined an internal event using the CLI\n" # Input description - - "\n" # Submit weekly description for monthly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_internal_events_cli_used_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_internal_events_cli_used_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml - description: Create a weekly/monthly metric for multiple events inputs: @@ -65,17 +57,13 @@ - "\e[B" # Arrow down to: Weekly count of unique projects - "\n" # Select: Weekly count of unique projects - "where a defition file was created with the CLI\n" # Input description - - "\n" # Submit weekly description for monthly - "1\n" # Select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/project_id_28d_multiple_events.yml - - path: config/metrics/counts_7d/count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/project_id_7d_multiple_events.yml + - path: config/metrics/counts_all/count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/metrics/project_id_multiple_events.yml - description: Create an all time total metric for a single event inputs: @@ -129,6 +117,33 @@ - path: ee/config/metrics/counts_all/count_total_internal_events_cli_used.yml content: spec/fixtures/scripts/internal_events/metrics/ee_total_single_event.yml +- description: Create a weekly metric for a single event that already has a monthly metric + inputs: + files: + - path: config/events/internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/events/event_with_identifiers.yml + - path: config/metrics/counts_28d/count_internal_events_cli_used_monthly.yml + content: spec/fixtures/scripts/internal_events/metrics/ee_total_28d_single_event.yml + keystrokes: + - "2\n" # Enum-select: New Metric -- calculate how often one or more existing events occur over time + - "1\n" # Enum-select: Single event -- count occurrences of a specific event or user interaction + - 'internal_events_cli_used' # Filters to this event + - "\n" # Select: config/events/internal_events_cli_used.yml + - "\e[A\e[A" # Arrow up to: Monthly/Weekly count of events + - "\n" # Select: Monthly/Weekly count of events + - "when an event was defined using the CLI\n" # Input description + - "2\n" # Enum-select: Modify attributes + - "\n" # Accept group from event definition + - "\n" # Accept product categories from event definition + - "\n" # Accept URL from event definition + - "2\n" # Override tier -> Select: [premium, ultimate] + - "y\n" # Create file + - "5\n" # Exit + outputs: + files: + - path: ee/config/metrics/counts_7d/count_total_internal_events_cli_used_weekly.yml + content: spec/fixtures/scripts/internal_events/metrics/ee_total_7d_single_event.yml + - description: Create a metric after helping the user figure out next steps inputs: files: @@ -153,6 +168,7 @@ files: - path: config/metrics/counts_all/count_total_internal_events_cli_used.yml content: spec/fixtures/scripts/internal_events/metrics/total_single_event.yml + - description: Create a metric and then another metric with copied events inputs: files: @@ -165,10 +181,8 @@ - "\n" # Select: config/events/internal_events_cli_used.yml - "\n" # Select: Weekly count of unique users - "who defined an internal event using the CLI\n" # Input description - - "\n" # Submit weekly description for monthly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "2\n" # Create another metric with the same events - "\e[A" # Arrow up to: Total count of events - "\n" # Select: Total count of events @@ -178,10 +192,8 @@ - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_internal_events_cli_used_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_internal_events_cli_used_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml - path: config/metrics/counts_all/count_total_internal_events_cli_used.yml content: spec/fixtures/scripts/internal_events/metrics/total_single_event.yml @@ -197,10 +209,8 @@ - "\n" # Select: config/events/internal_events_cli_used.yml - "\n" # Select: Weekly count of unique users - "who defined an internal event using the CLI\n" # Input description - - "\n" # Submit weekly description for monthly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "3\n" # Create another metric - "1\n" # Enum-select: Single event -- count occurrences of a specific event or user interaction - 'internal_events_cli_used' # Filters to this event @@ -213,10 +223,8 @@ - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_internal_events_cli_used_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_internal_events_cli_used_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml - path: config/metrics/counts_all/count_total_internal_events_cli_used.yml content: spec/fixtures/scripts/internal_events/metrics/total_single_event.yml @@ -232,10 +240,8 @@ - "\n" # Select: config/events/internal_events_cli_used.yml - "\n" # Select: Weekly count of unique users - "who defined an internal event using the CLI\n" # Input description - - "\n" # Submit weekly description for monthly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "2\n" # Create another metric - "\e[A" # Arrow up to: Total count of events - "\n" # Select: Total count of events @@ -256,10 +262,8 @@ - "4\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_internal_events_cli_used_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_internal_events_cli_used_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml - path: config/metrics/counts_all/count_total_internal_events_cli_used.yml content: spec/fixtures/scripts/internal_events/metrics/total_single_event.yml - path: ee/config/events/internal_events_cli_opened.yml @@ -324,18 +328,13 @@ - "\n" # Skip "Add extra property" filter - "who tried and failed to define an internal event using the CLI\n" # Input description - "failed_usage_attempts\n" # Input metric key path - - "\n" # Submit monthly description for weekly - - "\n" # Submit monthly name for weekly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_failed_usage_attempts_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_additional_props.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_failed_usage_attempts_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_additional_props.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_failed_usage_attempts.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event_additional_props.yml - description: Create a weekly/monthly metric for a single event with all additional properties inputs: @@ -355,18 +354,13 @@ - "60\n" # Input valid "value" - "who tried and failed to define an internal event using the CLI\n" # Input description - "failed_usage_attempts_under_60s\n" # Input metric key path - - "\n" # Submit weekly description for monthly - - "\n" # Submit weekly name for monthly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_failed_usage_attempts_under_60s_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_all_additional_props.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_failed_usage_attempts_under_60s_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_all_additional_props.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_failed_usage_attempts_under_60s.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event_all_additional_props.yml - description: Create a weekly/monthly metric for a single event with custom additional properties filters inputs: @@ -388,18 +382,13 @@ - "12.4\n" # Input value for "custom_key4" filter - "who tried and failed to define an internal event using the CLI\n" # Input description - "failed_usage_attempts_under_60s\n" # Input metric key path - - "\n" # Submit weekly description for monthly - - "\n" # Submit weekly name for monthly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_user_id_from_failed_usage_attempts_under_60s_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_custom_key_filter.yml - - path: config/metrics/counts_7d/count_distinct_user_id_from_failed_usage_attempts_under_60s_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_custom_key_filter.yml + - path: config/metrics/counts_all/count_distinct_user_id_from_failed_usage_attempts_under_60s.yml + content: spec/fixtures/scripts/internal_events/metrics/user_id_single_event_custom_key_filter.yml - description: Create a weekly/monthly metric for multiple events with and without additional properties inputs: @@ -479,17 +468,13 @@ - "\e[A\e[A\e[A\e[A" # Arrow up to: Weekly count of unique values for label - "\n" # Select: Weekly count of unique values for label - "values provided for label\n" # Input description - - "\n" # Submit monthly description for weekly - "1\n" # Enum-select: Copy & continue - "y\n" # Create file - - "y\n" # Create file - "5\n" # Exit outputs: files: - - path: config/metrics/counts_28d/count_distinct_label_from_internal_events_cli_used_monthly.yml - content: spec/fixtures/scripts/internal_events/metrics/label_28d_single_event_additional_props.yml - - path: config/metrics/counts_7d/count_distinct_label_from_internal_events_cli_used_weekly.yml - content: spec/fixtures/scripts/internal_events/metrics/label_7d_single_event_additional_props.yml + - path: config/metrics/counts_all/count_distinct_label_from_internal_events_cli_used.yml + content: spec/fixtures/scripts/internal_events/metrics/label_single_event_additional_props.yml - description: Creates metrics with explicitly no product category inputs: diff --git a/spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_7d.yml b/spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric.yml similarity index 81% rename from spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_7d.yml rename to spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric.yml index a11c778a75f8d22584be2f2e6dbdf2f9cbed78f3..1c114a620a1c3cc7fcb9c7dc82baa72774ddcbca 100644 --- a/spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_7d.yml +++ b/spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric.yml @@ -1,6 +1,6 @@ --- -key_path: redis_hll_counters.count_distinct_user_id_from_random_name_weekly -description: Weekly count of unique users random metric string +key_path: redis_hll_counters.count_distinct_user_id_from_random_name +description: Count of unique users random metric string product_group: import_and_integrate product_categories: - acquisition @@ -9,7 +9,9 @@ value_type: number status: active milestone: '16.6' introduced_by_url: TODO -time_frame: 7d +time_frame: +- 28d +- 7d data_source: internal_events data_category: optional distribution: diff --git a/spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_28d.yml b/spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_28d.yml deleted file mode 100644 index ef53203b2c963c23118a88789c55870829598bd7..0000000000000000000000000000000000000000 --- a/spec/fixtures/scripts/internal_events/metrics/keyboard_smashed_metric_28d.yml +++ /dev/null @@ -1,24 +0,0 @@ ---- -key_path: redis_hll_counters.count_distinct_user_id_from_random_name_monthly -description: Monthly count of unique users random metric string -product_group: import_and_integrate -product_categories: -- acquisition -performance_indicator_type: [] -value_type: number -status: active -milestone: '16.6' -introduced_by_url: TODO -time_frame: 28d -data_source: internal_events -data_category: optional -distribution: -- ce -- ee -tiers: -- free -- premium -- ultimate -events: -- name: random_name - unique: user.id diff --git a/spec/fixtures/scripts/internal_events/metrics/label_28d_single_event_additional_props.yml b/spec/fixtures/scripts/internal_events/metrics/label_28d_single_event_additional_props.yml deleted file mode 100644 index 192c9d8bfc82a4515f3669e9d96aa1bde96914e6..0000000000000000000000000000000000000000 --- a/spec/fixtures/scripts/internal_events/metrics/label_28d_single_event_additional_props.yml +++ /dev/null @@ -1,24 +0,0 @@ ---- -key_path: redis_hll_counters.count_distinct_label_from_internal_events_cli_used_monthly -description: Monthly count of unique values provided for label -product_group: analytics_instrumentation -product_categories: -- service_ping -performance_indicator_type: [] -value_type: number -status: active -milestone: '16.6' -introduced_by_url: TODO -time_frame: 28d -data_source: internal_events -data_category: optional -distribution: -- ce -- ee -tiers: -- free -- premium -- ultimate -events: -- name: internal_events_cli_used - unique: label diff --git a/spec/fixtures/scripts/internal_events/metrics/label_7d_single_event_additional_props.yml b/spec/fixtures/scripts/internal_events/metrics/label_single_event_additional_props.yml similarity index 80% rename from spec/fixtures/scripts/internal_events/metrics/label_7d_single_event_additional_props.yml rename to spec/fixtures/scripts/internal_events/metrics/label_single_event_additional_props.yml index 07a699da8e77f6e6c3cf05cd4d1eb73c51622165..d15d8a1caa5f33e399ff85ebaf1f6d898d9e4eff 100644 --- a/spec/fixtures/scripts/internal_events/metrics/label_7d_single_event_additional_props.yml +++ b/spec/fixtures/scripts/internal_events/metrics/label_single_event_additional_props.yml @@ -1,6 +1,6 @@ --- -key_path: redis_hll_counters.count_distinct_label_from_internal_events_cli_used_weekly -description: Weekly count of unique values provided for label +key_path: redis_hll_counters.count_distinct_label_from_internal_events_cli_used +description: Count of unique values provided for label product_group: analytics_instrumentation product_categories: - service_ping @@ -9,7 +9,9 @@ value_type: number status: active milestone: '16.6' introduced_by_url: TODO -time_frame: 7d +time_frame: +- 28d +- 7d data_source: internal_events data_category: optional distribution: diff --git a/spec/fixtures/scripts/internal_events/metrics/project_id_28d_multiple_events.yml b/spec/fixtures/scripts/internal_events/metrics/project_id_28d_multiple_events.yml deleted file mode 100644 index f51cdf5eaf9816d72b003f9f26a4d41fc8ca1974..0000000000000000000000000000000000000000 --- a/spec/fixtures/scripts/internal_events/metrics/project_id_28d_multiple_events.yml +++ /dev/null @@ -1,26 +0,0 @@ ---- -key_path: redis_hll_counters.count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_monthly -description: Monthly count of unique projects where a defition file was created with the CLI -product_group: analytics_instrumentation -product_categories: -- service_ping -performance_indicator_type: [] -value_type: number -status: active -milestone: '16.6' -introduced_by_url: TODO -time_frame: 28d -data_source: internal_events -data_category: optional -distribution: -- ce -- ee -tiers: -- free -- premium -- ultimate -events: -- name: internal_events_cli_closed - unique: project.id -- name: internal_events_cli_used - unique: project.id diff --git a/spec/fixtures/scripts/internal_events/metrics/project_id_7d_multiple_events.yml b/spec/fixtures/scripts/internal_events/metrics/project_id_multiple_events.yml similarity index 74% rename from spec/fixtures/scripts/internal_events/metrics/project_id_7d_multiple_events.yml rename to spec/fixtures/scripts/internal_events/metrics/project_id_multiple_events.yml index cb73f7d9095b1fcaf971e4a9f72f4622c95fdf47..5ca0c5585cf3397583046557d675c9b11914e5fc 100644 --- a/spec/fixtures/scripts/internal_events/metrics/project_id_7d_multiple_events.yml +++ b/spec/fixtures/scripts/internal_events/metrics/project_id_multiple_events.yml @@ -1,6 +1,6 @@ --- -key_path: redis_hll_counters.count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_weekly -description: Weekly count of unique projects where a defition file was created with the CLI +key_path: redis_hll_counters.count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used +description: Count of unique projects where a defition file was created with the CLI product_group: analytics_instrumentation product_categories: - service_ping @@ -9,7 +9,9 @@ value_type: number status: active milestone: '16.6' introduced_by_url: TODO -time_frame: 7d +time_frame: +- 28d +- 7d data_source: internal_events data_category: optional distribution: diff --git a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_additional_props.yml b/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_additional_props.yml deleted file mode 100644 index 97bb9f4904799d3ef240d7b25bec384cae9794bb..0000000000000000000000000000000000000000 --- a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_additional_props.yml +++ /dev/null @@ -1,30 +0,0 @@ ---- -key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_monthly -description: Monthly count of unique users who tried and failed to define an internal event using the CLI -product_group: analytics_instrumentation -product_categories: -- service_ping -performance_indicator_type: [] -value_type: number -status: active -milestone: '16.6' -introduced_by_url: TODO -time_frame: 28d -data_source: internal_events -data_category: optional -distribution: -- ce -- ee -tiers: -- free -- premium -- ultimate -events: -- name: internal_events_cli_used - unique: user.id - filter: - label: failure -- name: internal_events_cli_used - unique: user.id - filter: - label: incomplete diff --git a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_all_additional_props.yml b/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_all_additional_props.yml deleted file mode 100644 index c6014f02728108c6caec19b26ebb1b12a237890a..0000000000000000000000000000000000000000 --- a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_all_additional_props.yml +++ /dev/null @@ -1,46 +0,0 @@ ---- -key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_under_60s_monthly -description: Monthly count of unique users who tried and failed to define an internal event using the CLI -product_group: analytics_instrumentation -product_categories: -- service_ping -performance_indicator_type: [] -value_type: number -status: active -milestone: '16.6' -introduced_by_url: TODO -time_frame: 28d -data_source: internal_events -data_category: optional -distribution: -- ce -- ee -tiers: -- free -- premium -- ultimate -events: -- name: internal_events_cli_used - unique: user.id - filter: - label: failure - property: metrics - value: 60 -- name: internal_events_cli_used - unique: user.id - filter: - label: failure - property: events - value: 60 -- name: internal_events_cli_used - unique: user.id - filter: - label: incomplete - property: metrics - value: 60 -- name: internal_events_cli_used - unique: user.id - filter: - label: incomplete - property: events - value: 60 diff --git a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_custom_key_filter.yml b/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_custom_key_filter.yml deleted file mode 100644 index 7b54c7ce87f7c1110f4157dc885cc35c507adcce..0000000000000000000000000000000000000000 --- a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event_custom_key_filter.yml +++ /dev/null @@ -1,36 +0,0 @@ ---- -key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_under_60s_monthly -description: Monthly count of unique users who tried and failed to define an internal event using the CLI -product_group: analytics_instrumentation -product_categories: -- service_ping -performance_indicator_type: [] -value_type: number -status: active -milestone: '16.6' -introduced_by_url: TODO -time_frame: 28d -data_source: internal_events -data_category: optional -distribution: -- ce -- ee -tiers: -- free -- premium -- ultimate -events: -- name: internal_events_cli_used - unique: user.id - filter: - label: failure - custom_key1: metrics - custom_key3: 30 - custom_key4: 12.4 -- name: internal_events_cli_used - unique: user.id - filter: - label: failure - custom_key1: metrics - custom_key3: 13 - custom_key4: 12.4 diff --git a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml similarity index 76% rename from spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml rename to spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml index c8a469a10d3b64add212c03f17bdac5718f47788..60f5f2c1cb4ea951427ce13d210a7a8802d6a477 100644 --- a/spec/fixtures/scripts/internal_events/metrics/user_id_28d_single_event.yml +++ b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml @@ -1,6 +1,6 @@ --- -key_path: redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used_monthly -description: Monthly count of unique users who defined an internal event using the CLI +key_path: redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used +description: Count of unique users who defined an internal event using the CLI product_group: analytics_instrumentation product_categories: - service_ping @@ -9,7 +9,9 @@ value_type: number status: active milestone: '16.6' introduced_by_url: TODO -time_frame: 28d +time_frame: +- 28d +- 7d data_source: internal_events data_category: optional distribution: diff --git a/spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_additional_props.yml b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event_additional_props.yml similarity index 78% rename from spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_additional_props.yml rename to spec/fixtures/scripts/internal_events/metrics/user_id_single_event_additional_props.yml index f6990fc58f79985b7c90426c9f2e73c74b899b76..bad4df53205c60f9dabb06ca1adb2cdf367c0c63 100644 --- a/spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_additional_props.yml +++ b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event_additional_props.yml @@ -1,6 +1,6 @@ --- -key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_weekly -description: Weekly count of unique users who tried and failed to define an internal event using the CLI +key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts +description: Count of unique users who tried and failed to define an internal event using the CLI product_group: analytics_instrumentation product_categories: - service_ping @@ -9,7 +9,9 @@ value_type: number status: active milestone: '16.6' introduced_by_url: TODO -time_frame: 7d +time_frame: +- 28d +- 7d data_source: internal_events data_category: optional distribution: diff --git a/spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_all_additional_props.yml b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event_all_additional_props.yml similarity index 84% rename from spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_all_additional_props.yml rename to spec/fixtures/scripts/internal_events/metrics/user_id_single_event_all_additional_props.yml index 20d83c814d2b86ca3f857949b0660888fbac19db..3918eff966ca25a359b5bcbaed43f8ff8e0022c1 100644 --- a/spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_all_additional_props.yml +++ b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event_all_additional_props.yml @@ -1,6 +1,6 @@ --- -key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_under_60s_weekly -description: Weekly count of unique users who tried and failed to define an internal event using the CLI +key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_under_60s +description: Count of unique users who tried and failed to define an internal event using the CLI product_group: analytics_instrumentation product_categories: - service_ping @@ -9,7 +9,9 @@ value_type: number status: active milestone: '16.6' introduced_by_url: TODO -time_frame: 7d +time_frame: +- 28d +- 7d data_source: internal_events data_category: optional distribution: diff --git a/spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_custom_key_filter.yml b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event_custom_key_filter.yml similarity index 81% rename from spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_custom_key_filter.yml rename to spec/fixtures/scripts/internal_events/metrics/user_id_single_event_custom_key_filter.yml index b62360968fb3fa55436369f85250bd2e9f72e630..4d0d64052a0d5c308189bca33779a7ff7611a152 100644 --- a/spec/fixtures/scripts/internal_events/metrics/user_id_7d_single_event_custom_key_filter.yml +++ b/spec/fixtures/scripts/internal_events/metrics/user_id_single_event_custom_key_filter.yml @@ -1,6 +1,6 @@ --- -key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_under_60s_weekly -description: Weekly count of unique users who tried and failed to define an internal event using the CLI +key_path: redis_hll_counters.count_distinct_user_id_from_failed_usage_attempts_under_60s +description: Count of unique users who tried and failed to define an internal event using the CLI product_group: analytics_instrumentation product_categories: - service_ping @@ -9,7 +9,9 @@ value_type: number status: active milestone: '16.6' introduced_by_url: TODO -time_frame: 7d +time_frame: +- 28d +- 7d data_source: internal_events data_category: optional distribution: diff --git a/spec/scripts/internal_events/cli/flows/metric_definer_spec.rb b/spec/scripts/internal_events/cli/flows/metric_definer_spec.rb index 37f1f1feacb975058776f3c5f3717f383f4b71ca..0217e7d960ffd919ad913646771afacf4cf51cd9 100644 --- a/spec/scripts/internal_events/cli/flows/metric_definer_spec.rb +++ b/spec/scripts/internal_events/cli/flows/metric_definer_spec.rb @@ -48,8 +48,7 @@ " ", # Multi-select: __event2 "\n", # Submit selections "\n", # Select: Weekly/Monthly count of unique users - "aggregate metric description\n", # Submit description - "\n" # Accept description for weekly + "aggregate metric description\n" # Submit description ]) # Filter down to "dev" options @@ -81,8 +80,7 @@ " ", # Multi-select: __event3 "\n", # Submit selections "\n", # Select: Weekly/Monthly count of unique users - "aggregate metric description\n", # Submit description - "\n" # Accept description for weekly + "aggregate metric description\n" # Submit description ]) # Filter down to "dev:create" options @@ -118,7 +116,6 @@ "\n", # Select: 00__event1 "\n", # Select: Weekly/Monthly count of unique users "aggregate metric description\n", # Submit description - "\n", # Accept description for weekly "2\n" # Modify attributes ]) @@ -220,14 +217,9 @@ def select_event_from_list # existing metrics which use both events File.write( - 'config/metrics/counts_7d/' \ - 'count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_weekly.yml', - File.read('spec/fixtures/scripts/internal_events/metrics/project_id_7d_multiple_events.yml') - ) - File.write( - 'config/metrics/counts_28d/' \ - 'count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_monthly.yml', - File.read('spec/fixtures/scripts/internal_events/metrics/project_id_28d_multiple_events.yml') + 'config/metrics/counts_all/' \ + 'count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used.yml', + File.read('spec/fixtures/scripts/internal_events/metrics/project_id_multiple_events.yml') ) # Non-conflicting metric which uses only one of the events @@ -434,4 +426,77 @@ def select_event_from_list end end end + + context 'when succeeded in saving the file' do + let(:events) do + [{ + action: 'internal_events_cli_closed', internal_events: true, product_group: 'optimize', tiers: ['ultimate'] + }, { + action: 'internal_events_cli_used', internal_events: true, product_group: 'optimize', tiers: ['ultimate'] + }] + end + + before do + events.each do |event| + File.write("config/events/#{event[:action]}.yml", event.transform_keys(&:to_s).to_yaml) + end + end + + context "when creating a single metric" do + it 'shows link to the metric dashboard' do + queue_cli_inputs([ + "2\n", # Enum-select: New Metric -- calculate how often one or more existing events occur over time + "2\n", # Enum-select: Multiple events -- count occurrences of several separate events or interactions + 'internal_events_cli', # Filters to the relevant events + ' ', # Multi-select: internal_events_cli_closed + "\e[B", # Arrow down to: internal_events_cli_used + ' ', # Multi-select: internal_events_cli_used + "\n", # Submit selections + "\e[B", # Arrow down to: Total count + "\n", # Select: Total count + "where a definition file was created with the CLI\n", # Input description + "1\n", # Select: Copy & continue + "\e[B \n", # Skip product categories + "y\n" # Create file + ]) + + expected_output = <<~TEXT.chomp + - Metric trend dashboard: https://10az.online.tableau.com/#/site/gitlab/views/PDServicePingExplorationDashboard/MetricTrend?Metrics%20Path=counts.count_total_internal_events_cli_closed_and_internal_events_cli_used + TEXT + + with_cli_thread do + expect { plain_last_lines }.to eventually_include_cli_text(expected_output) + end + end + end + + context "when creating a multiple metrics" do + it 'shows link to the metric dashboard' do + queue_cli_inputs([ + "2\n", # Enum-select: New Metric -- calculate how often one or more existing events occur over time + "2\n", # Enum-select: Multiple events -- count occurrences of several separate events or interactions + 'internal_events_cli', # Filters to the relevant events + ' ', # Multi-select: internal_events_cli_closed + "\e[B", # Arrow down to: internal_events_cli_used + ' ', # Multi-select: internal_events_cli_used + "\n", # Submit selections + "\n", # Select: Weekly/Monthly count + "where a definition file was created with the CLI\n", # Input description + "1\n", # Select: Copy & continue + "\e[B \n", # Skip product categories + "y\n" # Create file + ]) + + expected_output = <<-TEXT.chomp # <<- used instead of <<~ to save indentation + - Metric trend dashboards: + - https://10az.online.tableau.com/#/site/gitlab/views/PDServicePingExplorationDashboard/MetricTrend?Metrics%20Path=counts.count_total_internal_events_cli_closed_and_internal_events_cli_used_monthly + - https://10az.online.tableau.com/#/site/gitlab/views/PDServicePingExplorationDashboard/MetricTrend?Metrics%20Path=counts.count_total_internal_events_cli_closed_and_internal_events_cli_used_weekly + TEXT + + with_cli_thread do + expect { plain_last_lines }.to eventually_include_cli_text(expected_output) + end + end + end + end end diff --git a/spec/scripts/internal_events/cli/flows/usage_viewer_spec.rb b/spec/scripts/internal_events/cli/flows/usage_viewer_spec.rb index ec1e25cd437dd5a4eb419200ede8e447d382e623..b474498ce2aecdba2bd23ca1d31109bbda473bf6 100644 --- a/spec/scripts/internal_events/cli/flows/usage_viewer_spec.rb +++ b/spec/scripts/internal_events/cli/flows/usage_viewer_spec.rb @@ -131,6 +131,111 @@ end end + context 'for an event with multiple metrics' do + let(:expected_rails_example) do + <<~TEXT.chomp + -------------------------------------------------- + # RAILS + + include Gitlab::InternalEventsTracking + + track_internal_event( + 'internal_events_cli_used', + project: project, + user: user + ) + + -------------------------------------------------- + TEXT + end + + let(:expected_rspec_example) do + <<~TEXT.chomp + -------------------------------------------------- + # RSPEC + + it_behaves_like 'internal event tracking' do + let(:event) { 'internal_events_cli_used' } + let(:project) { create(:project) } + let(:user) { create(:user) } + end + + -------------------------------------------------- + TEXT + end + + let(:expected_gdk_example) do + <<~TEXT.chomp + -------------------------------------------------- + # RAILS CONSOLE -- generate service ping payload, including most recent usage data + + require_relative 'spec/support/helpers/service_ping_helpers.rb' + + # Get current value of a metric + ServicePingHelpers.get_current_usage_metric_value('redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used_monthly') + ServicePingHelpers.get_current_usage_metric_value('redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used_weekly') + + # View entire service ping payload + ServicePingHelpers.get_current_service_ping_payload + -------------------------------------------------- + TEXT + end + + let(:expected_tableau_example) do + <<~TEXT.chomp + -------------------------------------------------- + # GROUP DASHBOARDS -- view all service ping metrics for a specific group + + analytics_instrumentation: https://10az.online.tableau.com/#/site/gitlab/views/PDServicePingExplorationDashboard/MetricExplorationbyGroup?Group%20Name=analytics_instrumentation&Stage%20Name=monitor + + -------------------------------------------------- + # METRIC TRENDS -- view data for a service ping metric for internal_events_cli_used + + redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used_monthly: https://10az.online.tableau.com/#/site/gitlab/views/PDServicePingExplorationDashboard/MetricTrend?Metrics%20Path=redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used_monthly + redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used_weekly: https://10az.online.tableau.com/#/site/gitlab/views/PDServicePingExplorationDashboard/MetricTrend?Metrics%20Path=redis_hll_counters.count_distinct_user_id_from_internal_events_cli_used_weekly + + -------------------------------------------------- + Note: The metric dashboard links can also be accessed from https://metrics.gitlab.com/ + + Not what you're looking for? Check this doc: + - https://docs.gitlab.com/ee/development/internal_analytics/#data-discovery + TEXT + end + + before do + File.write(event1_filepath, File.read(event1_content)) + File.write( + 'config/metrics/counts_all/count_distinct_user_id_from_internal_events_cli_used.yml', + File.read('spec/fixtures/scripts/internal_events/metrics/user_id_single_event.yml') + ) + end + + it 'shows backend examples for all metrics' do + queue_cli_inputs([ + "3\n", # Enum-select: View Usage -- look at code examples for an existing event + 'internal_events_cli_used', # Filters to this event + "\n", # Select: config/events/internal_events_cli_used.yml + "\n", # Select: ruby/rails + "\e[B", # Arrow down to: rspec + "\n", # Select: rspec + "7\n", # Select: Manual testing: check current values of metrics from rails console (any data source) + "8\n", # Select: Data verification in Tableau + "Exit", # Filters to this item + "\n" # select: Exit + ]) + + with_cli_thread do + expect { plain_last_lines(200) }.to eventually_include_cli_text( + expected_example_prompt, + expected_rails_example, + expected_rspec_example, + expected_gdk_example, + expected_tableau_example + ) + end + end + end + context 'for an event without identifiers' do let(:expected_rails_example) do <<~TEXT.chomp diff --git a/spec/scripts/internal_events/cli/helpers/metric_options_spec.rb b/spec/scripts/internal_events/cli/helpers/metric_options_spec.rb index c855c83aaef6ba4d1cdc1d8f499535198fe9e8f0..aa3eb3ba7e3290e35af6fb63980a9fbe726518f5 100644 --- a/spec/scripts/internal_events/cli/helpers/metric_options_spec.rb +++ b/spec/scripts/internal_events/cli/helpers/metric_options_spec.rb @@ -41,7 +41,7 @@ context 'when option is for a supported and not yet defined metric' do it 'highlights key words in the name' do expect(option.formatted).to eq({ - name: "<cyan>time frame</cyan> count of <cyan>unique users</cyan> who triggered a list of events", + name: "<cyan>Time frame</cyan> count of <cyan>unique users</cyan> who triggered a list of events", value: metrics }) end @@ -51,7 +51,7 @@ it 'highlights key words in the name' do expect(option.formatted).to eq({ - name: "<cyan>time frame</cyan> count of <cyan>unique users</cyan> who triggered a list of events " \ + name: "<cyan>Time frame</cyan> count of <cyan>unique users</cyan> who triggered a list of events " \ "<cyan>where label/prop/anything</cyan> is...", value: metrics }) @@ -64,7 +64,7 @@ it 'formats the option as disabled' do expect(option.formatted).to eq({ - name: "<bright_black>time frame count of unique users who triggered a list of events</bright_black>", + name: "<bright_black>Time frame count of unique users who triggered a list of events</bright_black>", value: metrics, disabled: "<bold><bright_black>(already defined)</bright_black></bold>" }) @@ -75,7 +75,7 @@ it 'highlights key words in the name' do expect(option.formatted).to eq({ - name: "<bright_black>time frame count of unique users who triggered " \ + name: "<bright_black>Time frame count of unique users who triggered " \ "a list of events where filtered</bright_black>", value: metrics, disabled: "<bold><bright_black>(already defined)</bright_black></bold>" @@ -89,7 +89,7 @@ it 'formats the option as disabled' do expect(option.formatted).to eq({ - name: "<bright_black>time frame count of unique users who triggered a list of events</bright_black>", + name: "<bright_black>Time frame count of unique users who triggered a list of events</bright_black>", value: metrics, disabled: "<bold><bright_black>(user unavailable)</bright_black></bold>" }) @@ -100,7 +100,7 @@ it 'highlights key words in the name' do expect(option.formatted).to eq({ - name: "<bright_black>time frame count of unique users who triggered " \ + name: "<bright_black>Time frame count of unique users who triggered " \ "a list of events where filtered</bright_black>", value: metrics, disabled: "<bold><bright_black>(user unavailable)</bright_black></bold>" @@ -114,7 +114,7 @@ it 'highlights key words in the name' do expect(option.formatted).to eq({ - name: "<cyan>time frame</cyan> count of <cyan>unique values for 'label'</cyan> " \ + name: "<cyan>Time frame</cyan> count of <cyan>unique values for 'label'</cyan> " \ "from a list of events occurrences", value: metrics }) @@ -126,7 +126,7 @@ it 'highlights key words in the name' do expect(option.formatted).to eq({ - name: "<cyan>time frame</cyan> count of a list of events occurrences", + name: "<cyan>Time frame</cyan> count of a list of events occurrences", value: metrics }) end @@ -144,7 +144,7 @@ it 'highlights key words in the name' do expect(option.formatted).to eq({ - name: "<cyan>time frame 1/time frame 2</cyan> count of <cyan>unique users</cyan> " \ + name: "<cyan>Time frame 1/Time frame 2</cyan> count of <cyan>unique users</cyan> " \ "who triggered a list of events", value: metrics }) diff --git a/spec/scripts/internal_events/cli_spec.rb b/spec/scripts/internal_events/cli_spec.rb index 698c79a2bec93df3d51e576cc5e1f0e54e16be4e..73c388c591b52c49e9771c4968449a662e9df3a1 100644 --- a/spec/scripts/internal_events/cli_spec.rb +++ b/spec/scripts/internal_events/cli_spec.rb @@ -82,14 +82,12 @@ "\e[B", # Arrow down to: Weekly count of unique projects "\n", # Select: Weekly count of unique projects "where a defition file was created with the CLI\n", # Input description - "\n", # Submit weekly description for monthly "2\n", # Select: Modify attributes "\n", # Accept group "\n", # Accept product categories "\n", # Skip URL "1\n", # Select: [free, premium, ultimate] "y\n", # Create file - "y\n", # Create file "5\n" # Exit ] end @@ -104,11 +102,8 @@ let(:output_files) do # rubocop:disable Layout/LineLength -- Long filepaths read better unbroken [{ - 'path' => 'config/metrics/counts_28d/count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_monthly.yml', - 'content' => 'spec/fixtures/scripts/internal_events/metrics/project_id_28d_multiple_events.yml' - }, { - 'path' => 'config/metrics/counts_7d/count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used_weekly.yml', - 'content' => 'spec/fixtures/scripts/internal_events/metrics/project_id_7d_multiple_events.yml' + 'path' => 'config/metrics/counts_all/count_distinct_project_id_from_internal_events_cli_closed_and_internal_events_cli_used.yml', + 'content' => 'spec/fixtures/scripts/internal_events/metrics/project_id_multiple_events.yml' }] # rubocop:enable Layout/LineLength end