"Analytics::CycleAnalytics::ConsistencyWorker" job 失败报错
Summary
从日中中查到有个 Sidekiq "Analytics::CycleAnalytics::ConsistencyWorker" job 失败的报错，从周三晚上 9:00 开始的，可能是跟系统更新有关系。
报错信息是 Incorrect cursor values were given. Missing items: end_event_timestamp。
点击查看 backtrace
lib/gitlab/pagination/keyset/order.rb:276:in `verify_incoming_values!',
lib/gitlab/pagination/keyset/order.rb:148:in `build_where_values',
lib/gitlab/pagination/keyset/order.rb:168:in `apply_cursor_conditions',
lib/gitlab/pagination/keyset/iterator.rb:21:in `block in each_batch',
lib/gitlab/pagination/keyset/iterator.rb:19:in `loop',
lib/gitlab/pagination/keyset/iterator.rb:19:in `each_batch',
ee/app/services/analytics/cycle_analytics/consistency_check_service.rb:26:in `block in execute',
ee/app/services/analytics/cycle_analytics/consistency_check_service.rb:22:in `each',
ee/app/services/analytics/cycle_analytics/consistency_check_service.rb:22:in `execute',
ee/app/workers/analytics/cycle_analytics/consistency_worker.rb:60:in `run_consistency_check_services',
ee/app/workers/analytics/cycle_analytics/consistency_worker.rb:30:in `block (2 levels) in perform',
ee/app/workers/analytics/cycle_analytics/consistency_worker.rb:29:in `block in perform',
ee/app/workers/analytics/cycle_analytics/consistency_worker.rb:25:in `loop',
ee/app/workers/analytics/cycle_analytics/consistency_worker.rb:25:in `perform',
lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call',
lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call',
lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform',
lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform',
lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call',
lib/gitlab/application_context.rb:118:in `block in use',
lib/gitlab/application_context.rb:118:in `use',
lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context',
lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call',
lib/gitlab/application_context.rb:118:in `block in use',
lib/gitlab/application_context.rb:118:in `use',
lib/gitlab/application_context.rb:57:in `with_context',
lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call',
lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call',
lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call',
lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call',
lib/gitlab/database/query_analyzer.rb:37:in `within',
lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call',
lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call',
lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call',
lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call',
lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call',
lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call',
lib/gitlab/with_request_store.rb:17:in `enabling_request_store',
lib/gitlab/with_request_store.rb:10:in `with_request_store',
lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call',
lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call',
lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument',
lib/gitlab/metrics/background_transaction.rb:33:in `run',
lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument',
lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call',
lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call',
lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job',
lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call',
lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call',
lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'可能要劳烦研发同学帮忙定位一下原因了。
/cc @qlxiang
Steps to reproduce
Example Project
What is the current bug behavior?
What is the expected correct behavior?
Relevant logs and/or screenshots
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)(we will only investigate if the tests are passing)