Consider adding the `FREQUENCY=2-hourly` variable to the scheduled pipeline to ensure cache updating jobs run

The cache updating jobs only run if the FREQUENCY variable is set to 2-hourly: https://gitlab.com/gitlab-jh/gitlab/-/blob/7eb9992b7e06eb45a68f9ecd88f82a6d34c323df/.gitlab/ci/rules.gitlab-ci.yml#L312

@godfat-gitlab Can you add the variable?

assigned to @godfat-gitlab

By Kyle Wiebers on 2021-06-15T11:30:10 (imported from GitLab)

@godfat-gitlab - These pipelines seem to have recovered, there's not an urgent need to do this like there was on Friday. https://gitlab.com/gitlab-jh/gitlab/-/pipelines/320863803

By Kyle Wiebers on 2021-06-15T11:35:30 (imported from GitLab)

We don't really have a scheduled pipeline here... now we have a code syncing scheduled pipeline, but that doesn't suppose to run anything other than code syncing.

As a result, this is probably more for adding a scheduled pipeline on the default branch. However I am not exactly sure if we want to do that yet, I suppose we want, but that would be eating more pipeline minutes and we might not want that. We already run double pipelines on the default branch due to https://gitlab.com/gitlab-jh/gitlab/-/issues/78 and I am worrying that adding yet another one will be too much.

As a short term solution, I made a merge request to run the jobs instead: https://gitlab.com/gitlab-jh/gitlab/-/merge_requests/30

This will at least make sure the cache is up-to-date for this moment, which should help.

And to be transparent: I accidentally pushed to main-jh directly. Apparently this is also a signal that I need to rest now. I quickly force pushed (sorry! ) and reverted to the previous commit right after a few seconds. I believe no one is working at this moment. I won't do this at the scale of GitLab Inc but for JH I believe this is ok, at this very moment. @qianzhangxa Sorry about that!

At some point we'll need to revisit the permission setting as well. We're not allowing maintainers to push to default branch at gitlab-org/gitlab>

And we shouldn't at JH as well. But we'll need to make sure the bot can push otherwise the code syncing will fail.

I can revisit this tomorrow.

By Lin Jen-Shin on 2021-06-15T18:44:14 (imported from GitLab)

@godfat-gitlab No problem. Can you please let me know what updating cache is for? What kind of cache do we want to update?

By Qian Zhang on 2021-06-16T11:44:32 (imported from GitLab)

@qianzhangxa This is documented at https://docs.gitlab.com/ee/development/pipelines.html#caching-strategy

Generally, merge requests are using read-only cache, and we run scheduled default branch pipeline to update the cache using in merge requests (and in default branch itself, of course).

This way we avoid constantly updating cache.

By Lin Jen-Shin on 2021-06-16T11:44:32 (imported from GitLab)

Thanks @godfat-gitlab! For the scheduled default branch pipeline, do you mean the the scheduled pipeline [2-hourly] Repo caching, Review Apps cleanup defined at https://gitlab.com/gitlab-org/gitlab/-/pipeline_schedules?

By Qian Zhang on 2021-06-16T13:19:39 (imported from GitLab)

@qianzhangxa Yes, that's exactly it.

By Lin Jen-Shin on 2021-06-16T15:28:09 (imported from GitLab)

I made the permissions change and explained at https://gitlab.com/gitlab-jh/gitlab/-/issues/93

By Lin Jen-Shin on 2021-06-16T17:21:20 (imported from GitLab)

mentioned in issue #93 (closed)

By Lin Jen-Shin on 2021-06-16T17:19:54 (imported from GitLab)

marked this issue as related to #78 (closed)

By Lin Jen-Shin on 2021-06-17T15:39:02 (imported from GitLab)

I'll set a scheduled pipeline after https://gitlab.com/gitlab-jh/gitlab/-/issues/78 so we don't run double scheduled pipelines.

By Lin Jen-Shin on 2021-06-28T00:37:18 (imported from GitLab)

@godfat-gitlab I see https://gitlab.com/gitlab-jh/gitlab/-/issues/78 has been closed, are you going to set a scheduled pipeline?

By Qian Zhang on 2021-06-28T00:37:18 (imported from GitLab)

Yes, done!

By Lin Jen-Shin on 2021-06-28T13:56:35 (imported from GitLab)