Add reporting logic

We want to know exactly how many times we auto-retried
a CI/CD job due to a flaky test.

This simple reporting will help us do just that.