Skip to content
代码片段 群组 项目
未验证 提交 1eb6b7f5 编辑于 作者: Peter Leitzen's avatar Peter Leitzen
浏览文件

Run specs in random order multiple times

Sometimes, running specs in random order only once is not sufficient to
catch flaky specs.
上级 8097dbfc
No related branches found
No related tags found
无相关合并请求
......@@ -11,15 +11,18 @@
# The following checks are available:
# * Run specs in _defined_ order
# * Run specs in _reverse_ order
# * Run specs in _random_ order
# * Run specs in _random_ order, 5 times by default.
# - Adjustable via RANDOM_ORDER_RUNS=10 scripts/rspec_check_order_dependence
if [ $# -eq 0 ]; then
echo "Usage: $0 <files...>"
echo " RANDOM_ORDER_RUNS=5 $0 <files...>"
exit
fi
TODO_YAML='./spec/support/rspec_order_todo.yml'
RSPEC_ARGS=(--format progress)
RANDOM_ORDER_RUNS=${RANDOM_ORDER_RUNS:-5}
abort() {
echo "$@"
......@@ -42,9 +45,17 @@ set -xe
export RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
echo "Running in defined order:"
bin/rspec --order defined "${RSPEC_ARGS[@]}" "$@"
echo "Running in reverse order:"
RSPEC_ORDER=reverse bin/rspec "${RSPEC_ARGS[@]}" "$@"
bin/rspec --order random "${RSPEC_ARGS[@]}" "$@"
for try in $(seq "$RANDOM_ORDER_RUNS")
do
echo "Running in random order ($try/$RANDOM_ORDER_RUNS):"
bin/rspec --order random "${RSPEC_ARGS[@]}" "$@"
done
set +xe
......
0% 加载中 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册