RSpec Integration with Mergify

This guide explains how to integrate RSpec with Test Insights using the rspec-mergify gem. Once installed, test results are automatically uploaded to Test Insights without any extra workflow changes.

You need to install the rspec-mergify gem to automatically upload your test results to Test Insights.

Add the gem to your Gemfile:

group :test do
  gem 'rspec-mergify'
end

Then run:

bundle install

Alternatively, install it directly:

gem install rspec-mergify

Your workflow should run your tests as usual while exporting the secret MERGIFY_TOKEN as an environment variable.

Add the following to the GitHub Actions step running your tests:

env:
  MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}

For example:

- name: Run RSpec Tests 🧪
  env:
    MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
  run: bundle exec rspec

Set MERGIFY_TOKEN in the environment of the agents running your tests. The step itself then needs no Mergify-specific configuration:

steps:
  - label: "Run RSpec Tests 🧪"
    command: bundle exec rspec

The gem automatically collects your test results and sends them to Test Insights.

The gem applies quarantine inside the RSpec run. When a quarantined example fails, the gem marks it pending with the message Test is quarantined from Mergify Test Insights, so it does not count as a failure and does not change RSpec’s exit code. The exit code of your test step already accounts for quarantine, so the step needs nothing more:

• Do not add continue-on-error: true. The recipes that upload a JUnit report with the mergifyio/gha-mergify-ci action need it because the action decides the job’s result after the tests. Here nothing does, so it would let every real failure through.

• There is no step id to set and no test_step_outcome to pass: both belong to that action, which this setup does not use.

A crash cannot pass for a green run either. Quarantine only covers failures inside an example: a spec file that fails to load, a failing before(:suite) or after(:context) hook, or RSpec dying mid-run still exits non-zero and fails the step. The gem uploads results when the run ends, so a process killed outright before then, such as by the out-of-memory killer, sends nothing to Test Insights.

If the gem cannot fetch the quarantine list, it quarantines nothing for that run, and a quarantined example that fails makes the step fail as usual.

After pushing these changes, your next CI run reports its RSpec results automatically.

You can then review your test results, including any failures or flaky tests, directly in the Test Insights dashboard.