Why change-impact analysis is surprisingly hard in Ruby on Rails

Refactoring a mature Rails app often starts with a simple question:

If I change this method, what else could break?

In a smaller codebase, grep might be enough.

In a larger Rails app, not really.

A method can be connected through:

And because Ruby is dynamic, even resolving something that looks like a simple method call can become ambiguous.

That made change-impact analysis an interesting problem to work on.

The rule I chose early

I'd rather miss an ambiguous relationship than confidently invent one.

So instead of pretending every relationship is certain, the analyzer records evidence and confidence around what it finds.

For example:

ripple-effect inspect 'BillingService#charge'

The goal is to answer questions like:

You can also inspect the impact of a branch:

ripple-effect diff main

RippleEffect is an open-source Ruby gem for static change-impact analysis in Rails apps.

It does not:

Everything is analyzed locally and deterministically.

I've been testing the first release against real Rails codebases including Lobsters, RubyGems.org, Solidus, and Mastodon.

There are still plenty of hard cases in dynamic Ruby, which is exactly what makes the problem interesting.

If you work on large Rails apps, I'd be curious:

How do you currently answer "what else could this change affect?" before a refactor?