How a pain point turned into an open-source project.
As developers, we often think about performance, naming or efficient algorithms.
But one of the biggest productivity and quality gains, comes from something much simpler.
It all started with a failed CI run
A few weeks ago, I contributed to NodeSecure Scanner.
If you haven't heard about it, NodeSecure Scanner inspects dependencies, package metadata and identify potential security risks.
The project is organized as a monorepo using npm workspaces.
- I opened a pull request.
- The CI failed.
- Nothing unusual, except I was a little bit ashamed.
So, again, I did what every developer does.
npm test --workspaces
The failure was there, but I simply couldn't see it anymore because it scrolled off screen, silently telling me:
too late, goodbye mate, see you in CI 😘
npm Workspaces are simple... until they aren't.
Like code, it starts breaking down as the repository grows.
A single command now mixes together:
- successful test output
- failure stack traces
- snapshots
- warnings
- logs
- coverage
- status
- tomato
- salad
- onions
This is a classic signal to noise problem.
The problem wasn't the failing test.
The problem was the feedback loop.
It turns out to be more than a one-liner.
Tim Cochran, writing on Martin Fowler website, argues that some of the biggest gains in developer productivity come from optimizing the feedback loops developers go through dozens or hundreds of times a day.
Running a test suite is exactly that: something you do constantly, almost without thinking about it, until the moment it breaks down.
He makes the point that these loops are easy to dismiss because each one only costs a few seconds or minutes.
It's hard to justify investing engineering time to shave a two-minute step down to fifteen seconds. But those seconds compound:
- A few extra minutes scrolling through logs
- multiplied across every CI run
- every day, across a whole team
What looks like a minor annoyance becomes real, measurable friction.
That's the lens I ended up applying to my own problem, not "how do I make this test suite pass," but "how do I make the loop of running it, reading it, and acting on it shorter."
My first solution was just 3 lines of code 🎉
Whenever I encounter a recurring problem, I try the simplest thing first, so I wrote this:
for ws in $(node -p "require('./package.json').workspaces.join(' ')");
do npm run test --workspace="$ws" || exit 1
done
The script was intentionally simple.
It iterated over every workspace, executed its test script and immediately stopped when one of them failed.
In fact, I ended up sharing it in a Stack Overflow answer because other developers were asking the same question.
Then I realized I had simply traded one problem for another.
Sequential execution made the feedback loop shorter when something failed early…
…but it also made every successful run slower.
The obvious question became:
Why should independent workspaces wait for each other?
I wanted both:
- concurrent execution.
- readable output.
That was the moment where a tiny Bash script stopped being enough and
became an opportunity to rethink the testing experience itself.
That's when Sumlyzer CLI appeared:
I decided to orchestrate workspace execution myself.
The idea was surprisingly simple:
- execute workspace tests concurrently;
- keep each workspace output isolated;
- only display detailed logs for failing workspaces;
- finish with a concise summary showing exactly where to look.
Instead of this:
You get something closer to:
Getting started is straightforward.
npm install -D sumlyzer
npx sumlyzer
Run with the fail fast mode.
npx sumlyzer --ff
Need to plug your results into a reporting tool (GitLab, Jenkins, or anything that reads test reports)?
Sumlyzer can also aggregate every workspace's output into a single JUnit XML file and also folds each workspace's logs into collapsible groups for GitHub Actions.
npx sumlyzer --ff --junit
Want faster feedback? Add the concurrency option.
npx sumlyzer --ff --c 2
Under the hood: how the scheduling works
So each workspace test run picks its own work, instead of being assigned one:
let nextIndex = 0;
async function worker() {
while (!stopScheduling && nextIndex < workspacesToRun.length) {
const index = nextIndex++;
const wsPath = workspacesToRun[index];
// run tests for wsPath...
}
}
await Promise.all(Array.from({ length: workerCount }, worker));
Think of it like a buffet instead of table service.
Here's the part that surprised me the first time I really thought about it: this needs zero locking, even though several worker() calls run "at the same time."
In a language with real OS-level threads, two threads could read nextIndex at the exact same moment and grab the same task so you'd need a mutex or an atomic counter to prevent that.
But JavaScript only ever executes one line at a time.
const index = nextIndex++; runs to completion before the event loop can hand control to anything else.
The concurrency here isn't happening inside JavaScript. It's happening between the child processes Sumlyzer spawns. The scheduling glue that decides who runs what stays trivially safe, for free, just because of how Node.js runs code.
At this point, the original goal-finding failing tests faster-had already been solved.
But the project was only getting started.
The hardest part isn't writing code anymore
John Ousterhout writes in A Philosophy of Software Design that managing complexity is one of the central challenges of software design.
Maintaining a small open-source tool made me appreciate that idea even more.
Writing features is enjoyable.
Deciding not to implement one is much harder.
Every new option makes the tool more flexible but also harder to understand.
The goal isn't to accumulate features.
The goal is to preserve a good developer experience as the project evolves.
Today, I probably spend more time thinking about scope than writing code.
Looking back
The funny part is that none of this started with an ambition to build an open-source project.
It started with a failing CI build.
Then a Bash script.
Then a better execution model.
Then a better reporting model.
And eventually, an open-source project centered around one simple idea:
A failing test shouldn't require scrolling through thousands of log lines just to maybe never find it.
If you're working with npm workspaces, I'd love to hear how you're approaching this problem.
If you'd like to try Sumlyzer, report an issue or contribute.
I'm especially interested in hearing about workflows I haven't considered yet.
That's usually where the next good idea comes from.
