The first version of my image-to-Minecraft-blocks converter locked up the browser tab for long enough that I assumed something had crashed. I dropped in a photo straight off my phone, hit convert, and the page stopped responding to clicks — no spinner animation, no scroll, nothing. I sat there for a while wondering whether I'd written an infinite loop.
I hadn't. It was just doing exactly what I told it to, on the main thread, one pixel at a time.
This post is about what I got wrong when I tried to fix it, because my first guess was confidently wrong and the actual bottleneck turned out to be somewhere I wasn't looking.
The naive shape of the problem
The job is simple to state. You have a photo. You have a fixed list of Minecraft blocks, each with a known average color. For every cell in the output grid you find the block whose color is closest, and you write it down.
The obvious implementation is two nested loops: outer over pixels, inner over the palette, keep the minimum. Textbook nearest-neighbor search, and for small inputs it's genuinely fine.
Then you run the numbers on real inputs. A photo out of a modern phone camera is around 4000×3000 — twelve million pixels before you touch anything. You downscale before matching, sure, since the output grid is maybe 128 or 256 blocks wide. But the decode and resample still happen at full resolution, and then the match loop runs every cell of the output grid against every entry in a palette of several hundred blocks.
And I was running all of it inside a click handler.
My wrong guess
My first theory was the color space. Matching happens in OKLab rather than raw RGB, because RGB distance produces the classic "cheap filter" look — most visibly on skin tones and mid-range greens, where a numerically small gap can read as a very different color to a human eye. OKLab conversion involves a matrix multiply and cube roots, and cube roots felt expensive, so I assumed that was where the time was going.
It wasn't. When I actually profiled it instead of guessing, the color conversion barely registered. Nearly everything was sitting in the palette scan — the inner loop — and in the plain fact that all of this was happening on the thread responsible for painting the page.
Two separate problems wearing one costume:
- The work was on the main thread, so the UI froze regardless of how fast the work was.
- The work was genuinely redundant, because photos repeat colors constantly and I was re-deriving the same answer thousands of times.
Moving it off the main thread, and the bug that came with it
The first fix is the boring correct one: put the pipeline in a Web Worker. The UI thread hands over the pixels, the worker grinds, the UI stays alive and can render progress.
What nobody warns you about is that the handoff itself has a cost. postMessage structured-clones its payload by default, and structured-cloning a multi-megabyte pixel buffer means copying a multi-megabyte pixel buffer, twice per conversion — once in, once out. You can move the buffer instead of copying it by passing it in the transfer list:
worker.postMessage({ buf, width, height }, [buf]);
That hands ownership across the boundary with no copy. It also neuters the original — after the transfer, buf.byteLength on the sending side is 0.
Which is how I spent an evening on a bug where the source thumbnail rendered as a black rectangle every time, but only after the first conversion. I was holding a reference to the same buffer for the preview, and the transfer had emptied it out from under me. The fix is unglamorous: keep a copy for anything the main thread still needs, or don't transfer that particular buffer. But it's the kind of bug that reads as "canvas is broken" for an hour before it reads as "you gave your array away."
The cache that made the inner loop mostly disappear
The second fix is where the actual speedup lived, and it comes from a property of photographs rather than a property of JavaScript.
Photos are enormously repetitive at the color level. A sky is thousands of pixels of nearly the same blue. A cheek is hundreds of pixels of nearly the same tone. If you're doing a full palette scan for every one of those pixels independently, you are computing the identical answer over and over.
So quantize the lookup key. Take the source color down to 5 bits per channel and use that as a cache key. Five bits per channel means 32×32×32 = 32,768 possible keys — small enough to keep in a flat typed array, and a real photo touches only a fraction of them. First time you see a bucket, do the full palette scan. Every time after, it's an array index.
The palette scan doesn't get faster. It just stops running for the overwhelming majority of pixels. That's the whole trick, and it's the difference between a frozen tab and a progress bar that finishes while you're still looking at it.
One consequence worth flagging if you build something similar: the cache key has to include the palette identity, not just the color. My block palette is versioned — Java releases from 1.13 up through 26.2, with Bedrock carrying its own separate block and texture set — because a palette frozen at whatever version the tool was written against will eventually disagree with what the game actually renders, and you get a preview that doesn't match your build. That's a correctness win, but it means "nearest block to this color" is not a stable function. Switch edition or version and every cached answer is potentially wrong. Ask me how I know. Version identity goes in the cache key, or the cache gets cleared on switch; either works, silently doing neither does not.
What the worker sends back
The last thing I changed was the return type, and it turned out to matter more than the performance work.
Originally the worker returned an ImageData — the rendered preview, RGBA. That's all you need to show someone what their build will look like. But it's the wrong artifact to hand a person who then has to actually place the blocks, because "a picture of the finished thing" throws away the one piece of information they need most: which block is at each position.
So the worker now returns two things: the preview pixels and a flat array of palette indices, one per cell. Once those indices are the source of truth, the things a builder actually needs stop being separate features and start being views over one array. A material list is a single tally pass with a counts array, which is where "you'll need 340 of this, 118 of that" comes from — the number you want before you go mining, not after. Slicing the same array into sections gives you a chunk-by-chunk build guide, so you can work one region at a time instead of squinting at a flat reference image and losing track of which row you're on. And .schem and .litematic export are just serializers over those indices, for anyone running WorldEdit or Litematica who'd rather skip manual placement altogether.
None of that needed its own pipeline. Returning an RGBA preview and nothing else had been quietly throwing away the only representation any of it could be built from.
The side effect I didn't plan for
There's an accidental property of this architecture that turned out to be one of my favorite things about it. The worker has no network code in it. There's no upload step, because there's nothing on a server to upload to — the whole conversion runs locally in the browser.
That means the privacy claim isn't something you have to trust me on. Open DevTools, watch the Network panel, drop in a photo, convert. Nothing goes out. It's a verifiable property of the page rather than a promise in a policy document, and I only ended up there because moving the work into a worker was the fastest way to stop the tab from freezing.
The whole thing lives at minecraftpixelart.xyz if you want to poke at it — it's a solo side project I maintain in my spare time, and the palette upkeep as new Minecraft versions ship is the part I expect to be doing forever.
If you take one thing from this: profile before you optimize, because I would have happily spent a weekend hand-optimizing a cube root that accounted for almost none of the runtime.
