I originally put PolyForge together as a ChatGPT skill, just for OpenSCAD, so I could describe a bracket or an enclosure and get back something I could actually print on my modified Ender 3 V2 or my QALAM Pro 400, instead of hand modeling every mount and shelf myself. It worked fine, but it only worked in ChatGPT, and it needed an agent and an API bill in the loop even for a plain shelf bracket. That felt backwards. A shelf doesn't need a language model to exist.
So I pulled it apart and rebuilt it as PolyForge, a standalone Python package. No agent, no LLM required for the core. It runs the same whether you point Claude Code, ChatGPT, Gemini, or a local model at it, because the actual capability lives in the package itself, not in something only one vendor's assistant knows how to run.
What it does today
A small library of parametric part templates (box, wall shelf, corner bracket, cable comb, standoff mount) gets filled in from plain text by keyword matching and regex extraction, then rendered to any of three CAD backends: OpenSCAD, FreeCAD, or Blender. All three produce the same geometry, dimension for dimension, for the same template and parameters, and every template is compile tested against the real binaries, not just unit tests.
This week I added a fourth capability: offline photogrammetry. polyforge reconstruct-from-photos takes a directory of overlapping photos of a real object and turns them into an STL, entirely offline, no cloud API involved.
The COLMAP discovery
The pipeline runs COLMAP for structure from motion. COLMAP also has its own dense reconstruction stage, the part that would normally take a sparse point cloud and turn it into something dense enough to mesh. I assumed it might need a GPU, but I didn't want to assume, so I ran it without one on purpose and watched what happened.
It aborted immediately: "Dense stereo reconstruction requires CUDA, which is not available on your system." No CPU fallback anywhere in the mainline binary. That single test changed the whole architecture. Instead of one tool doing everything, the pipeline now uses COLMAP only for its CPU capable sparse reconstruction, then hands the result to OpenMVS, whose densify and mesh stages build CPU only by default, for the part COLMAP itself can't do without a GPU.
Three patches to build OpenMVS
Getting OpenMVS running on a rolling release system with a newer OpenCV than the package expected took three separate source patches. Its pinned vcpkg commit had a data schema too old for the installed vcpkg binary. Two OpenCV template specializations it defined itself collided with ones OpenCV now provides natively. And one constant, cv::DIST_L1, had been removed from OpenCV's imgproc module entirely, so I inlined the literal value at the one call site that needed it. None of this is really about PolyForge, it's just what building against a system that moves faster than an AUR package's last update looks like in practice.
What actually breaks a reconstruction
Once the pipeline worked, I wanted to know where its real limits were, not the ones I assumed. So I ran an extensive test matrix: different photo counts, different camera coverage patterns, a more geometrically complex part, and timed every stage.
The biggest finding surprised me. I expected photo count to matter most. It doesn't. A camera orbit at a single elevation fails outright no matter how many photos you take, even 24 of them. You need at least three elevation rings. Once you have that, thirty photos across three rings reconstructed cleanly, twenty seven did not.
The second finding was about where the time actually goes. I assumed the matching stage, which compares every pair of photos, would be the bottleneck as photo count grew. It barely moved. The real cost is COLMAP's incremental bundle adjustment: about three seconds at sixteen photos, over two minutes at thirty six, a much steeper jump than the photo count alone would suggest. And it scales with the object's geometric complexity too, not just the count. A bracket with concave, non convex features took six times longer than a plain box at the same photo count, even though both reconstructed correctly in the end.
Where it stands
Four phases in, PolyForge can generate parts from text across three CAD backends and reconstruct a mesh from a folder of photos, all without a cloud API or a GPU. The reconstructed mesh is not perfect: it has no absolute scale until you measure something known on the real object and rescale it, and it usually isn't watertight since a camera orbit can't see a surface's underside. Both of those are honest limitations of photogrammetry itself, not bugs, and I'd rather say so plainly than pretend otherwise.
Repo's public if you want to look at the actual code: github.com/rx290/polyforge
