Part of the build log for PhotonicOps, a fully offline, air-gapped telemetry engine for silicon photonic biosensors. No cloud APIs, no internet dependency, Apple Silicon only. This post covers the worker pool sitting between the gRPC ingestion stream and the DSP handoff.
Short version: 10,000 frames/sec, one channel send on the hot path, 10 fixed workers, zero per-frame allocations via sync.Pool, backpressure instead of drops.
The Constraints
A photonic biosensor streams resonance wavelength shift (Δλ, picometers) at 10,000 samples/sec. That's one OpticalFrame roughly every 100μs over a gRPC client-streaming call. Two rules shaped the design:
- Nothing stalls the receive loop. If the goroutine reading off the stream also does the work, a slow downstream consumer pushes backpressure straight into the TCP connection and the sensor.
- No GC pauses on the hot path. At 10kHz, a few milliseconds of stop-the-world GC can silently drop a sample, and a dropped sample can hide the exact anomaly (a microbubble, a cell clog) the system exists to catch.
The Pool
type FramePool struct {
jobQueue chan *pb.OpticalFrame
wg sync.WaitGroup
forwarder *dsp.Forwarder
}
var frameSyncPool = sync.Pool{
New: func() any { return make([]byte, 0, 1024) },
}
func NewFramePool(workers, queueSize int, forwarder *dsp.Forwarder) *FramePool {
p := &FramePool{
jobQueue: make(chan *pb.OpticalFrame, queueSize),
forwarder: forwarder,
}
for i := 0; i < workers; i++ {
p.wg.Add(1)
go p.worker(i)
}
return p
}
Wired up in main.go as NewFramePool(10, 50000, forwarder): 10 workers, a 50,000-slot channel. At full throughput that's 5 seconds of buffering headroom before anything blocks.
The gRPC handler's entire job:
func (p *FramePool) Enqueue(frame *pb.OpticalFrame) {
p.jobQueue <- frame
}
One channel send. No allocation, no branching. If the channel fills, Enqueue blocks, which becomes backpressure through gRPC flow control back to the sensor. No silent drops. Non-blocking load-shedding is on the roadmap for genuine overload but isn't built yet.
Zero-Allocation Workers
func (p *FramePool) worker(id int) {
defer p.wg.Done()
for frame := range p.jobQueue {
buf := frameSyncPool.Get().([]byte)
if err := p.forwarder.Push(frame); err != nil {
log.Printf("worker %d: dsp forwarder push error: %v", id, err)
}
buf = buf[:0]
frameSyncPool.Put(buf)
}
}
Reusable []byte scratch space, recycled by capacity instead of reallocated per frame. At 10,000 frames/sec, allocating small and letting GC reclaim is a GC storm in slow motion. sync.Pool turns it into pointer recycling.
Where the Frame Actually Goes
Each worker hands its frame to dsp.Forwarder.Push, which:
- Shards frames by
sensor_idinto per-sensor accumulators - Buffers each sensor's stream into 100ms windows (1,000 frames each)
- Flushes full windows as a
FrameBatchover a Unix domain socket, via gRPC, to a local Python DSP process
No TCP, no HTTP, no cloud. The Go to Python DSP boundary stays entirely on-box.
Why Fixed Size, Not Goroutine-per-Frame
go process(frame) per frame would be simpler. Two reasons it doesn't work here:
- Unbounded goroutines under load means unbounded memory. A downstream hiccup (Python DSP pausing) shouldn't turn into an OOM in an offline clinical deployment with no one watching a dashboard.
- A fixed pool bounds lock contention.
dsp.Forwarder.Pushholds a mutex over the per-sensor accumulator map, so capping workers caps contention instead of letting it scale with load.
Worst case is "the send blocks", not "the process dies."
What's Next
Phase 5 (Ingestion Hardening) on the roadmap: per-sensor ring buffers (today's is a single global buffer, not keyed by sensor), non-blocking load-shedding for Enqueue, and a /metrics endpoint so pool saturation is observable instead of inferred from pprof.
For anything that has to sustain a hard real-time ingestion rate, keep the hot path allocation-free and non-blocking, push everything else behind a bounded channel, and let a fixed pool of workers apply steady, predictable pressure downstream.
