The Hard Part of an OpenTelemetry Logs Bridge Wasn't Encoding

Part 2 of Engineering HaloLog: making high-performance telemetry claims reproducible and falsifiable. Part 1 examined what a 23.9 ns logging benchmark does and does not prove.

The implementation and measurements discussed here are pinned to otelbridge/v1.0.3.

An OpenTelemetry Logs bridge looks like a mapping problem.

Message becomes Body. Level maps to Severity. Ordinary fields become Attributes; valid correlation fields can become trace context. Then the bridge calls Emit.

That part was straightforward.

The difficult part was preserving the contracts that live between the APIs: the context used to ask whether a record is enabled, ownership after a pooled entry is returned, masking precedence when a field has more than one storage form, and the lifecycle of a provider the bridge does not own.

Those are not details around the bridge. They determine whether it emits the right record, leaks a value that was supposed to be masked, retains memory it does not own, or loses the final log before a process exits.

This article is about those boundaries.

Measurement boundary: each timed operation calls Adapter.Write directly on a LogEntry built before the timer and ends at an OpenTelemetry API Logger whose Emit method is a no-op. The figures exclude HaloLog's public logger and builder path, masking, console/file fan-out, the OpenTelemetry SDK, processors, exporters, collector, network, and backend. They are adapter microbenchmarks, not production throughput or a comparison with other bridges.

The request was larger than trace IDs

HaloLog already had an optional Bind helper that attached trace_id, span_id, and trace_flags to request-scoped loggers. That made correlation fields visible in JSON output, but it did not send logs through the OpenTelemetry Logs pipeline.

The distinction matters. A user responding to the first HaloLog article put it plainly: they wanted logs sent to OpenTelemetry, not merely trace information printed beside a message.

Soon after, Daniel Loader opened HaloLog's first external pull request. It added an output adapter that maps a HaloLog LogEntry into an OpenTelemetry LogRecord. The mapping includes the timestamp, severity, body, structured fields and context, component/error/caller metadata, and optional reconstructed trace context. Daniel Loader's five authored commits remain in the merge history. Maintainer review then hardened the architecture and its tests.

The resulting path is:

HaloLog LogEntry
├── console/file adapter ──> formatted bytes
└── otelbridge.Adapter ────> OpenTelemetry LogRecord
                              └── API Logger
                                  └── SDK / processor / exporter
                                      └── collector / backend

otelbridge has its own go.mod. Programs that do not import it do not pull OpenTelemetry into HaloLog's core dependency graph. Programs that do can register the bridge beside an existing console or file adapter and let the normal fan-out send the same logical entry to both destinations.

Fan-out is sequential and best-effort. It does not make delivery atomic across sinks, propagate per-adapter write errors to the logging caller, or make the sinks' wire representations identical.

logger := core.New().
    Adapters(
        console.New(),
        otelbridge.NewAdapter(
            "github.com/acme/checkout",
            otelbridge.WithLoggerProvider(provider),
        ),
    ).
    MustBuild()

log := otelbridge.Bind(r.Context(), logger)
log.Typed().WithInt("status", 200).Info("handled")

The OpenTelemetry Logs API exists specifically so logging libraries can build this kind of bridge. Its data model distinguishes Timestamp, ObservedTimestamp, trace identifiers and flags, severity, Body, Resource, InstrumentationScope, Attributes, and EventName. HaloLog maps the fields it owns; the selected provider and SDK supply observed time, resource, and scope handling. Mapping fields is necessary. It is not sufficient.

Contract 1: Enabled and Emit must use the same correlation context

OpenTelemetry exposes Logger.Enabled so a bridge can skip expensive record construction when the pipeline does not want a record. The method accepts the context that will be associated with the record.

That last sentence is the contract.

It is tempting to probe cheaply with context.Background(), build the record, reconstruct its span context, and then call Emit with the richer context. But a processor may filter using the sampled flag or another value carried by that context. The bridge would then test admission under one context and emit under another.

The correct order in HaloLog is:

ctx, consumed, fieldCount := a.spanContext(entry)

if !a.logger.Enabled(ctx, otellog.EnabledParameters{Severity: severity}) {
    return ctx, record, false
}

// Build attributes only after Enabled accepts this exact context.
// ...
a.logger.Emit(ctx, record)

This choice has a measurable consequence. A disabled uncorrelated record can avoid all attribute conversion. A disabled correlated record still pays the cost of reconstructing its span context, because correctness requires that context before Enabled.

I would rather expose that cost than optimize the bridge into asking the wrong question.

The release contains both a recorder-level consistency test and a custom context-sensitive processor exercised through the real SDK. The frozen release validation document records that changing the probe back to context.Background() made the sampled SDK case fail.

Contract 2: masking must remain authoritative at the export boundary

HaloLog fields can arrive through typed, legacy, static, bound-context, indexed, or descriptor-backed storage. That flexibility becomes dangerous if a transform rewrites one representation while an adapter reads another.

During review, the important question was not “does the JSON formatter show the mask?” It was “can any export path still observe the original value?”

The core v1.0.2 repair established one post-mask field contract across those representations. The bridge consumes that published version directly, without a local module replacement or a test skip.

Correlation added another subtle case. Consider two occurrences of trace_id:

trace_id = 4bf92f3577b34da6a3ce929d0e0e4736
trace_id = REDACTED

A parser that remembers the last valid value restores the first identifier after the second one was deliberately masked. The record is syntactically valid, yet an identity the policy removed has returned as top-level trace context.

The bridge visits static fields, dynamic fields, static context, dynamic context, and finally indexed storage. In that order, the last occurrence of each correlation key is authoritative, even when masked, malformed, or wrongly typed. The bridge clears previously parsed state before examining that value. If it cannot become trace context, it remains an ordinary attribute so the malformed or redacted input is still observable; the earlier identifier does not come back.

Two 18-case matrices cover all three correlation keys in dynamic-field and context storage: one checks the SDK export boundary, and one inspects the API Record before SDK deduplication can hide a stale duplicate. The frozen release validation document also records that selecting core v1.0.1 failed the three named consumer-contract tests.

Contract 3: emitted bytes must outlive the pooled entry

HaloLog reuses log entries. The Go Logs API permits a Logger implementation to retain a Record after Emit, and log.BytesValue requires callers not to mutate the supplied slice after passing it. HaloLog may return its entry to a pool as soon as logging returns. The caller may also reuse the byte buffer at that point.

Those ownership models are incompatible unless the bridge takes ownership of every value that can alias caller memory.

The obvious case is []byte. Constructing an OpenTelemetry byte value directly from the entry's slice would let a queued record observe whatever bytes the caller or pool writes next. The fastest implementation would also be wrong.

The bridge therefore copies every emitted byte-valued attribute before Emit; this also makes the value safe for batching. For a 4 KiB payload, that means one allocation and exactly 4,096 copied bytes in the measured workload.

That allocation is a guarantee, not a regression.

The optimization belongs one step earlier: call Enabled before conversion. When the pipeline rejects the same 4 KiB record, the bridge performs no copy and the measured path remains 0 B/op and 0 allocs/op.

The retained-record test does not merely compare the record immediately after the call. It stores emitted records, mutates and reuses the source entry and its buffers, then verifies record independence across 1, 5, 6, 16, 17, and 64 attributes. The frozen release validation document records that replacing the ownership copy with aliasing made all six cases fail.

This is the boundary where “zero allocation” stops being a useful absolute. Unbounded retained storage must live somewhere. The engineering question is whether an allocation is accidental, bounded, avoidable, or required by an ownership contract.

Contract 4: drain the emitting provider when it exposes an identifiable target

Lifecycle looked simple until Fatal entered the model.

HaloLog's fatal path writes the record, calls Logger.Flush, and exits from inside the logging call. If an OpenTelemetry batch processor still owns the record, the process may terminate before the most important line it produced leaves the queue.

WithFlushFunc takes precedence. Without a configured callback, the bridge calls ForceFlush when the captured provider implements it. Close requests the same drain but deliberately does not call Shutdown: the provider belongs to the application and may be shared with other instrumentation. Closing one logging adapter must not tear down the application's telemetry pipeline.

There is an awkward global-provider case. OpenTelemetry's global proxy can delegate emission to a provider installed after the adapter is constructed, but the proxy does not identify that provider as a ForceFlush target. Looking up “the current global” later is not equivalent either: the global provider may have been replaced since the logger was created.

When the adapter captured the late-binding global proxy and no callback is supplied, Flush returns ErrFlushUnavailable rather than claiming to drain an unidentified target. Applications using late global initialization can pass a flush callback paired with the provider that receives their records:

otelbridge.NewAdapter(
    "github.com/acme/checkout",
    otelbridge.WithFlushFunc(provider.ForceFlush),
)

The default flush deadline is five seconds. It is cooperative, not preemptive: a provider that ignores context cancellation can still block. A non-positive configured timeout is intentionally unbounded. Creating an unbounded goroutine per flush would hide that limitation by introducing a new lifecycle problem, so the bridge documents the boundary instead.

There is one more distinction worth stating. otel/log.LoggerProvider does not require ForceFlush. In v1.0.3, an explicitly supplied provider that exposes neither ForceFlush nor a configured callback makes Flush and Close return nil without draining. Close is flush-only: it does not call Shutdown, mark the adapter closed, or prevent later writes. Fatal attempts Logger.Flush but ignores its error so termination still proceeds. Applications that require a callable drain path must supply a force-flushing provider or a correctly paired WithFlushFunc; the bridge cannot manufacture one from the generic provider interface. Code that must observe drain failure should call Flush during graceful shutdown and handle its error, because Fatal cannot return that error.

Where the allocations actually are

After correctness came the narrower performance problem: avoid temporary work without weakening ownership or changing record semantics.

The bridge walks HaloLog's static fields, dynamic fields, context, and indexed storage directly instead of first building merged slices. It computes an upper bound from the existing traversal and calls Record.AddAttributes once. In go.opentelemetry.io/otel/log v0.17.0, an API Record stores five attributes inline. A sixth scalar attribute makes the record allocate overflow storage.

The first bridge release introduced a second boundary of our own. It staged up to 16 attributes on the stack, then allocated a temporary slice at 17. The OTel record copied that slice into storage it owned, so the temporary allocation did not buy us an ownership guarantee. This was an avoidable cliff in our adapter.

We noticed it while reviewing the original release's performance evidence. The timing summary did not retain its raw benchmark output, so I would not use a later replay to assign a cause to a changed nanosecond figure. Reading the code and sweeping the field count did establish the structural problem: at exactly 17 fields, the adapter added one allocation and 704 bytes of temporary storage.

In v1.0.3, the common path still stages at most 16 values on the stack. A separate, non-inlined helper stages 17 to 32 values in a bounded stack array and lets AddAttributes make its owned copy before returning. Keeping the helper out of line prevents its 1,280-byte scratch array from enlarging every small record's stack frame. Above 32 values the adapter still uses heap staging. Moving an unbounded array onto the stack would merely move the resource problem.

The current adapter-only allocation shapes are:

The bytes above were observed in five 200 ms runs of the published module with Go 1.27.1 on Windows/amd64 and an Intel Core Ultra 9 285HX. The committed tests enforce the allocation ceilings at 5, 6, 9, 16, 17, 32, and 33 fields. The release-pinned raw replay includes the disabled-byte case. The ns/op results varied substantially in some short runs, so this article uses that replay for its allocation table, not for a before/after latency claim.

The benchmark output's -24 suffix is the default GOMAXPROCS setting of that invocation, not 24 concurrent emitters; these cases are non-parallel. The input LogEntry was built before timing, and the API logger's Emit was a no-op. Arbitrary values requiring fmt.Sprint, user callbacks, SDK processors, exporters, and downstream systems have their own costs.

Tests that have to fail when the implementation is wrong

A green test suite is weak evidence if it also stays green after the guarantee is removed.

The frozen release validation document records four negative-control mutations:

1. Replace the byte ownership copy with aliasing: all six retained-record width cases fail.
2. Select HaloLog core v1.0.1: all three masking consumer contracts fail.
3. Call Enabled with context.Background(): the context-sensitive sampled SDK case fails.
4. Resolve fields eagerly: the callback-count contracts fail for both enabled and disabled records.

The original v0.1.0 validation record lists published-core runs on Windows/amd64 with Go 1.24.0 and 1.27.1, plus published- and local-core runs on Linux/amd64 with Go 1.27.1. Its mutation checks remain historical evidence for the contracts that survived in v1.0.3. The PR #3 CI run passed the bridge module on Linux against both local and published core with Go 1.24.0 and stable, and ran root-module builds and tests on Linux and Windows. The Windows root-module jobs are not Windows bridge tests. Local Windows bridge tests passed with Go 1.24.0 and 1.27.1; the local race run could not start without a C compiler. The release's Linux bridge CI supplies race-instrumented coverage for the executed tests.

The verification script also rejects module replacements in published mode, requires the consumer-contract tests by name, runs go vet, rejects runtime skips, and verifies that go.mod and go.sum remain unchanged.

That last detail matters. A bridge tested against a repaired local checkout can still ship a broken versioned dependency to users.

Reproduce the boundary

The released bridge is a separate Go module. As checked on 2026-09-22, the public Go module proxy resolved @latest to v1.0.3, whose go.mod requires core v1.0.2. The earlier v0.1.0 tag did not become @latest because older v1.0.2 module tags sort above it; checking the resolver exposed that release mistake. Consumers should request the new version explicitly:

go get github.com/go-gen-ecosystem/halolog/otelbridge@v1.0.3

Check out the commit behind otelbridge/v1.0.3 before running these commands. They replay correctness on Linux or WSL; their timings are not expected to reproduce the Windows measurements.

git clone https://github.com/Go-Gen-Ecosystem/halolog.git
cd halolog
git checkout --detach otelbridge/v1.0.3

bash scripts/verify-otelbridge.sh published
bash scripts/verify-otelbridge.sh local

cd otelbridge
GOWORK=off go test -count=1 ./...
GOWORK=off go test -race -skip PipelineScale -count=1 ./...
GOWORK=off go test -run '^$' -bench . -benchmem -benchtime=1s -count=5

The current implementation and allocation guards are frozen at otelbridge/v1.0.3, commit 864b6ff. The original bridge and its historical validation record remain available at otelbridge/v0.1.0.

What would falsify the claims here?

Any of the following is a useful counterexample:

• a context-sensitive processor for which Enabled and Emit observe different trace context;
• a masked or malformed final correlation field that restores an earlier valid identifier;
• a retained record whose byte value changes after the source entry or buffer is reused;
• an intercepted Fatal path that exits without invoking its configured drain;
• a released module graph that uses a replacement or selects a core older than v1.0.2;
• a 17- or 32-field adapter-only scalar workload exceeding one allocation;
• a post-mask field value that reaches the OpenTelemetry API from a stale storage representation after the same source field was masked for console/file output.

Report the Go version, OS/architecture, module graph, command, and smallest reproducer. If the counterexample is valid, the contract or the documentation has to change.

What this does not prove

The release did not test a remote OTLP collector, TLS failures, network partitions, backend acknowledgement semantics, prolonged soak behavior, ARM64, or every custom processor. Its backpressure test checks deadline reporting and recovery; it does not promise lossless delivery from a bounded SDK queue.

The timeout is cooperative. Race-detector coverage instruments the executed tests; it does not prove the absence of every race. Allocation ceilings for the adapter do not apply to the SDK or exporter. The benchmark does not establish that this is the fastest OpenTelemetry bridge.

The release pins go.opentelemetry.io/otel/log and sdk/log v0.17.0. Those Go modules are pre-v1 and their APIs can still evolve, even though the OpenTelemetry Logs data model and Logs API specification define the external semantics the bridge follows.

The adapter was the small part

Creating a LogRecord was never the interesting problem.

The interesting problem was preserving the correlation context used for filtering and emission, carrying the core's post-mask value into each output, owning retained bytes after pooling, and pairing a drain operation with the provider that received the data whenever that provider exposes one.

Some of those guarantees cost time. In the measured single-[]byte case, ownership deliberately costs one allocation. Others make a cheap early exit possible. None of them can be recovered from a benchmark after the semantics are wrong.

That is the principle I am taking from this bridge: optimize the work that the contract does not require. Make the work it does require visible, bounded, and executable as a test.

Sources

• HaloLog repository
• Original bridge validation boundary
• Current bridge implementation
• Correlation security tests
• Retained-record ownership tests
• Lifecycle tests
• Allocation guard tests
• Published-core consumer contract
• Allocation fix and CI review
• Release-pinned allocation and proxy-consumer evidence
• OpenTelemetry Logs API specification v1.49.0
• OpenTelemetry Logs data model v1.49.0
• OpenTelemetry Go Logs API v0.17.0