An empty result is not an all clear

An infrastructure audit runs across your account and comes back with two findings on S3. You fix both, close the ticket, and move on.

What the report did not tell you is that the role running it lacks s3:GetEncryptionConfiguration. Fourteen buckets returned AccessDenied on that call. The tool caught the rejection, wrote encrypted: false, and moved on to the next bucket. Some of those fourteen are encrypted. Some are not. The report cannot tell you which, because the value it printed was not read from AWS — it was the default that got assigned when the read failed.

This is the bug I spent a release fixing across Infrawise, and it is not really an S3 bug. It is a shape-of-data bug that almost every infrastructure scanner has somewhere: a boolean field with two states being asked to carry three.

false is a claim, and a failed call has no claim to make

The S3 extractor issues four calls per bucket — notifications, versioning, encryption, public access block — through Promise.allSettled, so one rejection never takes down the other three:

const [notifResult, versionResult, encryptResult, pabResult] =
  await Promise.allSettled([
    client.send(new GetBucketNotificationConfigurationCommand({ Bucket: name })),
    client.send(new GetBucketVersioningCommand({ Bucket: name })),
    client.send(new GetBucketEncryptionCommand({ Bucket: name })),
    client.send(new GetPublicAccessBlockCommand({ Bucket: name })),
  ]);

allSettled is the right primitive. The mistake was what happened next. Reading a rejected result as false collapses two completely different situations into one byte:

• We read the bucket and versioning is off. An observation. Worth a finding.
• The call was denied, throttled, or timed out. Not an observation. Worth an error message and nothing else.

Downstream, nothing can separate them again. The analyzer receives versioned: false and emits a medium-severity finding recommending you turn versioning on for a bucket whose versioning state was never read. The finding looks exactly like a real one. It has a bucket name, a severity, a recommendation. It is indistinguishable from evidence, and it is not evidence.

The fix is that the field admits a third state:

const versioned =
  versionResult.status === 'fulfilled'
    ? versionResult.value.Status === 'Enabled'
    : null;

And every analyzer that consumes it tests for the observation, not for falsiness:

if (node.versioned === false) { /* finding */ }

null falls through. No finding, because there is nothing to find — only something unread.

The two exceptions are the interesting part

Not every rejection is an absence of information. GetBucketEncryption answers "this bucket has no encryption configuration" by throwing ServerSideEncryptionConfigurationNotFoundError rather than returning an empty body. GetPublicAccessBlock does the same with NoSuchPublicAccessBlockConfiguration. For those two specific error names, the rejection is the fact:

const encrypted =
  encryptResult.status === 'fulfilled'
    ? (encryptResult.value.ServerSideEncryptionConfiguration?.Rules?.length ?? 0) > 0
    : errorName(encryptResult.reason) === 'ServerSideEncryptionConfigurationNotFoundError'
      ? false
      : null;

This is the part you cannot skip by writing a generic "catch everything, return null" wrapper. Two of the four calls encode a real answer in an exception. Collapse all rejections to null and you stop reporting genuinely unencrypted buckets, which is a false negative traded for the false positive you just removed. The error name has to be inspected. There is no way around reading the API's documented behavior call by call.

Four ways absence gets promoted to fact

Once you start looking for this pattern, it shows up everywhere a scan touches an API it does not fully control:

A capped listing. ListBuckets paginates. The old code took the first page and stopped, so accounts past 200 buckets got a partial inventory rendered as a complete one. Nothing downstream can detect a missing bucket — an absent entry reads as "does not exist" to every consumer.

A synthesized node. When code says QueueUrl: process.env.QUEUE_URL, there is no queue name to resolve. A graph node still gets created so the function's edges have somewhere to point, and it was created with hasDLQ: false. The DLQ analyzer read that default as an observation and produced a high-severity finding: a queue literally named unknown had no dead-letter queue. Those nodes now carry placeholder: true, and every analyzer whose evidence is absent configuration skips them:

if (node.type !== 'queue' || node.placeholder) continue;

A failed service. One extractor throwing should not kill an analysis, so each one is wrapped and its outcome recorded rather than only logged. A warning printed to a terminal nobody read is not a signal. The status is one of four values — ok, failed, partial, disabled — and it rides along with the results.

A partial extraction. Sometimes an extractor gets most of what it needed and loses one piece. Throwing away the whole service costs more than it protects; keeping it silently is the exact false negative all of this exists to prevent. PartialExtractionError carries the usable data and the gap, and the source is marked partial.

Why this got urgent

An infrastructure report with a phantom finding wastes an engineer twenty minutes. They open the console, see versioning is already on, shrug, and move on. Annoying, self-correcting.

That was before the reader stopped being human.

Infrawise serves this data to AI coding assistants over MCP, and a model does not shrug. Ask it "which of my buckets are unencrypted" and it gets a JSON array. If the array is empty because the S3 read failed, the model does not infer a failed read. It answers "all your buckets are encrypted" in exactly the tone it uses for things it verified, and then it writes your bucket policy on that assumption. Empty is the most dangerous possible response, because empty is what "everything is fine" also looks like.

So every tool response carries a dataHealth block, and the part that matters here is the source list:

"sources": [{ "service": "s3", "status": "failed", "error": "AccessDenied" }]

Every key is always present. error is null rather than omitted, so nothing has to be inferred from a missing field. get_graph_summary goes further and stamps each node with its own source and sourceStatus, so a node from a degraded source is distinguishable from a clean one without cross-referencing anything.

The rule this gives an assistant is a single sentence, stated in the description of the first tool it is told to call: a source that is not ok means an empty result is "not read", not "none exist". get_table_schema is the sharpest case — with a database listed as failed, found: false means "not looked for", not "no such table". That is the difference between an assistant writing a query against the wrong schema and an assistant telling you to re-run infrawise analyze.

The general shape

If you build anything that reports on infrastructure you do not own — a scanner, a linter, a dashboard, an MCP server — the check is quick. Take any boolean in your output and ask what it holds when the call that fills it fails. If the answer is false, you are shipping a claim you never verified. If the answer is "the whole field is missing", you are asking every consumer downstream to guess, and at least one of them will guess wrong in the direction of "fine".

Three states, always. Observed true, observed false, not observed. The third one costs a nullable type and a few === false comparisons, and it is the entire difference between a report that is trustworthy and a report that is merely quiet.

Infrawise is MIT-licensed and runs locally against your own account: GitHub · npm.

Key takeaways

• A boolean that defaults to false on a failed read is asserting something nobody verified. Make the field nullable and have consumers test === false, not falsiness.
• Do not wrap every rejection into null blindly. Some APIs answer "not configured" with an exception — GetBucketEncryption and GetPublicAccessBlock both do — and for those, the error name is the fact.
• Unpaginated list calls silently shrink your inventory, and a missing item reads as a non-existent item to everything downstream.
• Objects synthesized to hold a reference together (from an env var, an ARN, a code path) must be marked as such, or their default values get audited as if they were read from the provider.
• If an LLM consumes your output, ship per-source status alongside the data. An empty array with no health signal will be reported to the user as an all clear.