Corrupt inputs are product behavior

A parser has two jobs, and only one of them is parsing. The other is deciding what to do with input that is wrong — and that decision is a product decision, whether or not anyone made it deliberately.

The input is not a contract

When I wrote NetSift, the goal was a dependency-free explorer for PCAP and PCAPNG files that produces answers a human can check. The first version handled well-formed captures, which is exactly the input you get from your own machine and almost never the input you get from an incident.

Real captures are broken in specific, repeatable ways:

• a snap length that truncates every frame at 96 bytes, so IP total-length fields describe bytes that were never recorded
• a pcapng block whose declared length disagrees with its trailing length field
• an interface description block declaring a link type the decoder does not implement
• a file that ends mid-packet because the capture process was killed
• a TCP header with a data offset below the 20-byte minimum, which means the "payload" starts inside the header
• checksums that disagree with the bytes, either because of offload or because someone edited the file

None of these are exotic. Every one of them will appear in a capture someone asks you to look at.

"Explicit data or precise failure"

The rule I settled on for NetSift: malformed input is either reported as explicit data or rejected with a precise reason. Never a plausible-looking guess.

That translates into three concrete behaviors:

Anomalies are computed, not heuristic. Every finding names the field it came from. "IPv4 total length is 512 but only 60 bytes were captured" is checkable by hand. "Suspicious packet" is not. If I cannot point at the bytes, I do not get to flag it.

A defect degrades the local answer, not the whole file. A pcapng block with a bad length field should not make the previous four packets unreadable. The decoder reports the bad block and continues where it can find the next block boundary.

Unsupported is a state, not an error. A link type with no decoder shows its bytes and says so. Hiding it would be worse than admitting the gap.

The reason to be strict here is that a parser's output is usually evidence. A wrong answer that looks confident is more expensive than a refusal, because somebody downstream will act on it.

Fuzz the parser, but also fuzz the story

Random mutation finds crashes. It rarely finds the interesting failure, which is a parse that succeeds and is wrong — a length field trusted where the capture was truncated, a checksum compared against the wrong region, a DNS name compression pointer that resolves into a loop and yields a plausible-looking name.

Those need different tests: hand-built inputs where you know the answer. The malformed capture in the interactive lab is exactly that — eight defects with known identities, assembled byte by byte, asserted in tests. If a decoder change makes one of them disappear, the test fails.

The pattern generalises: for each defect class, keep one fixture whose expected output is written down. Fuzzers keep you honest about the input space; fixtures keep you honest about the semantics.

The same reasoning in a streaming pipeline

PixelChangeCheck sends only changed screen regions over QUIC, with relay-backed NAT traversal and a browser viewer alongside the native one. Different domain, same class of problem: the transport will lie to you, and the product has to decide what that means.

A dropped link mid-frame is the interesting case. The tempting behavior is to keep the last decoded frame on screen and hope. That is a parser returning a plausible answer for truncated input — it looks fine and is stale. The honest behaviors are:

• Track what was actually received. A frame is complete or it is not. The receiver should never render a partial region as if it were current.
• Make retry idempotent. Re-requesting a region has to be safe to repeat, because it will be repeated. The unit of work needs an identity.
• Bound the staleness. If the link has been down past a threshold, say so in the UI. A frozen image with no indicator is worse than an error, because the person watching it cannot tell the difference between "nothing changed" and "nothing arrived".

The common thread is that "the connection is fine" is an assumption the receiver is not entitled to make. Neither is "the capture is well-formed". Both are hypotheses to be checked, cheaply, on every input.

Design rules I keep coming back to

1. Decide the failure mode before writing the happy path. Fail-closed for anything security-relevant, degrade-locally for anything analytical, and say which one you chose in a comment.
2. Never invent a value. Missing is a state. Zero, empty string, and last-known are three different claims.
3. Make the anomaly list part of the output. If your tool can tell that something is wrong, the person using it needs to know without asking.
4. Keep one fixture per defect class. Written expectations beat generated expectations when the point is semantics.
5. Budget for the ugly input. The malformed path is not an edge case; it is a supported mode of operation with its own tests.

What this buys you

A parser that treats corrupt input as a first-class case is not just more robust. It is more useful, because the moment you most need the tool is the moment the input is least cooperative — an incident capture, a crash dump, a stream that dropped. In that moment, "here is what I can prove from these bytes, and here is what I cannot" is the only answer worth having.