Computer vision and RF sensor fusion in the field

Wildfire detection is an interesting systems problem because the hard part is not the detector. It is the decision to wake someone up — and that decision has to be made from partial, asynchronous, occasionally wrong evidence arriving over links that fail.

I co-founded Phoenix Fire Labs and work on Prometheus, which pairs computer vision, drones, and RF sensing for early wildfire detection. This is the architecture as I would describe it publicly: the shape of the pipeline and the constraints that shaped it, not the internals of the deployed system.

Three sensors that disagree usefully

The reason to fuse modalities is not redundancy. It is that they fail differently.

• Vision sees smoke and flame, and fails on darkness, occlusion, and anything that looks like smoke — fog, dust, a truck's exhaust.
• RF sensing sees radio-frequency activity, and fails on distance, terrain, and interference.
• Drones give you a second look at a specific coordinate, and fail on endurance, wind, and the cost of being wrong about where to send them.

A single sensor with a high false-positive rate is unusable: every alert costs a human's attention, and attention is the scarce resource. Fusion is what buys you a detector whose alerts are worth acting on.

The fusion rule that matters is agreement, not averaging. If vision reports a smoke signature at a bearing and RF reports anomalous activity in the same sector within the same window, that is two independent observations of one event. If they disagree, you have two weak hypotheses, and the right move is usually to task a drone at the coordinate with the better revisit economics rather than to blend the scores into one confident-looking number. Averaging hides disagreement; the disagreement is the information.

The latency budget is the design

An alerting pipeline should be designed from the deadline backwards:

detect         sensor frame or RF sample in hand
correlate      same sector, within the fusion window
confirm        optional second look (drone, tower, human)
notify         reach a person with coordinates and confidence

Every stage consumes the budget, and the stages are not equally compressible. Correlation is cheap. Confirmation is not: a drone takes minutes to reach a coordinate, so the decision to task one has to be made on partial evidence. That is a design choice with a cost on both sides — task too eagerly and you burn flight time on exhaust plumes; task too late and the alert arrives after the fire has grown.

The useful discipline is to write the budget down as numbers and hold the pipeline to it, including the failure paths. "Alert in under N seconds from first detection, or escalate the uncertainty to the human" is a testable property. "Detect wildfires early" is not.

False positives and false negatives do not cost the same

This is the single most important asymmetry in the system, and it is why thresholds should never be a constant in a detector.

• A false positive costs a human's attention and, if it repeats, the credibility of the whole system. The second failure is worse than the first: an operator who has been burned twice stops reading alerts.
• A false negative costs the thing the system exists to prevent.

Those costs are not comparable, and they are not constant either. They change with terrain, season, vegetation, and how many other fires are already active. So the thresholds belong in configuration, versioned, with the value that produced a given alert recorded alongside it.

A detector that hardcodes its threshold is a detector you cannot tune when conditions change, and you will only find out it needed tuning from a postmortem.

Degraded modes are the normal case

Field systems spend most of their time in some degraded state: one sensor down, the uplink slow, the drone grounded, the model running on a smaller compute budget than it was trained for. The pipeline should have a declared answer for each:

• One modality missing. Keep detecting on the remaining evidence, and mark the alert as single-source so the human knows the confidence is different.
• Uplink loss. Buffer and forward; never drop detections silently. An alert that arrives four minutes late is still useful; an alert that never arrives is not.
• Compute contention. Fall back to the cheaper model and say so in the alert metadata, rather than silently degrading detection quality.

The property to protect is that the system's stated confidence always matches the evidence it actually had. Everything downstream — human judgment, drone tasking, escalation — depends on that being true.

What carries over to any sensing product

1. Design from the deadline backwards, and write the numbers down.
2. Fuse for disagreement, not for a smoother score. Independent sensors that agree are evidence; blended scores hide the case you care about.
3. Put thresholds in configuration, with provenance. Every alert should be able to name the parameters that produced it.
4. Declare the degraded modes. A system that only works in the ideal case has never worked.
5. Protect the alert channel's credibility. Attention is the resource; spend it carefully.

None of that is specific to wildfire detection. It is what any system needs when the answer matters more than the demo — which is the part of this work I find most interesting.