NULL Should Never Mean Favorable
The Problem
When you screen millions of materials for commercial viability, missing data is the norm, not the exception. The question is: what does your pipeline do when a field is NULL?
A comprehensive data-stitching audit of our warehouse and screening pipeline revealed a systemic pattern: every screen defaulted NULL to the favorable verdict. Missing data didn't mean "unknown" — it meant "good."
The Specific Defects
| Screen | NULL Interpretation | What It Should Mean |
|---|---|---|
| Phonon stability | Missing phonon data = stable (weight 1.5, can reach S-tier) | No evidence of stability — flag as "no-data" |
| Thermal conductivity | measured_kappa IS NULL passes the κ gate | Conductivity unknown — cannot pass |
| PFAS screening | pfas_clean is None → no penalty | PFAS status unknown — flag for review |
| Patent whitespace | patent_count == 0 (from unreliable name-keyed KG lookup) → full whitespace score 1.0 | Zero patents found by a broken query — not the same as "no patents exist" |
| Supply chain scoring | Scorer exception → neutral 0.5 supply score | Scoring failed — report as error, not neutral |
| Novelty scoring | Formula missing from coverage → "novel" | Not in our data — that's a data gap, not novelty |
How It Propagated
The NULL-favorable pattern compounds through the pipeline:
- A compound with no phonon data is scored as "stable" (weight 1.5)
- With no patent hits (because the patent matcher was broken), it scores as "novel"
- With no PFAS flag, it's "clean"
- The compound climbs to the top of the rankings — entirely on the strength of missing data
This is how candidates get promoted to Tier 1 based on the absence of negative evidence rather than the presence of positive evidence. The pipeline rewards ignorance.
The Additional Data Quality Issues
The audit uncovered compounding defects in the data layer itself:
- ~50% of
formula_reducedvalues are not formulas — 1.17M numeric IDs, 498kcomp_<hash>tokens, 445kEl_Elstubs, and filesystem path fragments. No consumer filters onparse_ok. - The patent-evidence chain is defective at every link.
dim_patent.titleis actuallyMAX(claim_text). The patent-composition linker regexes ANY capitalized token pair without an element-symbol whitelist — linking patent 8248032 ("notebook computer charger") to NaS3Ti. - Cross-source identity resolution is inconsistent. The same physical material gets different
composition_idvalues across sources, while distinct polymorphs sharing a formula get merged.
The Fix: Three-Way Verdicts
Every screen should emit one of three verdicts:
- PASS — positive evidence meets the threshold
- FAIL — evidence contradicts the criterion
- NO_DATA — insufficient evidence to judge
NO_DATA is not PASS. NO_DATA is not even neutral. NO_DATA means the compound cannot be ranked on this axis, and any ranking that treats it as favorable is lying about its confidence.
We implemented this by adding an evidence_level column to every screening verdict and refusing to promote candidates with more than two NO_DATA verdicts across critical axes.