The Discovery

During a third-pass deep audit of our compute results, we discovered that ALL Modal Quantum ESPRESSO DFPT and HSE06 results from April 14–16, 2026 were error payloads, not usable data. The files did land on the Modal volume and were downloaded locally, but every single one had a failure status. No usable physics was extracted from the entire GPU campaign.

The 13 Failed Results

Root Causes

1. Ir pseudo-potential bundling failure. add_local_file in the Modal image build likely failed silently — the pseudo-potential never actually mounted. The error fires in 0.002 seconds, before pw.x even launches.
2. HSE06 SCF convergence failures. Typical HSE06 tuning issues: q-mesh too large, needs nosym=.true., electron_maxstep=200+, mixing_beta=0.2.
3. DFPT parse errors. The regex pattern in _parse_dielectric didn't match the ph.x output format across QE versions.
4. Never-produced files. Unhandled exceptions before volume.commit() wrote result JSON — 5 compounds simply never ran.

What Saved Us: The Honest-Null Pattern

The compound ledger did NOT adopt any failed values. The build script only reads kappa_300k_chosen, eg_hse, and e_* fields — all of which are null in error JSONs. The ledger correctly left these columns null for every failed compound.

This is the critical design decision that saved the rankings from corruption. Rankings were honest — they just reflected that GPU validation for ~15 compounds was still pending.

The lesson: error payloads will masquerade as results. If your pipeline reads fields from JSON files without checking the status code first, failed compute will silently corrupt downstream rankings. The honest-null pattern — leaving columns null when compute fails, rather than writing default or partial values — is what maintains data integrity when the compute layer is unreliable.

What Changed Afterward

• Docs that said "HSE06 queued" were reclassified as "HSE06 attempted and failed; retry required."
• Modal pseudo-potential delivery switched from add_local_file to a dedicated Modal Volume mount.
• All provisional filings proceeded with PBE band gaps + an explicit HSE06 caveat.
• The DFPT parser was rewritten to handle QE version-specific output format variations.