The Goal
We needed to run Density Functional Perturbation Theory (DFPT) calculations on Modal's serverless GPU infrastructure — specifically, Quantum ESPRESSO 7.x for HSE06 band gap and dielectric constant calculations on ~15 candidate materials. Modal's function dispatch, runtime, and shared-volume writes all work perfectly. The blocker was specifically the QE image build / image-pull step.
The Four Strategies
| # | Strategy | Outcome |
|---|---|---|
| v3 | apt_install('quantum-espresso') on debian_slim | Image builds OK, but detached runs hang at tasks=0 indefinitely — QE apt-install post-script silently stalls |
| v3b | Same v3 with ephemeral --detach relaunch | Same tasks=0 symptom — tested across 4 compounds (Li2HfO3, Rb2ZrO3, Li2BeCl4, Ba8Li2Nb6O24) |
| v4a | Image.from_registry('ghcr.io/cp2k/cp2k:2024.1') with apt QE overlay | 403 Forbidden — GHCR requires authentication |
| v4b | Image.from_registry('quantumespresso/qe-full:latest') | Docker Hub: manifest not found — image doesn't exist at that tag |
Diagnostic Baseline
A diagnostic v4 baseline confirmed that Modal's infrastructure itself works fine:
- Small image with no QE, function body + shared-volume writes: PASS
- Modal function dispatch + runtime + volume writes all work
- The blocker is specifically the QE image build / image-pull step
Why Each Strategy Failed
v3 (apt_install): The Debian quantum-espresso package has a post-install script that likely tries to configure MPI or set up environment variables in a way that conflicts with Modal's container execution model. The image builds successfully, but when the function is dispatched, it hangs at tasks=0 — the function never starts executing.
v3b (ephemeral): Re-launching with ephemeral --detach doesn't change the underlying image issue. The same stall occurs because the problem is in the image, not the dispatch mode.
v4a (GHCR CP2K): Using CP2K's container as a base and overlaying QE on top fails at the registry level — GHCR requires authentication that Modal's image pull doesn't support by default.
v4b (Docker Hub QE): The official Quantum ESPRESSO Docker Hub image (quantumespresso/qe-full:latest) simply doesn't exist. The tag was either never published or has been removed.
The Only Viable Path
Build a custom Dockerfile with QE 7.3 compiled from source, push to a private GHCR registry, and use Image.from_registry with authenticated pull. Estimated development time: ~4 hours. Budget: $30 Modal + 1-day infrastructure sprint.
Immediate fallback: Run DFPT locally on a single CPU. For a 12-atom primitive cell like Li2HfO3: SCF ~30 minutes, DFPT at Γ ~6-8 hours. All 4 priority DFPTs can run serially over a weekend (~48 hours) on a laptop.
Lessons for Cloud HPC
- Serverless GPU platforms work well for ML workloads with standard PyTorch/CUDA images. Scientific computing packages like QE have non-trivial install processes that don't map cleanly to serverless image builds.
- Don't assume Docker Hub images exist just because a project is well-known. Verify the registry tag before building your pipeline around it.
- Always have a local-compute fallback. For small-cell DFT, a laptop running overnight beats a blocked cloud pipeline.