PhononBench:a large-scale phonon-based benchmark for dynamical stability in crystal generation

Generative models based on graph neural networks, diffusion models, and large language models have advanced the design of crystalline materials. Existing evaluations commonly follow the stability–uniqueness–novelty (S.U.N.) framework, where stability is primarily assessed thermodynamically and does not capture dynamical stability. However, the high computational cost of phonon calculations has hindered large-scale assessment of dynamical stability in generated crystals. In this work, we introduce PhononBench, the first large-scale benchmark for dynamical stability in AI-generated crystals. Leveraging the recently developed MatterSim interatomic potential, PhononBench enables efficient large-scale phonon calculations and dynamical-stability analysis for 133 838 crystal structures generated by seven leading crystal generation models. PhononBench reveals a widespread limitation of current generative models in achieving dynamical stability: the unweighted mean stability rate across the eight independent MP20 entries is only 30.7%, with MatterGen reaching 45.0%. Unless otherwise specified, these metrics use a phonon-frequency threshold of −0.1 THz and MatterSim-self-consistent labels, which may overestimate stability relative to density functional theory. Further case studies show that in property-targeted generation—illustrated here by band-gap conditioning with MatterGen—the dynamical-stability rate remains 41.0% even at the optimal band-gap condition of 0.5 eV. In space-group-controlled generation, higher-symmetry crystals exhibit better stability (e.g. cubic systems achieve rates up to 53.4%), yet the average stability across all controlled generations is still only 44.7%. An important additional outcome of this study is the identification of 32 995 MatterSim-labeled crystal structures that exhibit minimum phonon frequencies above −0.001 THz. We further analyze threshold sensitivity and structural and physical factors associated with dynamical instability. Beyond benchmarking, PhononBench provides a web-based service for rapid phonon-spectrum calculation and dynamical-stability assessment of crystalline materials, facilitating accessible and scalable validation for the materials community.