PhononBench:A Large-Scale Phonon-Based Benchmark for Dynamical Stability in Crystal Generation

In recent years, generative artificial intelligence has made significant advances in the design of crystalline materials, giving rise to a variety of approaches based on graph neural networks, diffusion models, and large language models. Existing evaluations commonly follow the stability-uniqueness-novelty (S.U.N.) framework, where “stability” is primarily assessed using thermodynamic criteria, which do not fully capture the dynamical stability essential for a material’s practical existence. Dynamical stability is an important factor influencing the synthesis and persistence of materials under practical conditions, although experimental realization also depends on anharmonic effects, finite temperature, defects, disorder, and synthesis conditions. However, the high computational cost of such calculations has, until now, prevented large-scale and systematic assessment of dynamical stability in generated crystals. In this work, we introduce PhononBench, the first large-scale benchmark for dynamical stability in AI-generated crystals. Leveraging the recently developed MatterSim interatomic potential, which achieves density-functional-theory (DFT)-level accuracy in phonon predictions across more than 10,000 materials, PhononBench enables efficient large-scale phonon calculations and dynamical-stability analysis for 133,838 crystal structures generated by 7 leading crystal generation models. PhononBench reveals a widespread limitation of current generative models in ensuring dynamical stability: unless otherwise specified, all reported dynamical-stability metrics are evaluated at a phonon-frequency threshold of -0.1 THz, with the average dynamical-stability rate across all generated structures being only 32.15%, and the top-performing model, MatterGen, reaching just 45.05%. Further case studies show that in property-targeted generation—illustrated here by band-gap conditioning with MatterGen—the dynamical-stability rate remains as low as 41.0% even at the optimal band-gap condition of 0.5 eV. In space-group-controlled generation, higher-symmetry crystals exhibit better stability (e.g., cubic systems achieve rates up to 53.4%), yet the average stability across all controlled generations is still only 44.7%. An important additional outcome of this study is the identification of 32,995 crystal structures that exhibit minimum phonon frequencies above the adopted threshold along the sampled highsymmetry paths. Furthermore, systematic multi-threshold sensitivity analyses are performed, and the structural and physical factors associated with dynamical instabilities in generated crystals are investigated, providing deeper insights into the limitations of current AI-based crystal generation approaches. By establishing the first large-scale dynamical-stability benchmark, this work systematically highlights the current limitations of crystal generation models and offers essential evaluation criteria and guidance for their future development toward the design and discovery of physically viable materials.