Phase Detection with Neural Networks: Interpreting the Black Box

Neural networks (NNs) normally do not allow any insight into the reasoning behind their predictions. We demonstrate how influence functions can unravel the black box of NN when trained to predict the phases of the one-dimensional extended spinless Fermi-Hubbard model at half-filling. Results provide strong evidence that the NN correctly learns an order parameter describing the quantum transition. Moreover, we demonstrate that influence functions not only allow to check that the network, trained to recognize known quantum phases, can predict new unknown ones but even guide physicists in understanding patterns responsible for the phase transition. This method requires no a priori knowledge on the order parameter, the system itself, or even the architecture of the ML model.