Abstract: The use of neural network parametrizations to represent the ground state in variational Monte Carlo (VMC) calculations has generated intense interest in recent years. As I will demonstrate in the context of the periodic Heisenberg spin chain, this approach can produce unreliable wave function approximations. One of the most obvious signs of failure is the occurrence of random, persistent spikes in the energy estimate during training. These energy spikes are caused by regions of configuration space that are over-represented by the wave function density, which are called “spurious modes” in the machine learning literature. More generally, accuracy away from the region of high physical wave function density is typically very low. After describing the phenomenon, I will report on our attempts to develop training procedures that lead to more robust wave function estimates.