Graph Neural Networks (GNNs), especially equivariant message-passing neural networks (MPNNs), have emerged as powerful architectures for machine learning force fields. However, MPNNs face challenges when modeling non-local interactions in systems such as long conjugated molecules, due to oversmoothing and oversquashing. To address these concerns, we introduce Matrix Function Neural Networks (MFNs), a novel architecture that parameterizes non-local interactions through learnable equivariant matrix functions. Employing resolvent expansions offers a straightforward implementation and the potential for linear scaling with system size. The MFN architecture achieves state of-the-art performance in standard graph benchmarks, and is able to capture intricate non-local interactions in quantum systems such as long conjugated molecules, paving the way to new state-of-the-art machine learning force fields.