In this talk, we introduce two new aspects of inverse problems formulated as PDE-constrained optimization. Firstly, while current approaches assume deterministic parameters, many real-world problems exhibit stochastic behavior. We present a novel approach that treats the PDE solver as a push-forward map to recover the full distribution of unknown random parameters. We introduce a gradient-flow equation to estimate the ground-truth parameter probability distribution. Secondly, as problem dimensions increase, Monte Carlo methods regain relevance. However, directly applying them to gradient-based PDE-constrained optimization poses challenges due to the product of forward and adjoint solutions involving Dirac deltas. We propose strategies to rescue Monte Carlo methods and make them compatible with gradient-based optimization. This talk is based on joint work with Qin Li and Yunan Yang.