Abstract: We introduce a model reduction technique for high-dimensional stochastic systems having a low-dimensional invariant effective manifold with slow dynamics, and high-dimensional, large fast modes. Given only access to short bursts of simulation, we design an on-the-fly consistent algorithm exploring the effective state space efficiently. This construction enables fast, efficient simulation of the effective dynamics that averages out fast modes, plus estimation of crucial features of such dynamics, including the stationary distribution, identification of metastable states, and residence times and transition rates between them. We also implement our algorithm on three model systems: “pinched sphere”, “butane model”, and “oscillating half-moons”, where in particular, the last example has large nonlinear fast modes. The results of these systems show the accuracy, efficiency, and robustness of our approach. If time permits, I will also briefly introduce in the end another recent work in hierarchical reinforcement learning, where we introduce a new fast multiscale procedure for repeatedly compressing Markov decision processes by using parametric families of policies to abstract sub-problems at finer scales. The multiscale representation yields substantial improvements in convergence rates both locally within sub-problems and globally across sub-problems. These multiscale decompositions also yield new transfer opportunities across different levels and different problems, by summarizing useful skills and higher-order functions from previously learned policies, which further enable systematic curriculum learning. In addition, we provide additional features like virtual policies and recursion. We demonstrate all the features above in a collection of illustrative domains."