This was part of Advances in Optimal Decision Making under Uncertainty

Policy Evaluation, Policy Gradient, and Actor-Critic Learning in Continuous Time and Space

Xunyu Zhou, Columbia University