We consider two types of rate-distortion problems associated with Bayesian learning. Formulating a rate-distortion problem by the distortion measure defined by the pointwise regret of the model parameter and another rate-distortion problem by replacing the empirical expectation with the true expectation, we show the relationships between the asymptotic behaviors of the rate-distortion functions and the learning coefficients of Bayesian learning. Furthermore, the two rate-distortion bounds for a fixed prior derive a relationship between the expected regret and Gibbs generalization error.