Abstract: Hospitals are typically uncertain about the readmission impact of a care unit placement decision for a patient. The placement decision is challenging due to the wide variety of patient characteristics, uncertain needs of patients, and the limited number of beds in critical and intermediate care units. We develop an optimization-learning algorithm, called the Data-driven Admission Control (DAC) algorithm, under the presence of limited reusable hospital beds and delayed bandit feedback. The algorithm is designed to adaptively learn the readmission risk of patients and choose the best care unit placement for a patient based on the observed contextual information. The objective is to minimize patient readmissions while capturing the trade-off between the benefit of better health outcomes versus the opportunity cost of reserving high-demand beds for potentially more complex patients arriving in the future. We prove that our proposed online optimization-learning algorithm admits a sub-linear Bayesian regret bound. We also investigate and assess the effectiveness of our methodology using hospital system data. Our empirical results suggest that implementing our approach provides promising results compared to different benchmark policies and improves the current policy of our partner hospital.