We consider the problem of sequentially allocating sample observations to learn personalized treatment strategies, motivated by the design of adaptive clinical trials that aim to learn the best treatment as a function of patient covariates. In such settings there may be clinical knowledge of which covariates are predictive (they may interact with the treatment choice) and which are prognostic (they may influence the outcome independent of treatment choice). We extend the expected value of information (EVI)/knowledge gradient framework to develop useful heuristics for a context with predictive and prognostic covariates and a delay in observing outcomes. We also propose and analyze closely related Monte Carlo-based allocation policies to enhance our proposal’s computational efficiency and applicability for adaptive contextual learning. We show that several of our proposed allocation policies are asymptotically optimal in learning treatment strategies. We run simulation experiments motivated by an application for clinical trial design to assess potential treatments of sepsis. We illustrate that the proposed EVI-based allocation policies, with knowledge about which covariates are predictive and prognostic, can improve the rate of inference relative to some existing approaches to adaptive contextual learning.