We describe a sequence of steps that facilitate effective learning of treatment policies in mobile health. These include a clinical trial with associated sample size calculator and data analytic methods. An off-policy Actor-Critic algorithm is developed for learning a treatment policy from this clinical trial data. Open problems abound in this area, including the development of a variety of online predictors of risk of health problems, missing data and disengagement.