One of the main challenges in small molecule drug discovery is efficiently finding novel chemical compounds with desirable properties. Such properties can be physico-chemical (like logD or solubility), pharmacokinetic (like permeability, clearance or metabolic stability) or pharmacodynamic (like biological activity on targets of interest). Computational chemistry has since long been involved in the drug discovery process from hit selection to lead optimization. In silico methods allow for fast and cost-effective filtering steps before the chemical matter is even synthesized. Here, we discuss how deep learning can be utilized for many different aspects of modeling in chemistry. In cheminformatics, the first step is to describe the chemical matter in a computer-readable way. We present two alternatives to the commonly applied circular fingerprints: graph convolutions on the molecular graph and a sequence-to-sequence autoencoder on SMILES notations. We then demonstrate how combining different ADMET endpoints together in one multitask deep learning model can boost the predictive performance compared to its single-task alternatives, especially on endpoints that are more difficult to model. Finally, combining our reversible encoding of the chemical space with improved predictive models and an optimization algorithm, we demonstrate how a query compound can be optimized with respect to multiple (predicted) molecular properties. We hope that our method will support medicinal chemists in accelerating and improving the lead optimization process by proposing synthesis ideas and handling multi-objective optimization.