Malaria is a life-threatening disease caused by the parasites transmitted through the bite of the female Anopheles mosquito. Thick and thin film microscopic examinations of blood smears are the most commonly used and reliable methods for diagnosis, however, its accuracy depends on the smear quality and human expertise in classifying the normal and parasitemic cells. Manual examination can be burdensome for large-scale diagnoses in endemic regions resulting in poor quality, unnecessary medication, leading to severe economic impact to the individual health program. Automated malaria screening using machine learning techniques, such as deep learning, offers the promise of serving as an effective diagnostic aid. In this study, we propose the advantages offered through visualizing the features and activations in a simple, customized deep learning model. We apply it to the challenge of malaria cell classification, and as a result the model achieves 98.61% classification accuracy with lower model complexity and computation time. It is found to considerably outperform the state of the art including other pre-trained deep learning models.