In many modern vision and graphics applications, deep neural networks (DNNs) enable state-of-the-art performance for tasks like image classification, object detection and segmentation, quality enhancement and even new content generation. In this talk, we will demystify basic concepts behind DNN training: from the definition of a deep neural network to critical parameters controlling deep learning model training. We will examine problems typically encountered when training a model and will share best practices for their mitigation. Finally, we will shed some light on commonly used software frameworks.