Deep learning has made a huge impact on a wide variety of computer vision applications. But while the capabilities of deep neural networks are impressive, understanding how to best apply them is not straightforward. In this talk, we highlight key questions that must be answered when considering incorporating a deep neural network into a vision application. What type of data will be most beneficial for the task? Should the DNN use other types of data in addition to images? How should the data be annotated? What classes should be defined? What is the minimum amount of data needed for the network to be generalized and robust? What algorithmic approach should we use for our task (classification, regression or segmentation)? What type of network should we choose (FCN, DCNN, RNN, GAN)? We’ll explain the options and trade-offs, and map out a process for making good choices for a specific application.