Adaptive traffic control systems (ATCSs) adjust traffic signal timing based on demand. We begin by presenting the main building blocks of a vision-based ATCS, including pre-processing, vehicle detection, vehicle classification and vehicle tracking. Next, we examine several of the key technical challenges in developing a computer vision-based ATCS and explore approaches for overcoming these challenges. These challenges stem from the need for an ATCS to perform accurate person and vehicle detection despite a huge variety of vehicle types, occlusion of objects of interest and difficult lighting conditions.