Removing Weather-Related Image Degradation at the Edge

For machines that operate outdoors—such as autonomous cars and trucks—image quality degradation due to weather conditions presents a significant challenge. For example, snow, rainfall and raindrops on optical surfaces can wreak havoc on machine perception algorithms. In this talk, we will explain the key challenges in restoring images degraded by weather, such as lack of annotated datasets, and the need for multiple models to address different types of image degradation. We’ll also introduce metrics for assessing image degradation. We will then explain our solutions and share our results, demonstrating the efficacy of transformer-based models and of a novel, language-driven, all-in-one model for image restoration. Finally, we’ll highlight the techniques we’ve used to create efficient implementations of our models for deployment at the edge—including quantization and pruning—and share lessons learned from implementing these models on a target processor.