As edge computing demands smaller, more efficient models, knowledge distillation emerges as a key approach to model compression. We delve into the details of this process, exploring what knowledge distillation entails and the requirements for its implementation, including dataset size and tools. We examine when to use knowledge distillation, its pros and cons, and showcase examples of successfully distilled models. Based on performance data highlighting the benefits of distillation, we conclude that knowledge distillation is a powerful tool for creating smaller, smarter models that thrive at the edge.
Empowering Edge AI: Knowledge Distillation for Smaller, Smarter Models
Session Speakers
-
David Selinger
CEO, Deep Sentinel
David Selinger is a technologist at heart and has founded a number of successful start-ups, including Redfin and RichRelevance (and some unsuccessful ones too!). He has served a number of nonprofits including My Two Front Teeth (now the Family Giving Tree), the Darfur Stoves Project (now Potential Energy) and Silicon Climate. His favorite technology right now is deep learning, specifically as applied to image and video object detection/recognition. David believes that this technology is undergoing a disruptive improvement and will change numerous industries. That’s why he started Deep Sentinel—which puts deep learning to practical use protecting families and businesses.
