Large vision models (LVMs) trained on a large and diverse set of imagery are revitalizing computer vision, just as LLMs did for language modeling. However, LVMs are not nearly as effective when applied to unique types of imagery. To handle labeled data scarcity without overfitting, we need models that are tuned to a specific domain of imagery. Whether it’s a single medical imaging modality, multispectral drone photos or snapshots from a manufacturing line, these fine-grained applications are best captured with a model that can accommodate the available data. A small vision model with fewer parameters improves generalizability with the added bonus of better computational efficiency so that it can run on an edge device. In this talk, I will show why domain-specific models are essential and how they can be trained without labeled data. I will conclude by demonstrating the efficacy of domain-specific models in handling small training sets, imbalanced data and distribution shifts for various types of imagery.