Testing ML Models and Code: Practical MLOps Techniques

MLOps (short for “machine learning operations”) applies the best practices from DevOps—collaboration, version control, automated testing, compliance, security and continuous integration/continuous delivery—to productionizing ML. MLOps and DevOps are similar when it comes to continuous integration, source control, unit testing, integration testing and continuous delivery; however, there are also notable differences. In this talk, we will examine the testing-related aspects of the MLOps pipeline. We’ll consider both testing the coding logic of a model and testing the accuracy of the model. Testing the coding logic is similar to testing software applications and should follow the same practices used in traditional software development. Testing the accuracy, however, involves testing the performance of significant model metrics and requires different techniques. We will present practical MLOps techniques covering feature and data tests, tests for reliable model development, and ML infrastructure tests. We’ll illustrate these techniques with real-world examples from our work in the automotive sector.