Knowledge distillation
Knowledge distillation is a machine learning technique where a smaller model is trained to replicate the behavior of a larger, more complex model. This process helps deploy efficient models by transferring knowledge from a ‘teacher’ model to a ‘student’ model.