Boxiang Zhang, Baijian Yang
CORP is a one-shot structured pruning method for Vision Transformers that preserves accuracy without retraining by using a closed-form solution to compensate for removed structures.
Vision Transformers are powerful but require a lot of computing resources. To make them more efficient, a new method called CORP can prune or remove parts of these models without needing retraining or extra optimization steps. This is done by using a small set of data to adjust the remaining parts of the model so they work well even after pruning. Tests show that using CORP allows for significant reductions in the model's size while maintaining high accuracy, making it faster and more efficient to use.
