Zishi Zhang, Jinhui Han, Ming Hu, Yijie Peng
This paper proposes a novel pretrain-then-finetune approach using a Transformer model to tackle small-data, large-scale optimization problems by leveraging synthetic data and domain knowledge.
The study introduces a new method for solving complex decision-making problems where a company has to make many decisions with limited data. Inspired by how large language models work, the researchers use a two-step process: first, they 'pretrain' a model using synthetic data that represents expert knowledge, and then 'finetune' it with real-world data. This method helps the model learn efficiently by combining general knowledge with specific real-world insights, making it adaptable to various situations. The approach is particularly beneficial when there are many instances to learn from, as it improves the model's performance over time.
