Henry Jiang
DARWIN is an evolutionary GPT model that uses a genetic algorithm approach to iteratively improve its performance by allowing GPT agents to modify each other's training code, resulting in a 1.26% improvement in FLOPS utilization and a 2.07% improvement in perplexity over five iterations.
DARWIN is a new approach to training AI models, specifically GPT models, by using a method similar to natural selection. In this system, several AI agents are trained separately, and each one can suggest changes to the others' training code to try and improve their performance. The best-performing agents are selected to continue to the next round of training. This process is repeated multiple times, and DARWIN showed improvements in how efficiently the models use their computing resources and in their ability to predict text. The system also allows for human input, where the AI can request additional resources or changes to its training setup.
