Supplementary file 1_CORAL: co-evolutionary optimization of routing and adaptive prompts for cost-efficient LLM deployment.docx

To address this issue, two methodologies are adopted: using the best model while keeping the prompt fixed, and changing the prompt according to the query while changing the model. A model has been designed that uses the Prompt–Router Co-Evolutionary Problem (PRCP) and CORAL, which stands for Co-Evolutionary Optimization of Routing and Adaptive Prompts.

It considers model selection and prompt patterns as interrelated operations and is structured to improve the performance of both simultaneously. This is achieved by maintaining two groups: one for prompts and the other for routing strategies, allowing a linked interconnection between them. This entire process is guided by multi-objective optimization techniques (NSGA-II/III) to improve correctness, cost balance, throughput, and latency.