Automated composition of agentic networks for multi-metric user preference satisfaction

The construction of agentic networks to satisfy user-specific preferences is becoming a key component for value creation beyond model serving. In this paper, we use the evaluation of an agentic network over a QA dataset to stir its architecture towards the maximal satisfaction for different user profiles. We model a user profile as a distribution of preference over a set of metrics and the network as a composition of stochastic processes. We add constraints on token budget and response time in the framework by adding constraints on the searchable domain. Our algorithm first analyzes the sensitivity of each metric as the network is modified and then suggests atomic edits to its composition. This process is repeated until a desired satisfaction or convergence is reached. In our experiments, we compare manual compositions for an industrial Retrieval Augmented Generation (RAG) solution with our stirred networks for multiple user profiles. Additionally, we compare the speed, cost, and performance of our algorithm with SOTA frameworks in deep search datasets.