Leveraging the Wizard of Oz Method for Stimulating Design: Benefits and Challenges From an Application Case Study

This paper presents a comprehensive understanding of the use of the Wizard of Oz (WOz) method for studying emerging de- sign tools. This method involves hidden human "wizards" simulating the functionalities of an undeveloped technology without the user’s knowledge, making it a valuable method for assessing technologies that are not yet mature or are too resource-demanding. Although WOz has been widely used in fields like human-robot interaction and autonomous vehicles, its application in design research is relatively novel. Through a case study aimed at stimulating design ideas generation, we applied the WOz method to simulate an AI design assistant based on designer’s sketches during 17 one-and-a-half-hour design sessions. We evaluated the success of this WOz simulation, and the results showed exemplary performance by the wizards in both time management and content creation, along with a strongly positive designer perception of the simulation, and a rich data collection to inform research analysis. The WOz method offers two major advantages: versatility across domains and tasks, and cost-effectiveness. However, it also presents challenges, including the workload on wizards and ethical concerns. We conclude with three key axes of recommendations for implementing WOz in design settings: training wizards for feasibility, clearly defining simulated functionalities and experimental protocols, and ensuring system credibility. These insights will support future design research anticipating techno- logical advancements to study emerging tools.