On the 13th of February Nava gave a talk at HCID at City University London: Evaluating the Visualisation of Plans in the SAsSY Project (Talk slides)
Abstract: The Scrutable Autonomous Systems (SAsSy) project aims to enable the scrutiny of autonomous systems by allowing agents to generate plans through argument and dialogue, while justifying the purpose of each step within the joint plan. Humans or agents can then critique these plans by suggesting and justifying alternative courses of action as needed, thus driving the planning process. However, this requires the system to first present the plan that has been chosen for execution by the system to the user. Given that these plans are often long and complex, one key challenge of providing transparency to humans regarding the internal workings of an intelligent system is therefore (good) information presentation. One of the challenges has been to decide how to adapt the plan presentation to a user’s role and area of responsibility. This talk addresses a key question of how to address the “goodness” of an adaptation. It describes a layered-evaluation approach, measuring performance in a dual task effectiveness) to complement measures such as time to complete (efficiency), and self-reported measures. This talk describes two case studies evaluating visualisations of a plan that illustrates the value of the approach.