Evaluating Experience Labs

For The Glasgow School of Art, Institute of Design Innovation

This project explores ways of evaluating design-led approaches to innovation. Building on insights from academic literature, it investigates conceptual and practical evaluation challenges through creative methods. Together with a fellow researcher, I considered academic literature on the changing understanding of design innovation, the role of innovation labs in this context, and how evaluation approaches have changed to address the challenges of evaluating such projects.

We then use and build on these insights to investigate ‘Experience Labs,’ a design-led approach to innovation developed by the Institute of Design Innovation. The resulting report introduces the concept of ‘learning based innovation’ as a framework for articulating the value produced by design research. Moreover, we developed visual tools to use this framework to help Experience Labs practitioners learn from their own work as a form of formative and participatory evaluation. This way, our research addresses both the conceptual and practical challenges of evaluating Experience Labs.

Introduction and Context

Experience Labs is at the core of Scotland’s Digital Health and Care Institute model of innovation and is led and developed by the Institute of Design Innovation at The Glasgow School of Art. Bringing together academic, business and civic partners, Experience Labs promote a new way of working within the health and social care context in Scotland. Within each lab, the Experience Labs team work collaboratively with participants to understand and investigate their experience using design research methods and to explore possible alternatives in order to tackle the complex health challenges facing Scottish society. Together with a fellow researcher, I was asked to develop an evaluation framework that can capture the impact and value of Experience Labs in light of its inherent complexity.

The Innovation Lab Landscape

To better understand the changing landscape of innovation labs worldwide, we conducted desk research and looked at the work of a range of teams who address social and public challenges and who design or uncover new ways of working. We located Experience Labs as part of a new understanding of innovation as flexible and emergent, where experimentation and collaboration are increasingly important. As part of this paradigm, one of the defining characteristics of many labs is their complexity: they do not prioritise implementation, they embrace risk and failure, they create indirect and non-linear impacts due to their emergent and processual nature, and they are itself part of a complex and wicked environment.

Experience Labs

Focusing on Experience Labs more specifically, and gradually widening the lens to also include its wider structure and context, we uncovered more and more layers of complexity. We delved into internal documentation such as operational documents, project reports, videos, team meetings, and papers published by the Experience Lab team, and visually mapped structures, contexts, and relationships in order to explore their implications. Throughout this process, we increasingly realised that the complex nature of Experience Labs and its flexible emergent approach to innovation, is not only an inconvenient but inseparable feature of this project. It is in fact the main way Experience Labs create value and where its greatest potential lies.

Evaluation

Defining what a meaningful evaluation of Experience Labs would look like and focus on, meant defining what is to be evaluated, and describing the activities, outcomes, impacts, and context of Experience Labs. Yet we quickly realised that due to its inherent complexity, Experience Labs and its impact cannot be captured by applying traditional evaluation approaches. Like many other evaluators, we see the evaluation of such projects as a territory open to experimentation where there are not yet well defined or recognised methods. Our research contributes to this novel territory by addressing both the conceptual and practical challenges of evaluating Experience Labs.