Introduction
REAL Supply’s ultimate aim is to meaningfully support better long-term decision-making on health and social care.
This sort of ‘impact’ is notoriously slow to emerge, largely because of the time it takes to build good relationships with relevant stakeholders. The long-term (7 year) funding and initial 2-year stakeholder engagement phase of the project are promising foundations from which to begin this journey. However, as we’ll explore below, this sort of complex challenge entails a range of dilemmas which we argue require a re-framing of rigour in order to navigate.
Dilemmas
Below we outline just three examples of areas – spanning various uncertainties, politics and competing needs – where dilemmas can arise. This is by no means comprehensive as we intend only to paint a picture of the complex nature of stakeholder engagement challenges at play for the REAL Supply Research Unit.
Knowing who to engage with and how to keep it up
Identifying key stakeholders is often the starting point for any process of engagement. The desire to find ‘the right’ people, teams and organisations is fraught with uncertainty of different types.
In some cases, it is clear what ‘right’ means. For example, it is clear that for the REAL Supply project engagement should include NHS Trusts. But pragmatic uncertainties here include which Trusts, who specifically within each Trust, and how do we best engage with them?
However, it is also clear that who counts as the ‘right’ stakeholders depends on our definitions of ‘long-term’, ‘decision-making’ and ‘supply’ in different health and social care contexts.
Finally, given the 7-year time frame, it is inevitable that people will move jobs, teams will come and go, and stakeholder priorities will change – often in response to short-term party politics. So the ‘right’ stakeholders in the early phases of the project will likely be different from those in the later stages.
How do we choose and maintain a research focus and build ongoing trusted relationships under such conditions?
Whose reality and knowledge counts?
Stakeholder engagement and evidence use is always political. Not necessarily in the big ‘P’ party political sense, but in the small ‘p’ sense that there are always unavoidable subjective and value-laden choices to be made. These can entail trade-offs in benefits, costs and the potential reinforcement or redistribution of power.
A central political dilemma in pursuing ‘effectiveness’ in the context of the context of stakeholder engagement relates to whose values, beliefs, and assumptions help define what the ‘problem’ is and whose evidence counts as valid and useful. Relationship-building and knowledge mobilisation are more effective when these ‘frames’ are aligned (i.e. when diversity is low) – but this compromises effectiveness in addressing complex societal problems, which requires deliberation across diverse perspectives and knowledges.
Added to this, different stakeholders have different degrees of power and capacities to engage, leading to unequal influence over framing choices. In this context, politics relates to the exercise of power to reduce ambiguity about what the problem is and to control what types of evidence are used to justify decisions. Our choices of who we engage with and how will unavoidably contribute and/or challenge these inequalities in different ways, with real-world implications for the types of impact we have.
How should we navigate the ethical and equity implications of this?
Whose needs are being met?
Researchers, policy makers and practitioners will come to the engagement table with different sets of needs, motivations and agendas that can be in tension.
Decision-makers often require clear, timely, quantifiable and actionable outputs with a focus on ‘what works?’. This may not align with what researchers can or feel compelled to provide, nor their publication needs for contributing to their academic field and career.
Furthermore, optimising the ‘fit’ between decision-making demand and research supply can result in a consultant/client type relationship. Leaving aside the associated politics that we outline above, academics are unlikely to perform as well as consultants in such cases, due to differences in time frames and professional cultures. Even when research is delivered satisfactorily, there are no guarantees it will be used, so academics can be left without the evidence of instrumental impact they so often need.
Finally, even when effective negotiation of mutual benefit between decision-maker demand and research supply takes place, this can neglect the needs and views of arguably the most important stakeholder group: those impacted by the decisions at stake.
How should we navigate these competing demands?
Inclusive rigour as a response to complexity
In dominant academic and professional usage, rigour is associated with the pursuit of certainty, objectivity and control. But as we’ve outlined above, the conditions under which the REAL Supply project is operating are far from certain, objective or controllable. So how do we navigate these complexities without compromising on rigour?
Building on an idea put forward by Robert Chambers (2015), a group of evaluation and learning professionals across the world are developing a promising re-framing of rigour better suited to conditions of complexity: inclusive rigour.
In the figure below, we have made an initial attempt to adapt their framework to explore how it might apply to projects such as REAL Supply.
At its core, we believe inclusive rigour is about creating the conditions and building capacities for learning in order to adapt through uncertainties, politics and competing needs. This entails embracing a core tension between evidence-informed planning (e.g. via a theory of change) and the emergent changes that come from working in uncertain contexts. Attention should be paid to notice such changes and flexibility built into projects in order to respond appropriately.
Inclusive rigour requires us to focus on how different approaches to, and aspects of engagement, research and learning can combine effectively to meet competing needs. There is an art and craft to this, as articulated by Aston and Apgar (2022), who make the case for ‘bricolage’ as involving the intentional repurposing and combining of a patchwork of (parts of) methods.
The quality of participation and relationships – both within the team and with stakeholders – is paramount to inclusive rigour. Where many ‘traditional’ notions of rigour are seen to be compromised by diverse participation, inclusive rigour is bolstered by it. It is necessary for the combining of different knowledges and triangulation between them, in order to produce relevant, robust, equitable and useful insights. It is also required for building the trust, legitimacy and shared ownership of these insights so that they are more likely to be used.
In these ways, the ‘rigour’ of inclusive rigour lies in the relationships between the different people, parts and purposes of a project, rather than solely within particular research methods.
Dr Clementine Hill O’Connor is a Kentigern Research Fellow in the School of Social and Political Sciences, at Glasgow University. She was a researcher in the Systems Science in Public Health and Health Economics Research (SIPHER) consortium, and now sits on the advisory board.
Dr Dave Blackbell is the Knowledge Mobilisation Lead for the REAL Supply Research Unit, and is Co-Director of the Scottish Policy and Research Exchange.
