In times of unprecedented uncertainty, there is an increasing need for open dialogue on how to direct research and innovation investments towards sustainable and inclusive solutions. The STRINGS project has been tackling this complex challenge, investigating how to better understand the ways in which science, technology and innovation (STI) impacts upon efforts to achieve the UN Sustainable Development Goals (SDGs).

On Monday 28 February and Tuesday 1 March 2022, STRINGS hosted an online workshop – including presentations by the STRINGS research team on their project findings and guest speaker talks – to facilitate constructive discussions around the challenge of steering STI to address the SDGs.

Below, we provide an overview and recording of each session. We advise you to watch the recordings on desktop to view the presenters’ slides, which are unavailable on mobile view.

Session 1: Can current science, technology and innovation (STI) pathways lead to sustainable development?

Chair: Andrew Stirling (Science Policy Research Unit (SPRU), University of Sussex)

Opening remarks by speakers from the United Nations Development Programme (Pedro Conceição, Director of the Human Development Report Office) and UK Research and Innovation (Michael Booth, Joint Head of International Partnerships), reinforced a guiding premise of the STRINGS project – that progress towards the SDGs could be enhanced by a more purposeful use of STI.

Tommaso Ciarli (MERIT, United Nations University and SPRU) shared key findings from the STRINGS project to begin discussion of this crucial question – whether or not current STI pathways are effectively contributing towards meeting the United Nations 2030 Agenda for Sustainable Development.
Tommaso highlighted the fundamental issue of entrenched misalignments due to the uneven global distribution of STI investments across different countries and societies, so that often what is prioritized does not match the problems of those most in need. For example, in analysing past publications and patents in innovation, it is shown that whilst high income countries dominate the SDG research agenda, they produce the lowest proportion of research and innovation related to the SDGs. In comparison, although research in lower income countries is strongly related to the SDGs and has strengths such as being multidisciplinary, unfortunately this research is less funded and less collaborative. The talk summarised some of the main policy shifts that are needed if we are serious about STI contributing to the SDGs by 2030.

Peggy Oti-Boateng (UNESCO, France), Rajeswari Raina (Shiv Nadar University, India) and Anil Kumar Guptar (Honeybee Network, India) presented their own experiences of aligning STI prioritisations and pathways with global goals. Each speaker reinforced the need for interconnectivity between all individuals and communities impacted by STI-SDG relations:
Peggy Oti-Boateng’s presentation shed light on UNESCO’s development agenda, how gender is one of their core priorities, and how open science can serve as a transformative tool to reduce inequalities in STI.

Rajeswari Raina’s presentation focused on prioritising sustainable relationships in working towards sustainable development, discussing the process of inclusive innovation and referencing examples of collaborative research projects in India.

Anil Kumar Guptar presented on ‘Policy options for leveraging indigenous knowledge/grassroots innovation for meeting SDG goals’. Anil spoke about his experiences as founder of the Honey Bee Network, providing insight into their inspiring work and emphasising the different challenges faced by grassroot innovators.

Feedback presentations by Bitrina Diyamett (STIPRO, Tanzania), Francisca Mutapi (University of Edinburgh) and David O’Brien (IDRC, Canada) provided insightful reflections on the four presentations. Challenging questions were raised that interrogated tensions between economic development and environmental protection, and the STI-SDG paradox. Key facets of research were also discussed such as the politics of knowledge production – how it is conducted and evaluated – and the choice of methodologies and priorities. Attendees raised further points on reinforcing national systems of innovation, and research funding and impact.

Session 2: what local and global governance is needed to steer STI for the SDGs?

Chair: Tommaso Ciarli (MERIT, United Nations University and SPRU, University of Sussex)

Presenting for STRINGS, Andy Stirling spoke on the need for governance to enable greater diversity and plurality in research and innovation for the global goals. After reinforcing that there are no single, definitive ‘innovation fixes’ for any single SDG, he argued that recognising the importance of diversity and a plurality of perspectives within pathways, offers a pragmatic way of responding to many crucial and intractable governance challenges. Andy highlighted STRINGS’ practical aims amidst this complexity and what could be useful in this context, for example, helping to build formative governance networks in onward engagement processes and platforms.

Geoff Mulgan continued this discussion on what the STRINGS project may practically propose for global governance to better align research and development allocations and STI strategies to the SDGs. Building upon previous examples, Geoff spoke on establishing global platforms and coalitions to improve decision making on STI investments before presenting four clusters of recommendations, such as, more global pooled budgets to boost impact, focused on global goals or emergent new tools.

Francisco Sagasti’s presentation drew on his wealth of political experience,  for example, referencing the 1979 UN Conference on Science and Technology for Development. He encouraged reflection upon lessons learned from the last 60-70 years when considering: what is to be done in governance and financing on the global and regional level?

Aldo Stroebel gave insight from a public funder perspective, specifically the National Research Foundation, South Africa, sharing a range of encouraging examples that demonstrate the impactful, collaborative work that can be achieved in relation to STI-SDGs, before formally announcing the African Open Science Platform.

Anabel Marin’s presentation focused on grassroots innovation with Bioleft, an open-source initiative for seed breeding in Argentina. Anabel discussed Bioleft’s goal to help deliver an alternative knowledge system that promotes the direct involvement of farmers in the production and use of knowledge leading the development process. Anabel shared some successes and challenges faced throughout the project.

Group discussions emphasised the need for inclusion – to bridge gaps of wealth, gender and access to knowledge, and especially to better incorporate local communities into the knowledge-exchange systems and decision-making processes to contribute to global governance structures. Colleagues also discussed the influence of political change upon policy success, and the need for improved education, training and working dynamics for international collaboration.

Session 3: What types of STI can lead to a positive impact on SDGs? Under what conditions? In which contexts? With which synergies and trade-offs?

Chair: Joanna Chataway (STEaPP, University College London)

Saurabh Arora introduced the STRINGS team case studies which analysed specific STI-SDG challenges in three locations, each investigating:

• how are sciences being developed to address the Chagas disease in Argentina?
• how are conflicts around overfishing being addressed using STI in the Lake Victoria Region of Kenya?
• how are rice seeds for resilience being produced and adopted in Odisha, India?

Each case study lead – Valeria Arza, John Ouma-Mugabe and Rasheed Sulamain V – provided an overview of their work, explaining their research methodologies and key findings. Each lead presented how different STI pathways are valued by different stakeholders to differently address the above challenges. They highlighted the importance of giving attention to such different pathways, for STI to better address the SDGs.

Guest speaker Wilhemina Quaye (Council for Scientific and Industrial Research, Ghana) provided a comprehensive overview of the ways in which Ghana is developing its approach to grounding the SDGs in the national context through her presentation on the STI4SDG roadmap for Ghana, which prioritizes specific SDGs like SDG 2 (zero hunger) and SDG 9 (industry, innovation and infrastructure). She explained how those prioritisations are made in Ghana, providing a very useful example to steer STI towards the SDGs.

Following the presentations, attendees discussed in more depth what types of STI can lead to a positive impact on SDGs? Key points included the challenge of navigating the complex interlinkages between the various SDGs and the notion of intentionality behind policymaking. It was fantastic to hear a geographically diverse range of examples with references to work in Kenya, India and Mexico.

Session 4: How do we recognise STI impacts on SDGs? Using more inclusive data, tools and methods for research and policy.

Chair: Hugo Confraria (University of Lisbon and SPRU, University of Sussex)

Hugo highlighted the importance of session four’s guiding topic given the need for funders to prioritize the research they support ahead of the 2030 Agenda.

The STRINGS team presented their proposals to combine different data, methods and tools to elicit STI directions and pathways, with presentations by Ine Steenmans (UCL) and Ismael Rafols (Leiden University). Ine reinforced the intentionality central to the STRINGS approach when analysing STI-SDG relations, particularly in the importance of considering multiple perspectives, methods and data types to “really explore the diversity of innovation, science, research and the future of the SDGs.”

Ismael discussed ‘Mapping the SDGs for empowering stakeholders: capturing diversity and supporting plurality’, towards facilitating stakeholders to make their own informed choices for supporting specific SDG research relevant to their own contexts. To support such decision making, Ismael introduced the SDG Mapping Tool developed by STRINGS, and proposed a “new way of thinking about mapping research” using “plural and conditional mappings” rather than definitive maps.

Three speakers then presented their experiences using different tools and methods in the delineation and decision-making processes, within the specific contexts of their own work.

Tatiana Fernández (Economic Production, Government of Catalonia) discussed the use of open data, semantic techniques and visualisation tools before sharing insight on how these were applied to map the contributions of research and innovation to Catalonia’s ecosystem.

Glenda Kruss (Centre for Science, Technology and Innovation Indicators in South Africa) presented on ‘Developing new measures and indicators for STI oriented to SDGs in the South African context’. Glenda provided insight from a practitioner point of view with a focus on STI investments and the frameworks and models by which we can understand them, exampling the STI strategy for Africa 2024.

María Verónica Moreno (Solutions Mapping for the Accelerator Lab, UNDP, Argentina) focused upon the acceleration of development through citizen science. María shared inspiring true stories and drew on multiple examples of environmental citizen science projects like eBird Argentina, demonstrating the far-reaching, positive impact that such projects can have upon sustainable development.

Further discussion of these experiences followed in a Q&A session with the speakers, before the commencement of breakout sessions on the use of more inclusive data, tools and methods.

Amongst the various though-provoking points raised, the need to look more closely at intersectionality stood out with calls to consider: what do responsible metrics look like? And what can be done to tackle obstacles to accessing data and learning? Such questions relate to, and remind us of, the central issues of power and privilege within STI-SDG research work that the STRINGS project highlights. In his closing comments, Tommaso Ciarli reinforced the need for researchers to listen to views that diverge and dissent from mainstream research communities to better understand and align STI towards the SDGs.

The STRINGS team would like to thank all participants for their generous contributions and the invaluable sharing of their diverse experiences. A special thank you to artist Raquel Durán for producing live illustrations throughout the event.

Publication of the STRINGS Final Report is forthcoming and will be accessible via the Research Findings page of this website.

Andy’s talk focused on the challenge of “acknowledging different ways of knowing, not just how to implement sustainability, but what sustainability means in itself.” In particular, Andy emphasised that “sustainability is an inherently political notion”, exampling how different discourses and knowledges of sustainability are influenced by factors like power.

For example, he reinforced the need for diverse, plural perspectives on sustainability in order to better understand and tackle the reality of its multifaceted challenges – “towards recognising that difference is a way of knowing.” This differs to dominant, politicized narratives that suggest ‘the‘ singular pathways towards sustainability – an issue also addressed in the forthcoming STRINGS report.

Key findings from the new UNESCO report on universities and the 2030 Agenda were also referred to. The 2022 report discusses the role of higher education institutions in contributing to the sustainability 2030 Agenda, and Andy was one of several expert co-authors.

Read the UNESCO report on Knowledge-driven actions: Transforming higher education for global sustainability

This STRINGS workshop, led by project partner CRISP, will bring together key stakeholders to discuss ways to steer Science, Technology and Innovation for better alignment with the SDGs. It will use evidence from the case study undertaken in Odisha, India, on alternative pathways to support rice farmers to adapt to changing climate. The implications of the misalignment of pathways to varied SDGs/Targets will be discussed, as well as ways of steering towards achieving better alignment.

The shift in R&D goals towards the SDGs is driving demand for new S&T indicators…

The shift in S&T policy from a focus on research quality (or ‘excellence’) towards societal impact has led to a demand for new S&T indicators that capture the contributions of research to society, in particular those aligned with SDGs. The use of the new ‘impact’ indicators would help monitoring if (and which) research organisations are aligning their research towards certain SDGs.

Responding to these demands, data providers, consultancies and university analysts are rapidly developing methods to map projects or publications related to specific SDGs. These ‘mappings’ do not analyse the actual impact of research, but hope to capture instead if research is directed towards problems or technologies that can potentially contribute to improving sustainability and wellbeing.

…but indicators on the contributions of science on the SDGs are not (yet) robust

Yet this quick surge of new methods raises new questions about the robustness of the mappings and indicators produced, and old questions about the effects of using questionable indicators in policy making. The misuse of indicators and rankings in research evaluation has been one of the key debates in science policy this last decade, as highlighted by initiatives such as the San Francisco Declaration on Research Assessment (DORA), the Leiden Manifesto or The Metric Tide report in the UK context.

Indeed, the first publicly available analysis of SDG impact, released recently by the Times Higher Education (THE), should be a motive for serious alarm. For almost two decades, the THE has offered a controversial ranking of universities according to ‘excellence’. This last May it has produced a new ranking of universities according to an equally questionable composite indicator that arbitrarily adds up dimensions of unclear relevance. For example, the indicator of the impact on health (SDG3) of a university depends on the one hand on its relative specialisation on health – as captured, e.g. by the proportion of papers related to health (10% of total weight), and on the other hand on the proportion of health graduates (34.6%). But the weight is also based on (self-reported) university policies such as care provided by the university, e.g. free sexual and reproductive health services for students (8.6%) or community access to sports facilities (4%). This indicator is likely to cause more confusion than clarity and it is potentially harmful as it mystifies university policies for the SDGs.

The relative specialisation on health captured by the proportion of papers related to health in the THE ranking is partly supported by an Elsevier analysis of the publications that are related to the SDGs – which might seem more reliable than those based on data self-reported by universities.

However, mapping publications to the SDGs is not as straightforward as it might seem. An article published last month by a team at the University of Bergen (see Armitage et al., 2020) sounded the alarm by showing that slightly different methods may produce extremely different results. When comparing the papers related to SDGs retrieved with their own analysis with those by Elsevier, they found that there is astonishingly little overlap – in most SDGs only around 20-30% as illustrated in Figure 1. The differences also affected the rankings of countries’ contributions to the SDGs. The Bergen team concluded that ‘currently available SDG rankings and tools should be used with caution at their current stage of development.’

Figure 1. Comparison between the Bergen and Elsevier approaches to mapping SDG-related publications. Based on Web of Science Core collection, 2015-2018. Source: Armitage et al. (2020)

Why are mappings of publications to SDGs so different? Lack of direct relation between science and SDGs

Perhaps we should not be surprised that different methods yield so different results. The SDGs refer to policy goals about sustainability in multiple dimensions – ending poverty, improving health, achieving gender equality, preserving the natural environment, et cetera. Science and innovation studies have shown that the contributions of research to societies are often unexpected and highly dependent on the local social contexts in which knowledges are created and used.

Nevertheless, most research is funded according to the expectations of the type of societal benefits that it may generate – and thus one can try to map these expectations or promises according to the language used in the (titles and abstracts of) projects and articles. Unfortunately, the expected social contributions are often not made explicit in these technical documents because the experts reading them are assumed to see the potential value.

As a consequence, the process of mapping projects or articles to the SDGs is ineluctably carried out through an interpretative process that ‘translates’ (or attempts to link) scientific discourse into potential outcomes. Of course, such translation is dependent on the analysts’ understandings of science and the SDGs. There is consensus on some of these understandings. For example most analysts would agree that research on malaria is important for achieving global health. However, other translations are highly contested: should nuclear (either fission or fusion) research be seen as a contribution to clean and affordable energy? Should all educational research be counted as contribution to the SDG on ‘quality education’?

Furthermore, in a number of SDGs such as gender equity (SDG 5) or reduced inequalities (SDG 10), there is a lot of ambiguity on the potential contributions. In particular, there is relatively little research specifically on these issues in comparison to the research with outcomes affecting gender relations and inequalities.

Another challenge of these mappings is that the databases used for analysis are not comprehensive, having a much larger coverage of certain fields and countries (See Chapter 5 in Chavarro, 2017). This is particularly problematic when analysing research of the Global South.

In summary, there are many societal problems where there is lack of consensus and ambiguities, and in these cases, the mappings will depend on the particular interpretation of the SDGs that the mapping methods implicitly adopt.

A plurality of SDG mapping methodologies

It follows from the previous discussion that different analyses carry out different ‘translations’ of the SDGs into science through the choice of different methodologies. The study by Clarivate (2019) is based on a core set of articles that mention ‘Sustainable Development Goals’ – thus it is related to research areas with an explicit SDG discourse.

The approaches developed by Bergen University, Elsevier, the Aurora Network and SIRIS Academic are based on searching for strings of keywords, in particular keywords found in the UN SDGs targets or other relevant policy documents. These searches are then enriched differently in each case. The hypothesis of this ‘translation’ is that publications or projects containing these keywords are those best aligned with the UN SDG discourse. The question is then where should the line be drawn. For example, why in some lists zika virus is included in the list of health SDG3, but not the closely related dengue virus, with a much higher disease burden?

An alternative approach being developed at NESTA and Dimensions uses policy documents and keywords to train machine learning algorithms in order to identify articles related to the SDGs instead of creating a list of keywords to search the articles. The downside of this approach is that is it a black box regarding the preferences (or biases) of the machine learning algorithms.

Comparisons as a pragmatic way forward

In the face of this plurality of approaches potentially yielding disparate results, the STRINGS project aims to be a space for constructive discussion and comparison across different methodologies. A comparison between methods will help in finding out to which extent there is consensus or dissensus in the mappings of various SDGs.

To this purpose, in collaboration with the Data Intensive Science Centre at the University of Sussex (DISCUS), on July 23-27 we have carried out a hackathon focussed on retrieving publications related to clean energy research (SDG 7) (to be reported). We have also organised a workshop to discuss the results obtained by the different teams mentioned above with the various methodologies, and how each methodology might capture a particular ‘translation’ or understanding of the SDGs. As proposed by the Bergen team, this comparison ‘will allow institutions to compare different approaches, better understand where rankings come from, and evaluate how well a specific tool might work’ for specific contexts and purposes.

Disclaimer

The discussion in this blog builds on ongoing work carried out by the STRINGS project. It presents my personal view (rather than the project’s) following my engagement debates on the use of indicators in policy and evaluation, for example a recent participation in an EC Expert Group on ‘Indicators for open science’.