Sub-Plenary 1-6
Research Evaluation at the Crossroads of Science:
The Effects and Implications of the Machinery
of Research Evaluation
Thursday, July 4, 2024, 16:00–17:30 CEST
U6 Building – “AGORÀ” | Room: U6-P0-09 | Piazza dell’Ateneo Nuovo, 1 | 20126 Milano
Organizers:
Eliana Minelli, Università Carlo Cattaneo – LIUC, Italy
Maria Rucsandra Stan, University of Milan-Bicocca, Italy
Panelists:
Lars Engwall, Uppsala University, Sweden
Alis Oancea, Kellogg College, University of Oxford, United Kingdom
Emanuela Reale, Research Institute on Sustainable Economic Growth (IRCrES), Italy
Alesia Ann Zuccala, University of Copenhagen, Denmark
This sub-plenary session will engage scholars in a deep reflection on research evaluation systems and practices and their implications for the development and wellbeing of our societies. Studying the effects and impact of research evaluation is crucial as it enhances research quality, ensures research integrity, guides resource allocation, accurately assesses research impact, and promotes inclusivity and innovation within the research environment.
Research evaluation mechanisms have a significant influence on publishing patterns and behaviours (Korytkowski & Kulczycki,
2019; Moed, 2008; Cronin, 2013), resource allocation (Carlsson, 2009; Neave, 1991; Gläser, 2007), and academic careers (Marini,
2017; Marzolla, 2015). Furthermore, evaluation has potential governance effects, as governance modes “reflect the missions
of higher education and research” (cf. EGOS Colloquium 2023, Sub-Plenary 1-1: Collegiality in Higher Education and Beyond:
Rethinking the Organized World), influencing the hierarchical relationships between actors, but also at level of the scientific
community, reconfiguring the academic work (Molas-Gallart, 2012).
The research evaluation traditions of European countries have often followed neo-liberal principles, aimed at steering at
a distance while ensuring the autonomy of universities, as well as promoting accountability through strengthening the capacity
to plan, control and measure achievements (Neave, 1998; Westerheijden, 2007; Rosa & Amaral, 2007). The literature has addressed
the critical aspects of research evaluation arising from the transformation of the institutional and social contexts, from
the perspective of the transnational authority of science (Whitley, 2003) and emerging interdisciplinary models of research
(Altbach & Lewis, 1996; Moscati, 2001; Rostan, 2011). Nonetheless, the effects of research evaluation on careers, hierarchies,
governance and management at institutional level and on research quality at the societal level are still underexplored.
Based on the evidence from an Italian research program of national interest, the sub-plenary will reflect on the principles
underlying evaluation systems and focus on national assessment practices, exploring the implications for the specific university
mission and career trajectories of researchers, as well as in terms of funding allocation. We encourage the audience to share
their experiences from their respective systems, fostering a diverse and comprehensive discussion. We will conclude the sub-plenary
by exploring the wider relevance of research evaluation beyond academia, particularly highlighting its impact on society and
how the current evaluation system either facilitates or hinders the delivery of high-impact research.
This sub-plenary aims at delivering three ‘key takeaways’:
Reflect on the ontological and epistemological foundations of the current evaluation mechanisms;
The role and responsibilities of the researcher in research evaluation discourses;
The need for a more nuanced approach to research evaluation, which considers both intrinsic and extrinsic characteristics of quality and recognized the complexity of defining and assessing research beyond academia.
The sub-plenary will complement the Colloquium’s theme of “Crossroads for Organizations: Time, Space, and People” by examining
the impact of research evaluation on organizations and individuals. Furthermore, the research aligns with the Colloquium’s
focus on adapting organizations to the interconnected “onlife” world, where individuals and technology shape and transform
organizational dynamics. While the sub-plenary primarily focuses on higher education, it aims to foster discussions about
the broader impact beyond academia and embrace diverse perspectives of the numerous disciplines that converge at these “crossroads”.
In this regard, we will revisit previous organizational theories emphasizing the role of research organizations, including
universities and research centres, in promoting innovation and collaboration to adapt to dynamic environments.
References
Lars Engwall is Professor Emeritus of Business Studies at Uppsala University, Sweden. He has published widely on institutional change
and the diffusion of management ideas, in particular the role of management education and of the media.
Eliana Minelli is an Associate Professor at Carlo Cattaneo University LIUC (Castellanza, Italy) in Organization and Management. She has
published on research assessment and evaluation in peer-reviewed journals, including Quality in Higher Education and Higher Education Policy, and has also published books on the same topic. Since 2019, Eliana has coordinated the LIUC unit of the PRIN Project “The
effects of evaluation on academic research: knowledge production and methodological issues”.
Alis Oancea is a Professor of Philosophy of Education and Research Policy at the University of Oxford, United Kingdom, and Project Lead
(Research on Research) at the Economic and Social Research Council (ESRC) Centre for Global Higher Education (CGHE). Alis
is the Joint Editor of the Oxford Review of Education and Chief Editorial Advisor on Routledge Open Research (Education). Alis has published in numerous peer-review journals, including Oxford Review of Education, European Educational Research Journal, Studies in Higher Education, and Research Evaluation.
Emanuela Reale has been Director of the Research Institute on Sustainable Economic Growth (IRCrES CNR) until 2023 and is currently Associate
Senior at IRCrES in Turin, Italy. Her main areas of interest are higher education policy, governance, R&D funding, STI indicators,
research evaluation, and impact assessment. Emanuela is member of the Board of the Consortium of Higher Education Researchers
(CHER), of the European Network of Indicator Designers (ENID), and President of the European Forum for Research and Innovation
(EU-SPRI).
Maria Rucsandra Stan is a currently a post-doctoral researcher in Strategy and Entrepreneurship at University Milan-Bicocca, Italy. During her
PhD she has participated in the PRIN Project ‘The effects of evaluation on academic research: knowledge production and methodological
issues’.
Alisa Ann Zuccala is an Associate Professor at the Royal School of Library and Information Science, University of Copenhagen, Denmark. Her
research specialization is in the field of bibliometrics, scholarly communication, research evaluation, and in recent years
she has focused primarily on the evaluation of research outputs across the humanities. Alisa Ann has published in numerous
peer-reviewed journals, including Journal of Informetrics, Scientometrics, and Research Evaluation.