Report on the TD1210 Workshop “Alternative metrics or tailored metrics: Science dynamics for science policy”, November 9-10, 2016 Warsaw
This workshop continued the many-events thread of research on new methods in scientometrics using the multidisciplinary nature of Knowescape, but it added purposefully new ‘flavors’ to this topic.
The main question set out in the call was “What kind and how much metrics do we need, and can effort?”. Hence critical reflection of metrics and possible new metrics was one target of the workshop. The other target was to engage in this discourse not only inside the scientometric community, but to bridge to other organisations, as EC, OECD, ICSU which deal with statistics, corresponding metrics, and sometimes even entering modeling science dynamics. It is due to the vigorousness of Jan Kozlowski and Janusz Holyst from the local organizer team that this reaching out goal was achieved. Credit belongs to Jan also by embedding this workshop into the far-back reaching tradition of organizing libraries, and books and manuscripts in them. This was tangible not only in the setting of the workshop in the building of the famous Zaluski library, but also in his own presentation. It was also a theme taken up in modern shape later by Alesia Zuccala’s presentation. Another outreach I would like to mention is also due to Jan: to link our meeting to other societal metrics as taken care off in statistical offices. The presentation of the main statistician of Poland Dominik Rozkrut about how the emergence of Big Data changes the work of statistical offices widens the view also on scholarly metrics.
The topic of the workshop was taken up in the first keynote by Kevin Boyack, known for front research on science maps. He put the question to the table which kind of motives are driving investment into research and development. In the US, funding of R&D does not come only from the public hand. There is a huge part of private engagement into research via foundations and other programmes. To which extend can we make ‘other motives’ for R&R visible using traditional bibliometric traces? His presentation opened another perspective on the role of R&D in society and resonated with the current European discourse about an impact of science into the whole society. If we think in terms of society, grand challenges, top sectors, what kind of information can we use as proxy of societal relevance?
The student papers rush from students from the LIS school of the Alexander Technological Educational Institute of Thessaloniki (ATEITH) showed which topics are prevailing in the education of future data curators, librarians and data scientists. Among them was a question about citation ontologies as developed for semantic web application. What are the effects of such a deeper-level indexing of scholarly work? What tools and approaches are already around? What new insights can be we expect (see also the presentation of van den Besselaar and Martin-Martin)?
The second day started with a presentation of Loet Leydesdorff, one of the most known scholars in scientometrics and the social studies of science. He started his presentation with the statement “I’m a sociologist” and consequently a critical reflection about metrics was the red thread through his presentation. Based on an extensive oeuvre of quantitative studies of science and innovation systems, he seemingly effortless unraveled and interrogated metrics, the complexity of phenomena metrics aims to capture, and policy implications of the application of metrics. This critical reflection was seconded later the day by presentations of Christopher Watts and Ray Harris about the British Research Assessment Exercise and its history. One take-away from Loet Leydesdorff’s presentation: Start with a research question, not with data! A truly scholarly attitude.
The presentation of Laura Rideneor about ‘Boundary Objects’ stud for such a use of scientometrics for understanding the science system. Throughout the workshop, the discussions emphasized that metrics is not primarily about monitoring. The science system is a self-organised system, with an autonomous, inner dynamics. It would be naïve to assume that it can be steered following a linear model. Having said this, science or academic operates not in isolation; its boundary conditions are set by society at large and national science policies in particular. The second day ending took this up with a very special session: a discussion about thoughts, ideas, proposals to re-shape higher education in Poland; the status of universities; their inner organization; their autonomy but also adaptability. The discussion was introduced by a presentation of Arkadiusz Radwan from the Allerhand Institute. The question at stake is how to make the Higher Education System in Poland fit for the challenges of the globalized world of the 21 century? Has metrics a role in this? What are the experiences in other countries? That international experts were allowed to take part in a remarkable open, honest and rational debate on alternatives felt as a great privilege. To enable this kind of exchange of view, thoughts, opinions and experiences is probably the biggest virtue of a COST meeting as the one in Warsaw.
Once more, TD1210 workshops have proven to be a platform where experts from various domains can meet and discuss topics in a more intimate setting. The variety of expertise, knowledge and skills present in our network, lets me return from our meeting always very energized and inspired.
A thank to the programme committee for advice and support and Jan Kozlowski and Janusz Holyst for being excellent local hosts for this meeting.
Warsaw-Berlin-Amsterdam
November 11, 2016
Andrea Scharnhorst
(With input from Janusz Holyst and Jan Kozlowski)
PS: The presentations will become on-line – step by step in agreement with the presenters – at the website here: http://knowescape.org/workshop-alternative-metrics-tailored-metrics-november-9-10-2016-warsaw-programme/ .
