Bibliometric indicators and scienti c evaluation; discussing the eff ect of di fferent measures in a vectorial-dendogram approach
A report on STSM-TD 1210-030414-042036 “RAS paper in progress” at U. Roma, La Sapienza, IT Host : Prof. Ssa. Giulia Rorundo
During my stay at U. Roma, La Sapienza from 2014-04-03 to 2014-04- 12, a paper to be submitted to a peer review journal had to be completed. Its tentative title was/is Bibliometric indicators and scienti c evaluation; discussing the eff ect of diff erent measures in a vectorial-dendogram approach
Main item
During my stay at U. Roma, La Sapienza from 2014-04-03 to 2014-04-12, a paper based on joint work about bibliometric indicators and scienti c evaluation has been much worked upon. The subject pertains to the following considerations. Bibliometric indicators are increasingly used in the (pre)selection of hiring academic sta ff and in promotion. The application of bibliometric indicators aims at supporting an e ffective and objective decision for the best candidate for its scientifi c institution. But, if the complex nature of scienti c knowledge production and the mathematical feature of used indicators are insufficiently understood, the eff ect can be opposite. The joint paper, resulting from the research work, fi rst outlines various evaluation criteria and related indicators, with a special focus, as an illustrative example, on practices in the recent Italian academic system.
The e ffect of di erent criteria are discussed, like the value (more precisely ranking) of the best 10 papers, according to their impact factor (IF), published over a given time interval and presented by 5 candidates for an arti ficial recruitment or promotion process. It is shown that no agreement on the ranking of candidates can be attained, thereby verifying Arrow’s theorem.
Specifi cally, the interplay between adapted bibliometric indicators (in particular impact factor related) and the distribution of “perceived results by the community” through publications, in such an arti cial applicant group is mathematically described through the pairwise preference notions used in economics, i.e. establishing the components of the “vector of evaluation criteria”.
Other items
During my stay, other relevant activity items can be mentioned. I have met many local colleagues of my host, with fruitful interactions on related subject, including vizualisation of data. I have met a future (Italian) student of my host, to be hopefully working on a related subject to the STSM. I have met a foreign (Albanian) Ph. D. student of the Department, and discussed with him related matter to the STSM. I have met and much interacted with a visitor form another Italian University, U. Macerata, and discussed subjects related to the present STSM.
Conclusion
It can be considered that the goal of the STSM has been well ful filled, maybe beyond usual expectations. Scienti c and social interactions have been quite successful.
Marcel Ausloos
April 14, 2014
