№ 10/2018

Fìnansi Ukr. 2018 (10): 28–43https://doi.org/10.33763/finukr2018.10.028

FINANCE OF EDUCATION AND SCIENCE

PAVLIUK Klavdiia ¹, KAMINSKA Olena ²

¹SESE “The Academy of Financial Management”
OrcID ID : https://orcid.org/0000-0002-9495-6630
²SESE “The Academy of Financial Management”
OrcID ID : https://orcid.org/0000-0001-5859-5623

Methods and tools for quantitative and qualitative evaluation of scientific-research labor

The article reveals the issues of evaluating productivity and quality of research work, the solution to which affects the efficiency of the scientific and innovative system of the country, that is a task of great importance for today . The definition of high-quality science is necessary to ensure the progress of science in order to ultimately solve economic and social problems. In the process of evaluating research work alongside with expert conclusions, the scientometric and bibliometric methods used to quantify scientific activity play an important role, influencing the allocation of financial resources. Relying on authoritative databases a comparison of expenditures on science in developed countries with the number of publications in these countries is made, and a clear correlation between these values is revealed. The use of such a tool as the impact factor to determine the quality of scientific publications, its advantages and disadvantages are considered. A more objective information on the scientific results of a research institution or an individual scientist is given by the h-index, which is also proposed to be calculated as an alternative indicator for assessing scientific influence of journals. The potential of known databases as a source of evaluation of scientific results in various fields of knowledge is considered. It is concluded that the most acceptable source of bibliographic data for assessing performance and quality of scientific research in the socio-humanitarian field is Google Scholar which provides the systematization of all publication activity of researchers from open sources on a grant basis, which makes it possible to transparently review and finance. The best way to assess productivity in science is to combine qualitative (expert) methods and quantitative (bibliometric) indicators. Greater productivity and the quality of research must be stimulated accordingly.

Keywords:science, productivity, evaluation, bibliometric methods, expert methods, databases, research incentives

JEL: І22, І23, І25, І28

PAVLIUK K. . Methods and tools for quantitative and qualitative evaluation of scientific-research labor / K. . PAVLIUK, O. Kaminska // Фінанси України. - 2018. - № 10. - C. 28-43.

Article original in Ukrainian (pp. 28 - 43)

Download

Downloads :506

1. Österreichischer Wissenschaftsrat. (2014). Die Vermessung der Wissenschaft. Messung und Beurteilung von Qualität in der Forschung. Wien. Retrieved from www.wissenschaftsrat.ac.at/news/Messung_Endversion_inkl%20Cover.pdf.

2. Porev, S. M., Sandyha, I. V. Indicators of science critical for the creation of Ukrainian research universities. Marketing and innovation management, 3, 246-262. Retrieved from mmi.fem.sumdu.edu.ua/sites/default/files/mmi2016_3_246_262_0.pdf [in Ukrainian].

3. Cabinet of Ministers of Ukraine. (2018). Some issues of the state attestation of higher education institutions in terms of carrying out their scientific (scientific and technical) activities (Decree No. 652, August 22). Retrieved from www.kmu.gov.ua/ua/npas/deyaki-pitannya-provedennya-derzhavnoyi-atestaciyi-zakladiv-vishchoyi-osviti-v-chastini-provadzhennya-nimi-naukovoyi-naukovo-tehnichnoyi-diyalnosti [in Ukrainian].

4. Cabinet of Ministers of Ukraine. (2017). On approval of the procedure for conducting state certification of scientific institutions (Decree No. 540, July 19). Retrieved from zakon.rada.gov.ua/laws/show/540-2017-%D0%BF [in Ukrainian].

5. Müller, H. (2012). Zitationen als Grundlage von Forschungsleis - tungsrankings - Konzeptionelle Überlegungen am Beispiel der Betriebswirtschaftslehre. Retrieved from www.bzh.bayern.de/uploads/media/2-2012-Mueller.pdf.

6. Bornmann, L., Marx, W. (2008). The Wisdom of Citing Scientists. Retrieved from arxiv.org/ftp/arxiv/papers/1308/1308.1554.pdf.

7. Frey, B., Rost, K. (2008). Do Rankings Reflect Research Quality? Retrieved from crema-research.ch/papers/2008-22.pdf.
doi.org/10.2139/ssrn.1280763

8. Belter, C. W. (2015). Bibliometric indicators: opportunities and limits. Retrieved from www.ncbi.nlm.nih.gov/pmc/articles/PMC4613388/.

9. Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, Vol. 69, Iss. 1, 131-152.
doi.org/10.1007/s11192-006-0144-7

10. Sanchez, T. W. (2015). Citation Analysis of urban planning Scholars in the U.S.
doi.org/10.22227/2305-5502.2015.1.3

11. Harzing, A.-W., Van Der Wal, R. (n. d.). Google Scholar: the democratization of citation analysis. Retrieved from www.researchgate.net/publication/228856805/download.

12. Ferrara, E., Romero, A. F. (2013). Scientific impact evaluation and the effect of self-citations: mitigating the bias by discounting h-index. Retrieved from arxiv.org/ftp/arxiv/papers/1202/1202.3119.pdf.

13. Van Raan, A. F. J. (2005). Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, Vol. 62, Iss. 1, 33-143.
doi.org/10.1007/s11192-005-0008-6

14. Rauter, J. (n. d.). Wissenschaftliche forschung und ihre evaluation expansive zitationsanalyse auf deskriptiv-intertextueller basis. Retrieved from www.univie.ac.at/voeb/fileadmin/Dateien/Publikationen/Schriften_der_VOB/Band_1_Beitraege/Band_1_RauterJ.pdf.

15. Bornmann, L., Daniel, H.-D. (2009). The state of h index research. Is the h index the ideal way to measure research performance? EMBO Reports, 10 (1).
doi.org/10.1038/embor.2008.233

16. Lacasse J., Hodge, D. R., Bean, K. F. (2011). Evaluating the Productivity of Social Work Scholars Using the h-Index. Research on Social Work Practice, 21(5), 599-607.
doi.org/10.1177/1049731511405069

17. Honcharuk, A. H. (2012). On the quality of scientific research. Intellectuals and power, 25, 137-142. Retrieved from www.irbis-nbuv.gov.ua/cgi-bin/irbis_nbuv/cgiirbis_64.exe?I21DBN=LINK&P21DBN=UJRN&Z21ID=&S21REF=10&S21CNR=20&S21STN=1&S21FMT=ASP_meta&C21COM=S&2_S21P03=FILA=&2_S21STR=iiv_2012_25_13 [in Ukrainian].

18. Impact factor is a citation index. (2015, November 7). Open science in Ukraine. Retrieved from openscience.in.ua/impact-factor.html [in Russian].

19. Baethge, C. (2015). Nichtperfekter Impact-Faktor.
doi.org/10.3238/arztebl.2012.0267

20. AWMF. (2001). AWMF-Vorschlag zur Verwendung des Impact-Faktors. Retrieved from www.awmf.org/ forschung-lehre/kommission-fl/forschungsevaluation/bibliometrie/impact-faktoren.html.

21. Archambault, E., Gagné, É. V. (2004, August). The Use of Bibliometrics in the Social Sciences and Humanities. Retrieved from www.science-metrix.com/pdf/SM_2004_008_SSHRC_Bibliometrics_Social_Science.pdf.

22. Seglen, P. O. (2007, April 29). Citations and journal impact factors: questionable indicators of research quality. Retrieved from onlinelibrary.wiley.com/doi/abs/10.1111/j.1398-9995.1997.tb00175.x.

23. h-index. (n. d.). Retrieved from uk.wikipedia.org/wiki/H-%D1%96%D0%BD%D0%B4%D0%B5%D0%BA%D1%81 [in Ukrainian].

24. Todeschini, R., Baccini, A. (2016, August). Handbook of Bibliometric Indicators: Quantitative Tools for Studying and Evaluating Research. Retrieved from books.google.com.ua/books?id=5PwdDAAAQBAJ&pg=PT446&lpg=PT446&dq=Lovergove+and+Johnson+2008&source=bl&ots=Hny4ED-_X_&sig=hDJb8dRa9xDfnsaRkm6rO0iZDq0&hl=ru&sa=X&ved=2ahUKEwjQlrfF7rfdAhUJblAKHVzfCsAQ6AEwB3oECAkQAQ#v=onepage&q=Lovergove%20and%20Johnson%202008&f=false.

25. Bornmann, L., Daniel, H. D. (2007, June 13). What do we know about the h index? Retrieved from onlinelibrary.wiley.com/doi/pdf/10.1002/asi.20609.

26. Google Scholar. (n. d.) Retrieved from scholar.google.com.ua/.

27. Cabezas-Clavijo, A., Delgado-Lopez-Cozar, E. (2013, March 19). Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment.
doi.org/10.1016/j.medine.2013.05.002

28. Minasny, B., Hartemink, A. E., McBratney, A., Jang, H. J. (2013, October 22). Citations and the h index of soil researchers and journals in the Web of Science, Scopus, and Google Scholar.
doi.org/10.7717/peerj.183

29. De Groote, S. L., Raszewski, R. (2012, June 30). Coverage of Google Scholar, Scopus, and Web of Science: a case study of the h-index in nursing. Retrieved from www.ncbi.nlm.nih.gov/pubmed/22748758.

30. Khalid, M., Mahmood, K., Dean, A. H. (2017). Review of Google scholar, Web of Science, and Scopus search results: The case of inclusive education research. Retrieved from digitalcommons.unl.edu/cgi/viewcontent.cgi?article=4419&context=libphilprac.

31. Pauly, D., Stergiou, K. I. (2005). Equivalence of results from two citation analyses: Thomson ISI's Citation Index and Google's Scholar service. Ethics in Science and Environmental Politics, 33-35.
doi.org/10.3354/esep005033

32. Bush, V. (1945). As we may think. The Atlantic monthly. Retrieved from www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881.

33. Movchan, K. (2017). Open access to scientific information is the basis of the development of science, education and culture. Humanitarian space of science: experience and perspectives. Pereiaslav-Khmelnytskyi. Retrieved from dspace.msu.edu.ua:8080/bitstream/123456789/413/1/%D0%9C%D0%BE%D0%B2%D1%87%D0%B0%D0%BD%20%D0%9A.%D0%9C..pdf [in Ukrainian].

34. Dilger, A., Müller, H. (2012). Ein Forschungsleistungsranking auf der Grundlage von Google Scholar. Zeitschrift für Betriebswirtschaft, 82.
doi.org/10.1007/s11573-012-0617-5

35. Dilger, A., Müller, H. (n. d.). A Citation Based Ranking of Researchers in German Business Administration on the Basis of Google Scholar.
doi.org/10.2139/ssrn.1703823

36. Breuer, W. (2009, June 14). Google Scholar as a Means for Quantitative Evaluation of German Research Output in Business Administration: Some Preliminary Results.
doi.org/10.2139/ssrn.1280033

37. Kotlyarevsky, Ya. V., Radchenko, A. I., Melnikov, O. V., Semenyuk, E. P. (2018). Strategic Priorities in Measuring the Publication and Publishing Works in Scholarly Research Activity. Science and Innovation, 14 (5). Retrieved from scinn.org.ua/ua/archive/14(5) [in Ukrainian].
doi.org/10.15407/scine14.05.005

38. Bornmann, L., Wohlrabe, K. (n. d.). Working Paper Normalization of Citation Impact in Economics. Retrieved from www.econstor.eu/bitstream/10419/171056/1/cesifo1_wp6592.pdf.

39. Quan, W., Chen, B., Shu, F. (n. d.). Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016). Retrieved from arxiv.org/ftp/arxiv/papers/1707/1707.01162.pdf.