Publication and citation metrics have been used for many years now as apparently objective parameters to evaluate educational institutions as well as individual researchers. A recent report in Science, about the Saveetha Institute of Medical and Technical Sciences (SIMATS), near Chennai, Tamil Nadu [1], highlights concerns about the value and limitations of such metrics in evaluating the importance of research publications, authors, journals and institutions.

Initially established as a dental college in 1988, SIMATS is now a deemed university, with additional schools of nursing, physiotherapy, engineering, medicine, management, law, occupational therapy, physical education, health sciences, pharmacy, and architecture and design. It has been receiving increasingly good ratings on the National Institutional Ranking Framework [2]. Based on these ratings, in 2023, Saveetha Dental College was ranked as the top dental college in India by the government’s Ministry of Education [2].

The Science report states that all undergraduate dental college students at Saveetha were required to develop research papers with the help of faculty, as part of their programme. This ultimately resulted in a large number of publications from the institution. Students were also required to insert, in their papers, unnecessary citations to other work by Saveetha students and faculty; the practice of faculty colleagues citing each other was also observed [1].

The allegations in the Science report merit a detailed forensic investigation. However, irrespective of what the truth is, such controversies serve as a timely reminder of how easy it is to manipulate numbers. It is, of course, often necessary to cite one’s own work; but self-citation can also be done inappropriately, in order to boost the importance of that work, of the cited author, and of the author’s institution. There are also instances of editors — and reviewers — recommending to authors that they cite specific papers in their manuscripts [3]. If these papers are from the journal that the manuscript has been submitted to, or if it is a paper that has been written by the reviewer, it provides the benefit of improving the journal’s impact factor or the reviewer’s citation count, respectively. The problem of inappropriate self-citation is widespread, and it must be noted that such research misconduct is not the preserve of low- and middle- income countries [4].

Ratings systems — whether the national institutional ranking framework in India or commercial enterprises like the Times Higher Education World University Rankings which uses Scopus, the database of the scientific publisher Elsevier — assign marks for teaching, research (including publications per staff member in indexed journals), and citations (the number of publications from the institution, and how often they are cited in the scientific literature) [5]. Ranking in rating systems, journal impact factor (a value calculated by the analytics company Clarivate based on the ratio of the number of times a journal indexed in Clarivate’s Web of Science database has been quoted in the past year to the number of citable papers in the journal in the previous two years), and the citation rates of institutions, departments and individual researchers are also considered when applying for research grants. This can encourage misconduct in many ways, one of which is inappropriate citation [6].

The problem with the impact factor

Inappropriate citation is not the only concern. Both Indian and foreign universities lay strong emphasis on the impact factor of the journal in which the faculty member has published for matters related to annual appraisal, promotions, etc. Faculty members are encouraged to publish in “high-impact journals” (a phenomenon that has been termed “Impactitis”) [7]. This has multiple adverse effects: because most Indian journals have relatively low impact factors, Indian academics are forced to submit to the “high-impact” journals controlled by commercial scientific publishers in high-income countries. These publishers, along with data analytics companies in these countries, also define the impact factor — citations of articles in their databases, by other articles in these databases. This results in a Catch-22 situation: the best research from India with the greatest potential value is often submitted to high-impact journals in the West. Consequently, the impact factor of Indian journals persistently stays low. To make matters worse, such research published in high-impact journals is often behind a paywall and out of reach of the community which is most likely to benefit from it.

Impact factors put a stamp of approval on the research itself. For example, the databases of these high-impact and high-profit journals such as those from the Elsevier group are used to calculate impact factors. Articles in high-impact journals are generally viewed as more credible, making these articles more likely to be cited. Moreover, an excellent paper that is published in a journal that is not in one of the databases may not receive the citation rate that it might obtain in a high-impact journal. In effect, these various ranking systems are an agreement of sorts between the academic publication industry and the higher education industry which feeds them, and are used to draw students, faculty, and grants, as well as to decide on the value of research.

The challenges of running a journal in India

Indian journals will continue to have a low impact factor because the same system that demands publication in a high-impact journal does little to support the establishment and development of good journals in India. Most journals in India run on limited resources and voluntary or underpaid staff. Journals require multiple inputs, from money and enabling appointments of professional editors and editorial staff, to helping journals gain a national and international presence, encouraging researchers to submit to these journals, and so on. In the absence of such support, many Indian journals struggle to survive, even to do routine work such as prompt review of submissions. In this situation, publishers like Wolters Kluwer (which bought over the Indian publisher Medknow a decade ago) and Springer approach Indian journals with agreements to take over the routine journal work, and also possibly help them improve their impact factors. Some of these journals are behind a paywall.

It is a shame that the academic community in India uses the measures determined by the academic publishing industry in rich countries to get research funding, jobs, promotions, etc, instead of developing a strong academic publishing community here. We do accept that objective data are needed for the evaluation and ranking of universities and institutions, especially in a global setting. Citation metrics are indeed a relatively good guide to judge the importance of a research paper or topic, though there is a need to determine whose citation metrics should be used. The citation metrics being used currently are flawed, and do not necessarily give an accurate evaluation of what they are supposed to judge — the importance of an article, a journal, an author, or an institution. We do not advocate the excessive focus on metrics such as the journal impact factor and citation rates for promotions and related assessments for the above reasons. Our excessive dependence on such criteria exposes the flaw in the system. It is wise to recall Goodhart’s law: When a measure becomes a target, it ceases to be a good measure [8].

Much has been said about the pressure to publish in Indian academia and its consequences — such as fraud in research as well as the emergence of predatory journals ready to publish anything for a fee. We must widen our gaze and also recognise the flaws in metrics such as the impact factor. Indian universities do a disservice to Indian science, researchers and journals by adopting criteria set by an academic publishing industry in the West. We need to focus on developing and strengthening our own journals and developing reasonable criteria to measure their influence on society and in the scientific community. This should not be a private enterprise; instead, it should be a public service and must be the responsibility of the academic community.