Assessing the quality of publications

Papers

If you research in a field where the major outlets for your research are journals then the number of papers you publish is an indicator of your productivity as a researcher.

The more papers you publish the more productive you appear to other researchers. However, this does not necessarily indicate the quality of your research. You must also look at citations and impact factors.

Publishing research (Library) provides more information about selecting a journals to publish in.

Citations

When one research paper references your work published in another paper this is a citation and can be thought of as an indicator of quality. The more citations your paper gets, the higher the quality of your work.

Each time you reference someone else in one of your publications you are giving that person a quality 'vote'. Citation counting does not take into account the quality of the citing paper or whether the cite is a positive or negative cite.

Being cited (Library) provides access to and information about Web of Science

Citations per paper (CPP)

The number of citations you have received divided by the number of your publications. This simple calculation of citations per paper (CPP) demonstrates, on average, how many times each of your papers is cited.

The higher the CPP number the higher the quality of your research. This number is often used to represent the quality impact you are having in a field. CPP is an average however and can be misleading due to a couple of very highly cited papers, or by having hundreds of low cited papers.

Journal impact factor

The higher the impact factor of the journal you are publishing in the higher the quality of that journal. The journal impact factor tells you on average how many times articles published in the last two years have been cited.

It is good to publish in journals with high impact factors as in theory this means that your research will be cited more often. However, impact factors cannot be easily compared between disciplines and there are many other factors that influence impact factor such as, the number of review articles, self-citations, publication frequency and subject category.

Measuring journal quality (Library) provides more information and links to workshops and resources.

Relative Citation Impact (RCI)

The citation impact for a group of researchers is basically the number of citations per paper that that group has received over a certain time period. A relative citation impact can be calculated for any group of researchers and compared to an appropriate baseline.

The citation impact for a field of research can be calculated for, say, the world, and then the citation impact can be calculated for various institutions around Australia and their impact, relative to the world, can be compared as a measure of quality. By this definition the RCI for the world is always 1.00 for any field and therefore if the RCI for an institution is greater than 1.00 then it is performing above the world average for that field. Conversely if the RCI is less than 1.00 the institution is performing poorly compared to the average for that field in the world.

Often the source of the data may be provided in such a way as to make direct comparisons with groups or individuals quite difficult. This may include inconsistencies in the naming of research fields and subject categories.

H-index

Do you know what your h-index is?

To calculate your h-index take a list of your publications and sort them from the most cited to the least cited. Scan down the list until you find the minimum number of publications with an equal number of citations.

For example, if you have at least 5 publications with 5 or more citations each then your h-index is 5 (this is regardless of your total number of publications), or if you have at least 30 publications each with 30 or more citations then your h-index is 30. If you have a high h-index you can consider yourself to be a high quality and highly productive researcher. Note though that the h-index was originally designed for the field of physics research. The h-index works quite well for the physical sciences but may not directly transfer to other fields of research such as the humanities and social sciences. There are also now alternatives to the h-index which take into account the length of your research career.

Tiered ranking lists

What do you do if the outlets for your research are not ranked by Thomson?

What if they are not even journals?

Research outputs that may have related metrics could include conferences, books, book chapters, or journals not indexed by Thomson. In the light of increasing demand for bibliometric data to analyse research quality, many professional associations have been busy creating discipline specific lists of ranked outputs and outlets.

Publication Plus - UniSA ERA Journal Finder (staff access only) provides a search facility to find out the ranking of journals in your research discipline (you can search by FOR code or keyword).