Inappropriate citation practices can lead readers astray or, at worst, generate scientific myths. To prevent this, researchers should be aware of the pitfalls of bias, distinguish fact from speculation and look up original sources.

M  any people believe that spinach is especially rich in iron, whereas others are convinced that this is an urban legend resulting from a decimal-point error. Decades later, it turns out that both parties are mistaken1 and that both beliefs are the result of sloppy citation practice. The academic myth about spinach is just one of many caused by inappropriate citation practices, ranging from debates on whether salt is healthy to whether the proclaimed rise in jellyfish populations is true2.

Citations are vital to academic research because they contribute to the edifice of knowledge and build scientific belief – factual claims that we eventually accept as established knowledge. To avoid inappropriate citation practices, we advise researchers to: i) be aware of biases and minimize them; ii) distinguish facts from hypotheses and speculation; and iii) read the original sources, rather than secondary ones.

Recognize and minimize bias

Citation biases come in different flavours. One involves favouring the citation of supporting studies while neglecting contradicting evidence. Work by Steven A. Greenberg, an associate professor of neurology at Harvard Medical School, reveals how citation biases can promote scientific myths. To understand the emergence of the false scientific belief about the occurrence of an Alzheimer protein in a skeletal disease, Greenberg analysed a complete citation network – that is, how various papers relating to the topic are linked through citations. He found that authors were prone to predominantly cite studies that supported the link between the protein and the disease (94% of the total citations after 12 years). By contrast, primary results that did not support the claimed link received only 6% of the total citations.

Another type of citation bias is regional bias, whereby studies performed in certain geographical regions are preferentially cited. By looking at a worldwide geographical citation network, Santo Fortunato and colleagues found that scientists from a given geographical region were more likely to cite other scientists from the same or nearby regions. The authors also found that the citation streams between different locations correlated strongly with the collaborations between these places.

Indeed, it is often essential to cite the work of collaborators because of its relevance to your own work. But it becomes inappropriate when friends and peers are cited to curry favour. According to Fister and Perc, a rare but often-overlooked problem is that of the so-called citation cartels: organised groups of authors who have agreed to preferentially cite each other. There are also reported instances of journal editors asking authors to cite papers from the journal they are submitting to in order to boost the journal’s impact factor. And, finally, we are sometimes told by some of you that you feel the need to cite certain papers in case the authors on those papers would act as reviewers on your paper.

To avoid these types of citation biases, we at Elevate advise you to stick to two simple rules. First, only cite works that are relevant to your scientific story, nothing more, nothing less. Cite relevant papers based on the science, not the popularity of the journal in which the paper was published or the reputation of the authors. Cite papers that not only support your findings but also challenge them. This will establish trust with your readers (including the editors and reviewers) and also help provide valuable context for your paper.

Second, be selective and balanced in how you cite. Unless you’re writing a review, you need to present a selection of the literature, rather than a thorough overview. Maintain balance by looking for relevant studies that are published outside your own geographical area. Also, balance self-citations and citations to collaborators’ works with citations to the works of others. And, as the International Committee of Medical Journal Editors (ICMJE) states in their ethical guidelines: “References should not be used […] to promote self-interests”.

Distinguish fact from speculation

A more insidious issue with citation practice relates to how claims in a particular published work are treated by subsequent studies. Consider, again, the spinach study. The myth of a decimal-point error can be traced to one scientist hypothesizing about this possibility. However, when this hypothesis was referred to in the literature, it was turned into a fact, a phenomenon called citation transmutation. Untested or incorrect claims can also attain the status of facts by citing papers that don’t actually contain the information (dead-end citations) or by not clearly marking citations to non-peer-reviewed material, such as conference abstracts, making assumptions seem more substantiated (back-door citations).

Our advice in this regard is to clearly and accurately distinguish between hypotheses, facts and speculation, and to describe the extent to which claims are backed up by (peer-reviewed) evidence.

Read the original source

The ICMJE urges authors to use primary sources containing the original data whenever possible, which echoes what we advise the researchers we work with. It is always worth going back to the original sources of claims – rather than relying solely on descriptions in secondary sources – especially when claims appear to be exaggerated, unexpected or incredible. It was the reliance on secondary information coupled with the lack of healthy scientific scepticism that fuelled the decimal-point myth in the spinach story. An amplification effect can also result in the broad acceptance of a false scientific belief, as Greenberg found in his citation network study. Amplification describes the entrenchment of an unfounded claim by a cascade of authors citing a supportive secondary source (often a review paper) instead of the primary sources. This can lead to an isolation of the critical data, and it makes it difficult to later investigate whether the claims are correct.

To sum up, read the original literature, be honest and balanced in your citations, and clearly separate fact from speculation to ensure that you don’t start the next ‘spinach myth’!

1 The iron content in spinach is similar to other green vegetables and inaccurate scientific methods appear to be the reason for the high value reported by one scientist in the late 19th century.
2 Other examples include the assumed protective factors of breastfeeding on obesity, the role of brain iron, zinc and copper in the Alzheimer’s disease, the relation between a serotonin transporter gene, life stress and depression, and the effectiveness of radiofrequency catheter ablation for atrial fibrillation.