Introducing enhanced retraction detection, including matching against the Retraction Watch database

Edifix detects and warns when articles in the reference list have been retracted or withdrawn.

Because there are no scholarly publishing standards for display, notification, or metadata handling for retractions, Edifix uses multiple approaches to try to ensure that all retractions are detected and given appropriate warnings in Edifix results. NISO has established the CREC Working Group (formerly CORREC) to develop recommendations to address the issues of display, notification, and metadata of retractions.

→ Note: For the most complete retraction detection possible, we recommend running Edifix jobs with both PubMed and Crossref linking turned on.

“Classic” Edifix Retraction Detection

Originally Edifix detected retractions when metadata in PubMed or Crossref indicated the item was retracted. PubMed has a standard method in their metadata to indicate when a reference is retracted. Crossref retraction detection was done by examining the Crossref metadata for a Crossmark record indicating a retraction or withdrawal. These indications may result in warnings like this:

5. Xu YC, Liu X, Li M, et al. A Novel Mechanism of Doxorubicin Resistance and Tumorigenesis Mediated by MicroRNA-501-5p-Suppressed BLID.  Mol Ther Nucleic Acids. 2018;12:578-590. https://doi.org/10.1016/j.omtn.2018.06.011   PMID:30195794

CrossMark reports a retraction (or similarly important issue). The CrossMark type is “retraction”. Additional information can be found at https://doi.org/10.1016/j.omtn.2018.06.011. (Ref. 5 “Xu, Liu, Li, et al, 2018”)

PubMed reports that this reference was retracted in “Mol Ther Nucleic Acids. 2022 Nov 09;30:378”. (Ref. 5 “Xu, Liu, Li, et al, 2018”)

in cases when the retraction is correctly indicated in the metadata at both services. However, in other cases we might find this result:

3. Van Luit JE, Van der Molen MJ. The effectiveness of Korean number naming on insight into numbers in Dutch students with mild intellectual disabilities. Res Dev Disabil. 2011;32(5):1822–8. https://doi.org/10.1016/j.ridd.2011.03.014   PMID:21498042

PubMed reports that this reference was retracted in “Res Dev Disabil. 2011 Nov-Dec;32(6):3018”. (Ref. 3 “Van Luit, Van der Molen, 2011”)

if a reference did not have Crossmark metadata; or, if a reference did not have retraction metadata indexed by PubMed, this result:

61  Zhu X, Cao Y, Liu W, et al. Stereotactic body radiotherapy plus pembrolizumab and trametinib versus stereotactic body radiotherapy plus gemcitabine for locally recurrent pancreatic cancer after surgical resection: an open-label, randomised, controlled, phase 2 trial. Lancet Oncol 2021; 22: 1093–102. https://doi.org/10.1016/S1470-2045(21)00286-2   PMID:34237249

CrossMark reports a retraction (or similarly important issue). The CrossMark type is “retraction”. Additional information can be found at https://doi.org/10.1016/S1470-2045(22)00064-X. (Ref. 61 “Zhu, Cao, Liu, et al, 2021”)

→ Note: Classic Edifix retraction detection has also detected and warned on Expressions of Concern since 2019. These warnings are not counted as retractions in the retraction count metadata at the top of the exported JATS XML file.

Extended Edifix Retraction Detection

Not all publishers use Crossmark, many do not report retractions or the changed status of an article to PubMed and Crossref, and not all references have PubMed IDs or DOIs. Because of these gaps, Edifix was updated in April 2023 to detect retractions using two additional methods: identifying article titles that indicate a retraction and matching against the Retraction Watch database.

Updated article title indicates retraction

Many publishers who participate in Crossref do not participate in the Crossmark system to alert when there are updates to an article. In some cases, these publishers may deposit an updated title at Crossref to indicate that a reference is retracted. For example, the title of one article at Crossref has been updated to say:

RETRACTED ARTICLE: GDF11 enhances therapeutic functions of mesenchymal stem cells for angiogenesis.

In this case, Edifix now gives the following warning:

12. Zhang C, Lin Y, Zhang K, Meng L, Hu X, Chen J, et al. GDF11 enhances therapeutic functions of mesenchymal stem cells for angiogenesis. Stem Cell Res Ther. 2021 Aug;12(1):456. https://doi.org/10.1186/s13287-021-02519-y   PMID:34384486

The title of this item indexed at CrossRef starts with “RETRACTED ARTICLE:”, indicating that this item has probably been retracted. (Ref. 12 “Zhang, Lin, Zhang, Meng, Hu, Chen, et al., 2021”)

→ Note: This warning is provided only if there is no Crossmark metadata indicating a retraction.

Edifix will also give a similar warning if an updated title indicating a retraction is found at PubMed, although, similar to Crossref, only as a secondary check if there is no PubMed metadata indicating a retraction. While such cases are not infrequent at Crossref, we have not yet seen any similar cases of an updated title at PubMed.

Matching against the Retraction Watch database

The unfortunate reality of today’s world (2023) is that the number of retractions is increasing rapidly, there are no standards or industry best practices for reporting retraction information once an article has been retracted, and research integrity including detection of retracted articles is more important than ever.

Because of the lack of industry best practices and inconsistent reporting of retractions to PubMed and Crossref, Inera has licensed the Retraction Watch database and incorporated it into Edifix. This database, which is the single most comprehensive source of accurate retraction information, combined with the other methods listed above, helps Edifix provide you the most complete retraction information possible.

→ Note: Retraction Watch covers more than journal articles. Included in the database are retractions/withdrawals of preprints, book chapters, and conference papers. Edifix will also report retractions from these content types.

Retraction Watch matching with an article identifier

The most accurate way to achieve matches with the Retraction Watch database is via a PubMed ID (PMID) or a DOI. For this reason, we strongly recommend turning on both PubMed and Crossref linking to get the best quality Retraction Watch matches.

When one or more article identifiers are available, Edifix will look up the identifier(s) in the Retraction Watch database. If a match is made, a warning like this will be given:

40. Isacsson G, Reutfors J, Papadopoulos FC, Ösby U, Ahlner J. Antidepressant medication prevents suicide in depression. Acta Psychiatr Scand. 2010 Dec;122(6):454–60. https://doi.org/10.1111/j.1600-0447.2010.01561.x   PMID:20384599

Retraction Watch reports that this item was Retracted on March 10, 2012 because the PubMed ID and DOI in the reference match the Retraction Watch database. Retraction Watch gives the reason as: “+Error in Analyses;”. For more information, please see: http://retractionwatch.com/2012/03/19/author-retracts-paper-claiming-antidepressants-prevent-suicide/.

In this case, the warning indicates a match has been made on both the PMID and DOI, and the warning also provides the date of the retraction, the reason for the retraction from the Retraction Watch database, and (if available) one or more links to additional information about the retraction.

If a match is made only on a PMID or DOI, the text in the message will indicate which identifier was matched.

→ Note: Warnings from PubMed and Crossref, if available, will still be given with the reference. Edifix provides warnings based on information from all three services, as each service may provide unique information about the retraction.

Retraction Watch matching with title/author/year

If no match is made in the Retraction Watch database with a PMID or DOI, or if there is no article identifier available in the reference, a secondary check is made using the title, first author, and year in the reference with a fuzzy match look-up in the Retraction Watch database. When a match is made, a warning is inserted like this:

41. Morsi NM, Nabil Shamma R, Osama Eladawy N, Abdelkhalek AA. Bioactive injectable triple acting thermosensitive hydrogel enriched with nano-hydroxyapatite for bone regeneration: in-vitro characterization, Saos-2 cell line cell viability and osteogenic markers evaluation. Drug Dev Ind Pharm. 2019;45(5):787–804.

Retraction Watch reports that this item was Retracted on January 19, 2023 because the first author, title, and year in the reference match the Retraction Watch database. Retraction Watch gives the reason as: “+Concerns/Issues About Image;+Manipulation of Images;+Original Data not Provided;”. The original item can be found at 10.1080/03639045.2019.1572184

that indicates how the match was made (first author, title, year) and provides additional information from the Retraction Watch database including, if available and not in the reference, the DOI.

This secondary check is less reliable than a check based on article identifiers because authors often have typographical errors in the title of the reference. For that reason, we recommend running Edifix with PubMed and Crossref linking turned on whenever detection of retractions is important in your workflow. However, we provide this alternative method to detect cases that may not be found using article identifiers.

Further information

For more extensive information about the Retraction Watch database, we recommend reading the database user guide at: https://retractionwatch.com/retraction-watch-database-user-guide/

Appendix for API Users

Some users of the Edifix API may want to specifically filter on the message warning codes for retracted references. Please filter on these codes:

Warnings from PubMed and Crossref linking

Warnings from Retraction Watch checks