EBMS Tickets

It has come to my attention that the Related Articles feature is not identifying and automatically linking errata to citations due to the timing of their publication. Unlike comments, very few errata are published at the same time as the base article. A mistake in an article must first be noticed and then the author/publisher must be contacted and then the errata is published. I have seen errata that are published a year or more after the base article is published. The Related Article feature is also limited to core journals only which is not an appropriate limit when it comes to errata.

Historically, Minaxi and I did not worry too much about errata because pubmed, for well over a decade, has linked errata to the base citation record. Users of the EBMS, CDR and cancer.gov have access to the PMID link to pubmed where errata are available for review with most having a link to view the full text. However, with the increasing urgency to push articles to be available ahead of print, we are seeing more errata even in the core journals that pride themselves for their vigorous peer review and editorial processes. Also due to NLM’s recent move to 100% automatic indexing, we are not catching errata in our monthly searches. Automatic indexing works from the bibliographic info, title, abstract, and author supplied keywords only, limiting the amount of information needed to properly index a citation. For errata which often have no abstract or keywords provided and have titles like “Department of Error” or “Erratum”, little to no indexing is assigned which means our searches will never retrieve it.

So moving forward, we need a way to automatically identify and link errata to EBMS citations, both new and existing but maybe not all citations. Many citations in the EBMS get eliminated by NOT journal filters, or rejected early on in the review process and errata for such citations may not need to be evaluated. We may want to consider limiting this new feature with a designated citation state to reduce unnecessary work. EBMS users would also need a means to track errata evaluation and errata information should be included in the citations full record as well as searchable in the database. Also as mentioned above, errata can be published any time after the base article is published and citations can have more than one erratum published at different times so timing will be an important aspect of handling errata.

Timing is also a factor in whether the full text pdf of the base citation in the EBMS reflects the current corrected state of the publication. If Bonnie uploads the pdf before an erratum for that base article has been published, then the pdf is not going to reflect the correction reported in the errata and the pdf will need to be replaced.

Results of my research on errata:

An ad hoc report was run in the CDR (10/26/23) to generate a list of all the PMIDs for all citations currently linked to at least one summary. I then used these PMIDs to search pubmed in combination with the “haserratumin” limit and identified which of the linked citations had errata.

Adult – 405 citations with erratum

Peds – 239 citations with erratum

Genetics – 183 citations with erratum

Screening and Prevention – 128 citations with erratum

Supportive care – 97 citations with erratum

IACT - 35 citations with erratum

See attached spreadsheet. Note: the pmids listed for each board in their corresponding errata tabs are for the base citation not the errata itself. To see the erratum, go to the base citation record in pubmed and click on the erratum link.

After searching the EBMS for errata, I can safely say that the majority of these errata identified do not have records in the ebms. The errata records that we have in the EBMS were mostly added and manually linked to the base article by myself or Bonnie at some point over the years. There are a few errata that did come through via the Related Articles feature because they were already published by the time the citation was imported.

Not all errata are going to be significant enough to impact the content in the summary. Errata report corrections to any part of the article. Mistakes made in the author affiliation or acknowledgements, for example, will probably have zero impact whereas mistakes in the results section, tables or conclusions could have significant impact. But there is no way to know for sure without reviewing the errata individually which may also require the user to review the content in the summary being referenced as well. To get an idea of the LOE and types of corrections being reported in errata, we conducted a pilot study of 60 adult treatment citations. All 60 were citations currently linked to at least one adult treatment summary and had at least one erratum. Jeff reviewed the errata for each of the base citations and recorded the type and location of the correction as well as giving a guess as to whether or not the errata may have an impact on the summary. Ning and I then reviewed his results and highlighted 24 of the 60 that we thought were significant enough to warrant further review. This pilot project shows that it is likely that 50% or more of the errata can be evaluated and identified as not significant without needing to review the summary content being referenced. It also shows that there are plenty of errata that are possibly significant and should be evaluated. Jeff also noted if the EBMS had the most recent corrected version of the pdf and for several citations it did not. See attached spreadsheet for more details for the pilot study.

Discussion in EBMS meeting:

All of the above was presented to Robin J, Victoria, William, Bonnie and other board managers in the Monthly EBMS meeting on November 21, 2023. It was agreed that this issue was substantial enough to require that some changes be made with the handling of errata in the future and that further discussion would be necessary. It was suggested in the meeting that perhaps we could have an errata report that could be run periodically to identify any new errata for citations in the EBMS with selected states. It was also suggested that maybe we could use existing features such as Tags to tag identified errata and use the Tag Comment to add notes regarding the errata evaluation. Having a flag (or equivalent) to temporarily identify new unevaluated errata that could be unflagged after evaluation was also suggested.

It was also decided in the meeting that we would focus efforts on moving forward ** with identifying new errata in the EBMS because it has been determined that an evaluation of all the errata identified to date (specific numbers per board listed above) would take years. It was suggested that retro errata evaluation could be incorporated into the comprehensive review process and handled on a summary by summary basis which could be facilitated by new or adhoc CDR reports to limit the number of errata to a specific summary or summary section. It was also suggested that if we were to consider evaluating the errata identified to date that this effort could be focused on maybe the past year or two only. Also each board manager may want to make their own decision as some boards had much fewer errata identified than others.  No specific decisions were made in the meeting regarding the evaluation of errata identified to date and will need to be discussed further at a later date.

Board managers decided to discuss the errata issue further and continue brainstorming ideas of how best to handle them in the EBMS.