EBMS Tickets

Issue Number 87
Summary [Import Citations] Programmatic sweep to updated PubMed info
Created 2013-10-31 10:22:43
Issue Type Improvement
Submitted By Juthe, Robin (NIH/NCI) [E]
Assigned To Kline, Bob (NIH/NCI) [C]
Status Closed
Resolved 2013-12-31 17:15:48
Resolution Fixed
Path /home/bkline/backups/jira/oceebms/issue.114400
Description

We'd like to discuss the possibility of implementing a programmatic sweep (weekly? monthly?) to check PubMed and update our citation records with any changes. This is similar to a function that we have in the CDR.

Comment entered 2013-12-27 17:37:22 by Kline, Bob (NIH/NCI) [C]

This is at least worth 40 story points. I've implemented a command which is accessible to site administrators, to refresh a large batch of articles. Can't check everything all at once, as that would time out the connection to the web server. I'll investigate methods of hooking this up to a scheduler.

Comment entered 2013-12-31 14:20:55 by Kline, Bob (NIH/NCI) [C]

This turned out to be closer to a 60 than a 40. I have done the implementation and testing on DEV.

  • R12222 /trunk/ebms/scheduled/update-pubmed-data.py

  • R12220 /trunk/ebms/ebms.nci.nih.gov/modules/custom/ebms/ebms.module

  • R12220 /trunk/ebms/ebms.nci.nih.gov/modules/custom/ebms/import.inc

Comment entered 2014-01-06 15:50:11 by Kline, Bob (NIH/NCI) [C]

The software is on QA, though the cron job still hasn't been installed. I put in a ticket for CBIIT to install the cron job on DEV, and that still hasn't happened yet, either. It doesn't really make any difference if the cron job is installed on the lower tiers, as long as the supporting software is present, so we can refresh the XML in bulk manually. In fact, even if CBIIT never gets around to installing the cron job on the production tier, we'll still have the basic functionality. To test, find an old article which has been updated by NLM since we imported it (but longer ago than last week), and verify that the current information is present.

Comment entered 2014-01-07 08:21:41 by Kline, Bob (NIH/NCI) [C]

The cron job was installed on DEV last night, and I verified that it ran as scheduled this morning.

Comment entered 2014-01-13 09:34:16 by Kline, Bob (NIH/NCI) [C]

In response to https://tracker.nci.nih.gov/browse/OCEEBMS-152 I had to make some modifications to the Import section of the full citation page, to implement special treatment for the import jobs created for this ticket. Those jobs are displayed at the end of the section, below all the jobs associated with a single topic. I also modified the logic to suppress the jobs created by the nightly sweep when it was determined that a refresh of the XML was unnecessary because the document was unchanged.

Comment entered 2014-01-17 13:34:43 by Juthe, Robin (NIH/NCI) [E]

Hi Bob,
Could you please send me an example to test with for this issue?
Thanks,
Robin

Comment entered 2014-01-17 14:00:20 by Kline, Bob (NIH/NCI) [C]

I'm working on it.

Comment entered 2014-01-17 16:41:42 by Kline, Bob (NIH/NCI) [C]

You can compare EBMS 19131 on QA and PROD, and see that author KH Kim was changed to Ki Kim. However, I think it will be more appropriate for me to take responsibility for verifying the correctness of the software for this ticket.

Comment entered 2014-01-17 16:48:54 by Juthe, Robin (NIH/NCI) [E]

Thanks, Bob. I've verified that change. I'll mark this QA verified.

Elapsed: 0:00:00.000600