Issue Number | 432 |
---|---|
Summary | Unable to Import Citations on PROD |
Created | 2017-04-19 11:14:02 |
Issue Type | Bug |
Submitted By | Juthe, Robin (NIH/NCI) [E] |
Assigned To | Kline, Bob (NIH/NCI) [C] |
Status | Closed |
Resolved | 2017-04-19 13:55:54 |
Resolution | Fixed |
Path | /home/bkline/backups/jira/oceebms/issue.206750 |
We seem to be currently unable to import citations on PROD using the fast-track option. I'm not sure if this applies to batch imports that Cynthia and Minaxi do. I am attaching a screenshot of the error message ("Secure Connection Failed") that is displayed. Other work in the EBMS seems to be going OK.
What happened when she clicked "Learn more"? Is this repeatable? Can you reproduce it on any of the lower tiers? Can you provide the exact sequence of steps leading up to the failure?
Thanks.
Added Amy as a watcher, so she'll be aware that I'm looking into this. If fact, probably a good idea to keep her in the loop like this for any tasks which aren't part of my currently assigned project.
Thanks.
I just confirmed that the production EBMS server can communicate with PubMed's server and to write to its own database (by running the Journal Table Refresh command, which does both).
This is the page that was displayed when she clicked "Learn More": https://support.mozilla.org/en-US/kb/what-does-your-connection-is-not-secure-mean
Bonnie reported that she received a "Page Cannot be Displayed" error in her browser - I'm guessing that was IE, but I will confirm. She has attempted to import several fast-track citations this morning and keeps getting the same error message (I have also tried and failed), so it is reproducible.
To reproduce the problem, here's an example.
1. Go to Import Citations page.
2. Select "Cancer Genetics" as the Board.
3. Select "Genetics of Breast and Ovarian Cancer" as the topic.
4. Enter the following PMID: 28398847
5. Check "Fast Track".
6. Enter the following placement level: Passed abstract review.
I was able to reproduce the failure on PROD, but the problem cannot be reproduced on the lower tiers. I will open a ticket with CBIIT to see if they have any log information which might be useful. It does not appear that any imports have been made at all today. Can you check with the librarians to see if they have tried and been unsuccessful? A couple more questions: do you know when you were last able to run a successful fast track import? And are you able to upload files with a substantial size (we'll want to determine whether the file system is full)?
Thanks.
Entered CBIIT ticket.
I asked Minaxi about any imports and this was her response. I have not asked her to create a small file to test with, but we can if you think that would help.
"Hi Robin,
I have not imported any batch citations this morning and I will not
have any batch that needs to be imported before last week of
April.
If there is a need to experiment, then I will have to generate a small
text file and import it. Please let me know if you want me to do
that.
Thanks,
Minaxi"
As for your other questions, it appears that the last successful citation import happened yesterday. There were four batches of 1-8 citations. Journal articles were uploaded with each of those, but I don't know how big they were. Bonnie also uploaded a few summaries on Monday.
~duganal: I set the priority of the ticket for CBIIT to the highest available (not the same of the "Impact" field, which I interpret to be something like the number of users affected). Can you use your channels to see if we can get someone assigned to look at this?
Thanks!
I wouldn't want her to skew any numbers by importing things she wouldn't normally import, and I'm even reluctant to have her do anything which would alter the normal distribution of activity over time. Perhaps a path which would do what we want here without any of those undesirable side effects would be to try an import the article you're trying to bring in without checking the "Fast Track" option, and instead move it through the states as you normally would for an article imported by the librarians. Would that work? There would be two methods for doing this:
enter the PMID
perform a PubMed search which narrows the results to this one article (using the PMID in the search criteria) and posting the MEDLINE format for results page (I am attaching this reults page to this ticket)
Ideally, if one method failed we'd try the other one.
Looking at the MEDLINE results page for the article, I do see one odd thing. Under the block for each author the AD field (Affiliation) lists the affiliations for every author, so this information is repeated over and over again, significantly bloating the record. As weird as that is, it's hard to see how that could be related to this failure. I bet if we looked at other articles which were successfully imported, we'll see the same oddness.
Just to clarify, there isn't anything special about the article I selected in the instructions to reproduce the problem. There have been several articles that we've tried (unsuccessfully) to import this morning. I just searched PubMed for a reasonable candidate for us to use for testing and that's where this one came from. It isn't the one that originally triggered the problem.
I'll try doing step 1 as you suggest and then ask the librarians to try step 2 (since I've never done that) if I'm unsuccessful.
In fact, that was a really bad example since I just discovered it's already in there! However, the good news is that when I tried to import it (without checking fast track), I received the usual "statistics" information indicating that it was a duplicate. I didn't get this the last time. Let me try again with a new-to-the-system citation.
~juther and ~bkline, I corresponded with Kiran on Jabber. He believes he fixed the issue and is asking us to check again. I'll follow up to try to find out why this happened/what fixed it. Please let me know whether everything is working as expected after checking again. Thanks!
I just successfully imported PMID: 28301458 with the "fast track" tag. Maybe we're out of the woods?
https://ebms.nci.nih.gov/citations/full/495454
We'll try a few more before we claim "victory"...
If he fixed it that fast, my money's on disk space problems.
Is it ok for me to tell Kiran that things look good now? He's standing by for me.
Sure as long as he tells us what he did to fix it. :-)
FYI, from Kiran re: the issue and the fix
not disk space
1:16 PMsystems team copied a OS patch related to nss-sysinit-3.27.1 to the server to get ready for upcoming production patching on this Saturday; but it negatively impacted our apache; Apache restart fixed the issue;
1:18 PMwe are doing analysis on how we can prevent this in the future
Hmm. I wonder what "negatively impacted" means. It would be nice if they tested these patches on stage before applying them to production.
I assume "not disk space" is shorthand for "[the problem was] not [connected with the amount of available] disk space" (as opposed to "[there was] not [sufficient] disk space"). :-)
I asked that question...and was told that this didn't happen on the lower tiers when the patch was added because the lower tiers were restarted soon after the patches were pushed. I was also told for the prod update: this is kind of preparation step to reduce the total maintenance window. I suggested that this would be a good lesson learned for the systems team.
This is working well - we've imported several citations. Thank you!!
I suggested that this would be a good lesson learned for the systems team.
Good suggestion. I hope they take it seriously. :-)
You can close this ticket if the problems haven't recurred.
File Name | Posted | User |
---|---|---|
EBMS Import Citation Error Message.docx | 2017-04-19 11:14:57 | Juthe, Robin (NIH/NCI) [E] |
Elapsed: 0:00:00.000744