CDR Tickets

Issue Number 3445
Summary [CiteMS] importing error
Created 2011-11-03 16:17:23
Issue Type Improvement
Submitted By Osei-Poku, William (NIH/NCI) [C]
Assigned To alan
Status Closed
Resolved 2011-11-09 16:45:40
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.107773
Description

BZISSUE::5139
BZDATETIME::2011-11-03 16:17:23
BZCREATOR::William Osei-Poku
BZASSIGNEE::Alan Meyer
BZQACONTACT::Minaxi Trivedi

Please see the error message in attachment1 - "CiteMS_import error 11-3-11.doc". CiteMs returned the error when Minaxi attempted to import a file that contained the attached citation "CAM_overview_problemCitation.txt".

Attachment2 to follow shortly…

Comment entered 2011-11-03 16:17:23 by Osei-Poku, William (NIH/NCI) [C]

Attachment CiteMS_import error 11-3-11.doc has been added with description: CiteMS error

Comment entered 2011-11-03 16:18:45 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2011-11-03 16:18:45
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::1

attachement2 added

Comment entered 2011-11-03 16:18:45 by Osei-Poku, William (NIH/NCI) [C]

Attachment CAM_overview_problemCitation.txt has been added with description: problem citation

Comment entered 2011-11-03 16:45:16 by alan

BZDATETIME::2011-11-03 16:45:16
BZCOMMENTOR::Alan Meyer
BZCOMMENT::2

I'll look at it this evening.

Comment entered 2011-11-03 22:15:13 by alan

BZDATETIME::2011-11-03 22:15:13
BZCOMMENTOR::Alan Meyer
BZCOMMENT::3

There is a bug in the import program. The program scans a line looking for tags at the beginning of each line, e.g., "STAT-" or "TI -". If it finds one, it records that it has seen this tag, chops the tag off the line, and processes the data portion of the line (i.e. the text after the tag) - all of which is fine and correct.

Now comes the bug. After chopping off the tag and processing the text it then repeats the tag scan. It has already found the tag for this line and I would think it should not be looking for a tag at the beginning of the data. But alas, it does.

In this particular record, the program encounters a "STAT-" tag, processes it and continues on. Later it finds the "TI " tag for the title of the citation. It chops off the "TI ", processes the text and then rescans the line for tags. By unfortunate chance it finds a match at the beginning of the line because the text of the title begins like this: "STAT-3 inhibitors..." It gets a match on "STAT". Worse luck, or maybe good luck because I'm not sure what it would have done if this didn't get caught and aborted by the check I put in the code some time ago, we've actually already seen a "STAT" tag and only one is allowed in a record.

So, I see what's going wrong and why and wanted to report it.

Before I attempt a fix I have to figure out why the original programmer rescanned the line. I can't see any obvious reason. It looks like a plain old bug (of a kind that I make all the time - I'm not throwing stones here.) But I don't want change it yet without trying hard to figure out if a rescan ever should be done. I don't want to fix this problem and break something else.

I'll do more research and report back when I've decided what to do.

Minaxi:

If you can hold off entering the record, that would be good. If you need to enter it and I haven't fixed the problem yet, I believe you can get this record into the system by changing the title to something like "STAT3" or "STAT 3" OR "STAT 3" or "Stat-3" or anything that doesn't begin with "STAT". I hate to mangle good data to work around a bug but it's an option here in an emergency.

Comment entered 2011-11-04 00:05:48 by alan

BZDATETIME::2011-11-04 00:05:48
BZCOMMENTOR::Alan Meyer
BZCOMMENT::4

I read the code a number of times but couldn't find any way that it looked right. I believe that what I found has to be a bug.

I fixed it and compiled a new executable which is attached here. To install it do the following:

Exit the import utility if you are in it.
Go to the directory with your existing import utility.
Rename the existing utility, e.g. to Cips_CMS_2011-11-03.exe
Download the new executable from this attachment into the same directory.

I tested by successfully processing the file that failed and by importing another file successfully.

In order to make your testing more realistic, I backed up the production database and installed it on the development/test system. That system is now current with the production data as of today (end of Thursday, 2011-11-03.) To test in the development/test system, choose "Test" on the login radio buttons and prefix your password with "test".

If you think everything is okay, you're ready to run in production. If you think there are problems, delete or rename the new exe (you can re-download it if you need to) and rename or copy the old one back to Cips_CMS.exe.

Comment entered 2011-11-04 00:05:48 by alan

Attachment Cips_CMS.exe has been added with description: Executable import utility with fix for bug that scans data in line for tags

Comment entered 2011-11-04 00:06:46 by alan

BZDATETIME::2011-11-04 00:06:46
BZCOMMENTOR::Alan Meyer
BZCOMMENT::5

I'm marking this resolved-fixed.

Comment entered 2011-11-04 09:42:19 by priced

BZDATETIME::2011-11-04 09:42:19
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::6

(In reply to comment #5)
> I'm marking this resolved-fixed.

I have successfully imported the problem citation. Thanks!

Minaxi

Comment entered 2011-11-09 16:45:40 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2011-11-09 16:45:40
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::7

The problem is resolved. Closing bug. Thank you!

Attachments
File Name Posted User
CAM_overview_problemCitation.txt 2011-11-03 16:18:45 Osei-Poku, William (NIH/NCI) [C]
Cips_CMS.exe 2011-11-04 00:05:48
CiteMS_import error 11-3-11.doc 2011-11-03 16:17:23 Osei-Poku, William (NIH/NCI) [C]

Elapsed: 0:00:00.001646