CDR Tickets

Issue Number 3483
Summary [Summaries] Shorter URLS Needed For Successful Conversion of QC Reports into Word
Created 2012-03-06 15:36:10
Issue Type Improvement
Submitted By Juthe, Robin (NIH/NCI) [E]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2012-07-26 17:37:20
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.107811
Description

BZISSUE::5178
BZDATETIME::2012-03-06 15:36:10
BZCREATOR::Robin Juthe
BZASSIGNEE::Volker Englisch
BZQACONTACT::William Osei-Poku

Sharon’s Board is adding a lot of images to their summaries, and she likes to run QC reports that display these images when she shares them with Board members. However, when she converts a QC report with 6 images to Word (example: Late Effects of Cancer Treatment), the images are lost (they become boxes with X marks in them).

Comment from Volker:

We found out what the problem is with Sharon's summary: The URL is too long.
I talked to Bob about this and we're thinking of implementing some sort of 'redirection' that allows us to present a much smaller URL. This problem already exists in IE for several years and it's unlikely to be resolved by Microsoft soon.

Comment entered 2012-03-06 19:16:03 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-03-06 19:16:03
BZCOMMENTOR::Volker Englisch
BZCOMMENT::1

I thought we had agreed to assign everything to Alan while he's on vacation???

I played around with the service at http://www.tinyurl.com. This service allows you to enter a long URL and get something very short back such as
http://tinyurl.com/6mobcym

This would work to create the QC report, unfortunately, the QC report is then created and displayed with the long URL again and therefore this idea wouldn't work.

Comment entered 2012-05-03 11:29:29 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-03 11:29:29
BZCOMMENTOR::Volker Englisch
BZCOMMENT::2

Is this still a problem?
I'm not able to recreate the problem at the moment provided that MS-Word was open when the conversion is initiated.

Comment entered 2012-05-03 12:03:56 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-05-03 12:03:56
BZCOMMENTOR::Robin Juthe
BZCOMMENT::3

(In reply to comment #2)
> Is this still a problem?
> I'm not able to recreate the problem at the moment provided that MS-Word was
> open when the conversion is initiated.

Sharon is still having problems with The Late Effects of Cancer Treatment summary Redline/Strikeout QC Report selecting for Editorial Board markup, approved attribute, proposed attribute, external comments, display HP reference section, and display images. (on BACH)

Comment entered 2012-05-09 18:23:01 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-09 18:23:01
BZCOMMENTOR::Volker Englisch
BZCOMMENT::4

After, oh so many supermoons, I have been able to implement the TinyCdrUrls version which will eventually allow us to convert a QC report to Word without problems. It will even work to convert to Word without having to open Word first.

The implementation, though, is not exactly how I wanted it to work and Bob might still try to convince me to change what I have done but for now it works the following way:

  • Sharon clicks the button to run a QC report from XMetaL

  • The page to select the options comes up and Sharon checks many of those

  • The options are written to a database table named url_parm_set

  • Now, instead of calling the program QcReports.py again to print the report
    I am redirecting the report to run QCforWord.py passing the ID of the
    parameter set saved to the url_parm_set table

  • QCforWord.py will read the parameter ID, select the parameter set from
    the database table and run and display the QC report with a URL like
    ...QCforWord.py?DocId=CDR0000062902&DocType=Summary:rs&parmstring=yes&parmid=12
    instead of the original URL
    ...QcReport.py?Request=Submit&Session=guest&DocType=Summary&DocId=CDR0000062902&ReportType=rs&DocVersion=-1&Editorial-board=on&approved=on&external=on&AudExternalComments=on&SrcEditorComments=on&DurPermanentComments=on&DurRegularComments=on&CitationsHP=on

Comment entered 2012-05-23 14:35:52 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-23 14:35:52
BZCOMMENTOR::Volker Englisch
BZCOMMENT::5

This has been set up now and is ready for user testing with changes to the following programs:
QcReport.py
cdr.py
cdrcgi.py
QCforWord.py

I'd suggest for Sharon, who had reported the problems originally, to test this on MAHLER to identify that everything is displaying according to the selected options.
After that maybe William can test this.

Please note that the change can potentially affect every single QC report. I would therefore suggest to expand testing to more than just running a RS/BU report. Please make sure to test with HP, patient, EN, ES documents, run PP and a few non-summary QC reports just to be safe. If a document fails, please identify the CDR-ID of the document and the report that failed.

Comment entered 2012-05-23 17:43:49 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-05-23 17:43:49
BZCOMMENTOR::Robin Juthe
BZCOMMENT::6

(In reply to comment #5)
> This has been set up now and is ready for user testing with changes to the
> following programs:
> QcReport.py
> cdr.py
> cdrcgi.py
> QCforWord.py
> I'd suggest for Sharon, who had reported the problems originally, to test this
> on MAHLER to identify that everything is displaying according to the selected
> options.
> After that maybe William can test this.
> Please note that the change can potentially affect every single QC report. I
> would therefore suggest to expand testing to more than just running a RS/BU
> report. Please make sure to test with HP, patient, EN, ES documents, run PP
> and a few non-summary QC reports just to be safe. If a document fails, please
> identify the CDR-ID of the document and the report that failed.

I tested this with the Merkel Cell summary on Mahler and none of the images displayed once I converted it to Word. I tested the following summary-QC report combinations:

CDR 441548 - Patient RLSO QC Report
CDR 441548 - Patient B/U QC Report
CDR 62884 - RLSO QC Report
CDR 62884 - B/U QC Report

I also noticed that summary fragment refs are displaying with a number of CDR IDs around the link. For example, the following text appears in the QC report for CDR 62884:

"Refer to CDR0000062884 || CDR0000062884 || CDR0000696926Table 3 (Module > Main) for more information."

Comment entered 2012-05-24 11:35:57 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-24 11:35:57
BZCOMMENTOR::Volker Englisch
BZCOMMENT::7

I've tried the same summaries and the converted Word document has those images included for me. I'll have to work with Robin to identify what's different between both of our approaches.

As for the additional CDR-IDs that are displayed: Those are debugging messages I included for testing the Summary Module changes.
I'll remove those shortly.

Comment entered 2012-05-24 14:14:49 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-05-24 14:14:49
BZCOMMENTOR::Robin Juthe
BZCOMMENT::8

(In reply to comment #7)
> I've tried the same summaries and the converted Word document has those images
> included for me. I'll have to work with Robin to identify what's different
> between both of our approaches.
> As for the additional CDR-IDs that are displayed: Those are debugging messages
> I included for testing the Summary Module changes.
> I'll remove those shortly.

As I mentioned in our meeting, the images ARE working - I was too impatient! They come up after 5 seconds or so. Sorry.

I will ask Sharon to take a look at her summary on Mahler since she reported this problem, but I think it would be best to wait until Mahler is refreshed (since we just agreed to do that). Then she can run the report for the same summary she had experienced problems with on Bach. (The version on Mahler doesn't have any images.)

Once we confirm that that is working properly, then we will need to do more extensive testing as you described in comment 5.

Comment entered 2012-06-06 10:06:14 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-06 10:06:14
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::9

I am getting the following error messages while running QC reports on Mahler:

"Failure inserting parms: Invalid object name 'url_parm_set'.
Content-type: text/html"

"Something went wrong"

It seems to happening all summary types.

Comment entered 2012-06-06 10:43:23 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-06 10:43:23
BZCOMMENTOR::Volker Englisch
BZCOMMENT::10

Yes, I'm aware of this error. It's related with the refresh of MAHLER and I haven't had time to create the new table that's used for the tiny URL changes.
I will create the table sometime today.

Comment entered 2012-06-12 13:49:10 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-12 13:49:10
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

This is working again on MAHLER and is ready for testing.

Comment entered 2012-06-14 10:52:00 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-14 10:52:00
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::12

(In reply to comment #11)
> This is working again on MAHLER and is ready for testing.

We've reviewed all the QC reports and everything seems to be working well except that I am not able to see the images when I convert the QC report to Word. It opens up without having to first open word. However, I see only placeholders for the images.
Also, pub preview doesn't seem to be working on Mahler. It comes up with python script errors for most of the longer summaries.

(In reply to comment #11)
> This is working again on MAHLER and is ready for testing.

Pub preview is coming up with Python script errors for relatively longer summaries.

Comment entered 2012-06-14 11:01:26 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-14 11:01:26
BZCOMMENTOR::Volker Englisch
BZCOMMENT::13

(In reply to comment #12)
> However, I see only placeholders for the images.

Did you wait long enough for the images to be loaded? Robin had reported at some point that the images weren't displayed but it took a little while for all of the images to be loaded. I'm wondering if this is the same problem which I would not have any control over.

> Also, pub preview doesn't seem to be working on Mahler.

This has nothing to do with this issue, right? I'm not saying that it's not a problem for PP not to work but you wouldn't convert the PP output to Word, would you?

Comment entered 2012-06-14 11:13:05 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-06-14 11:13:05
BZCOMMENTOR::Robin Juthe
BZCOMMENT::14

(In reply to comment #13)
> (In reply to comment #12)
> > However, I see only placeholders for the images.
> Did you wait long enough for the images to be loaded? Robin had reported at
> some point that the images weren't displayed but it took a little while for all
> of the images to be loaded. I'm wondering if this is the same problem which I
> would not have any control over.

I waited a few minutes for the images to come up in the B/U QC report of the Late Effects summary (CDR343584), and all of them came up but the first one. Also, everything looked good when I converted it to word (all images showed up except for the first one). I am asking everyone to test a few documents on our end, too.

Comment entered 2012-06-14 17:16:52 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-14 17:16:52
BZCOMMENTOR::Volker Englisch
BZCOMMENT::15

I've updated the two files
tables.sql
CreateLogins.sql
to make sure the table can be recreated and the roles are set after a refresh.

I also created the table on FRANCK.

Comment entered 2012-06-20 16:05:36 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-06-20 16:05:36
BZCOMMENTOR::Robin Juthe
BZCOMMENT::16

(In reply to comment #15)
> I've updated the two files
> tables.sql
> CreateLogins.sql
> to make sure the table can be recreated and the roles are set after a refresh.
> I also created the table on FRANCK.

I have tested this with a number of documents on MAHLER and it's working well. Please promote to BACH.

Comment entered 2012-06-21 15:10:32 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-21 15:10:32
BZCOMMENTOR::Volker Englisch
BZCOMMENT::17

The following files have been copied to FRANCK and BACH and the table has been created on BACH:
QCforWord.py - R10449
QcReport.py - R10449
cdr.py - R10448
cdrcgi.py - R10448
tables.sql - R10450
CreateLogins.sql - R10450

Please verify on BACH and close this bug.

Comment entered 2012-06-29 10:10:32 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-06-29 10:10:32
BZCOMMENTOR::Robin Juthe
BZCOMMENT::18

Sharon is experiencing a problem with the Childhood ALL HP TEMP summary (CDR 716080) whereby some of the numbered and lettered lists are changing their appearance when the summary is converted to Word. For example, an outline that contains letters a, b, and c is changing to d, e, and f in MS Word. It's not clear whether this is related to the shorter URL fix or just a Word problem in general. However, she needs to be able to generate a correct version for her Board meeting coming up in a couple of weeks.

Sharon is also not able to view images. William said that the CIAT staff are also unable to view images when invoking the new shorter URL fix. Could that be specific to the user's computer, since Volker and I are both able to view the images?

I'm raising this priority to P4.

Comment entered 2012-06-29 12:40:33 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-29 12:40:33
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::19

(In reply to comment #18)
> Sharon is also not able to view images. William said that the CIAT staff are
> also unable to view images when invoking the new shorter URL fix. Could that be
> specific to the user's computer, since Volker and I are both able to view the
> images?
>

I tried it on one of our loaner laptops this morning and it is working without any problems on that machine. I will investigate further to determine why it is not working on other machines. It appears to have something do with MS Word feature that is either not installed or enabled on some computers.

Comment entered 2012-07-05 17:59:33 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-05 17:59:33
BZCOMMENTOR::Volker Englisch
BZCOMMENT::20

(In reply to comment #18)
> Sharon is experiencing a problem with the Childhood ALL HP TEMP summary (CDR
> 716080) whereby some of the numbered and lettered lists are changing their
> appearance when the summary is converted to Word. For example, an outline that
> contains letters a, b, and c is changing to d, e, and f in MS Word.

I just want to make sure: Is this a problem that has started since the shorter URL code has been moved to production or have we only now noticed this problem?

I did a quick test with Sharon's summary and I see the same problem with the numbering on my machine. I've also tried to use the copy/paste technique and identified that both versions in Word (copy/paster and convert) display the same numbering issue.

At this point I am fairly certain that the problem is related to the Word conversion and has nothing to do with the QC report itself. I will have to investigate how Word identifies a list and what triggers such list to be numbered starting at 1, A, I or such and what would trigger a list counter to be continued from the previous list.
For instance, the first time I see this problem in Sharon's summary the numbering goes like this:
1.
2. Title

  • Bullet item one

  • Bullet item two

Some text in a paragraph.

c.
d.
e.
3.

The list number "c." is wrong because it should be "a" according to the user. However, in the context of the previous bulleted list it is correctly numbered because it's the third list item on this list level which is displayed as a letter instead of a bullet.
Word by default can handle 9 levels of lists. The level of a list is defined by the indentation of the text. Word provides styles and allows to have styles for those levels created manually but it obviously cannot know from the indentation of the text alone what list level needs to be applied.

In this particular summary Sharon is trying to create sections with a section-level TOC by using lists. My suggestion is that if the user intends to create a document layout for a specific application the document should be written with that application in mind when the summary is created and not as an afterthought.
In my opinion, the solution to the problem is to manually adjust the list numbering in Word after the conversion finished or - as Margaret hinted - to use sections and sub-sections to format the text. I do not believe that it will be possible to included directives for MS-Word in our HTML QC output to ensure these lists are numbered correctly according to the user's intention. The summary documents are most likely too complicated to be handled correctly by the Word conversion in all cases.

Comment entered 2012-07-05 18:04:00 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-05 18:04:00
BZCOMMENTOR::Volker Englisch
BZCOMMENT::21

(In reply to comment #20)
> In my opinion, the solution to the problem is to manually adjust the list
> numbering in Word after the conversion finished

I forgot to mention how to do this:

  • Right-click on the number that's wrong

  • Select "Set Numbering Value"

  • Set the value of the "Set value to:" field

  • Press OK

Comment entered 2012-07-20 16:44:24 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-20 16:44:24
BZCOMMENTOR::Volker Englisch
BZCOMMENT::22

Margaret is going to talk to Sharon because this "feature" is created as part of the conversion of HTML to MS-Word.

There is no simple fix that could be implemented as part of the CDR to avoid this particular problem.

Comment entered 2012-07-26 17:37:20 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-26 17:37:20
BZCOMMENTOR::Volker Englisch
BZCOMMENT::23

As discussed at today's status meeting we're closing this issue.
The implementation to create shorter URLs in order to convert the QC reports to Word is working. The problem with the itemized list numbering that Sharon is experiencing, however, is a Word problem and can't be fixed in the CDR.

Elapsed: 0:00:00.000421