CDR Tickets

As discussed yesterday, I'm raising the priority of this issue for AFTER the release work is done (and Volker gets some sleep!).

Hi Robin, I'd like to roll this into the CDR "Mailer Hotfix" and assign to Alan to work on when he returns from vacation.

Volker spent some time with me to go over the issue and point out some
of the parts of the system that relate to the Compact="No" attribute.

It looks like there are five potential areas of change:

 1. Remove Compact="No" processing from the QC and publishing filters
    and any other filters that have it.

    This will eliminate outputs in the QC reports and maybe in other
    outputs (I didn't get so far as to see whether Volker has already
    eliminated this attribute from outputs to cancer.gov and
    licensees.)

    If the attribute is going out to licensees, we should alert them
    to the upcoming removal, but I would hope that the change would
    have minimal impact on them because (I think) the attribute is
    optional in all cases.

 2. Remove Compact="No" from the XMetal data entry templates.

    I found five templates that appear to be defaulting this attribute
    to "No" when a user creates an ItemizedList.  The templates are
    for these document types:

        DrugCombinationSummary
        DrugInformationSummary
        PatientCAMSummary
        PatientScreeningSummary
        PatientScreeningSummaryES

    Making this change will stop the system from automatically adding
    this attribute to new lists in documents.

 3. Global change all of the Summary documents to remove the Compact
    attribute.

    This is necessary before we can remove the attribute from the
    schemas.  Otherwise existing documents will become invalid.

 4. Remove the Compact attribute from the CdrCommonBase schema.

    This is the final step that would banish the Compact attribute for
    the future.

 5. Adjust the CSS to more closely resemble the CSS on cancer.gov.

    I've edited this comment to add the above fifth area of change in
    light of Volker's comment of 13 Jan 2015 11:02 AM.  See Volker's
    comment.

I would propose to work on them in the above order, at a very minimum
getting through items 1 and 2.  After that, the priority of the task
could be lowered if necessary.

I finished item #2 and updated the following templates to remove the Compact attribute:

• R13072: DrugCombinationSummary.xml

• R13072: DrugInformationSummary.xml

• R13072: PatientCAMSummary.xml

• R13072: PatientScreeningSummary.xml

• R13072: PatientScreeningSummaryES.xml

It looks like there are four potential areas of change:

There may be a fifth item to address: Adjusting the CSS used to resemble more closely the CSS on Cancer.gov but the users would need to identify how close they'd like it to be.

Volker's change to remove "Compact='No'" from the XMetal templates was checked in to Ampere and did get promoted to QA. This is entirely independent of the rest of the changes and is probably a Good Thing to do even though other changes will only be made later.

With the change, XMetal will no longer automatically add the Compact='No' attribute to ItemizedList elements.

It turns out that Bob, working under a different issue (OCECDR-3820), already did some work on item #3 in the above list of tasks. He wrote a program that can be found in branches/Ampere/DevTools/GlobalChange/strip-compact.py. I'm recording that here so that we won't forget that this was done.

In his usual systematic way, Bob did a search and discovered that many more document types than just the five Summaries types, can potentially include Compact='No' attributes. The global change has to search all of them. His list includes:

• Citation

• Documentation

• DocumentationToC

• DrugInformationSummary

• GlossaryTerm

• GlossaryTermConcept

• GlossaryTermName

• InScopeProtocol

• Media

• MiscellaneousDocument

• Organization

• OutOfScopeProtocol

• ScientificProtocolInfo

• Summary

• Term"

I'm working on item #1 (modify filters) and have a few remarks:

• adjusting the list items to use the look-and-feel of NVCG also modifies the table of contents (TOC)

• the same is true for the items in the KeyPoints box

In order to address #1 and #5 the following filters have been updated:

• CDR434822: Module: QC Filter Templates - Summary

• CDR339576: Module: InLine Markup Formatter

• CDR335169: Module: STYLE Default

• CDR380958: Module: STYLE QC Summary

I have CSS that handled the spacing for the list items replaced with the CSS from Cancer.gov (nvcg.css) and removed the "spacing" attributes for list items. I also tried to remove all templates responsible for the two template paths - compact and none-compact - leaving just one of the two. These changes make any use of the Compact attribute useless.

This is ready for review on DEV.

Should we create separate tickets handling #3 + #4 (removing the Compact attribute from documents and schemas) or should all of that be done under the current ticket?

We decided in the status meeting to break out the issues Volker mentioned in his comment.

Please go ahead and take a look at the output on DEV if you have time. I'm sure we'll do more testing on QA. One issue I've already seen is the display of the no bullet lists (a.k.a. address blocks).

I'm reviewing this now. One thing I've noticed - references in the reference lists seem to be indented too far.

Actually, it looks like the bulleted and numbered lists are indented pretty far, too (farther than on Cancer.gov). Is it possible to decrease the indent a bit? Just thinking about certain summaries with very long reference lists that will become a little bit longer with this change 🙂

I will have to talk to a CSS expert regarding the indentation. I specified the same CSS rule that NVCG is using but it does seen that our QC reports are indenting about 4 times the space compared to Cancer.gov.

I've adjusted the list-item indentation and added bullets to the TOC items.

Volker and I discussed this and he is going to remove the bullets from the TOC and increase the indent some. He will also add some white space to the left margin. I will review this next on QA.

These latest changes have now been setup and - to my surprise - they are working the same in IE and FF.

I've reviewed this in IE on DEV and I think it looks good. It will be good to get everyone's thoughts on this during QA testing, though.

The following filters have been copied to subversion:

• R13220: CDR434822: Module: QC Filter Templates - Summary

• R13220: CDR339576: Module: InLine Markup Formatter

• R13220: CDR335169: Module: STYLE Default

• R13220: CDR380958: Module: STYLE QC Summary

In HP bold underline and RLSO, the first level bullet is a filled circle and the second level bullet is a filled square (when the attribute is set to “dash”). In PT RLSO and bold underline, the first level bullet is a filled circle and the second level bullet is a filled square as well (the bold underline is correct). All bullets should be filled circle (the NVCG style) since we no longer have the ability to create a “dash” on NVCG. Adult Primary Liver Cancer (Patient version, CDR0000256491) is a good example of this. Try Childhood Rhabdomyosarcoma (CDR0000062792) for an HP example. We may want to remove the "dash" option from the CDR schema in a future release.

I have made a small modification to the patient R/S QC report which is causing a big change. Because of the way that the bullets are defines for Cancer.gov I was trying to use the NVCG CSS. This does display the bullets just like on Cancer.gov. It also displays everything else just like on Cancer.gov.
I feel we have two options now:

• Go ahead and use the NVCG CSS. This would display many element exactly like on Cancer.gov but also break some others. It would mean we'll have a lot more to do in terms of adjusting/fixing our HTML output for the Darwin release or

• Just adjust the bullets for now and plan to convert to using the NVCG CSS as part of the Darwin release.

We should probably go with option 2 for now.

Option 2 seems like a good plan to me, although I'm not sure if we need to fully convert to the NVCG CSS if there are so many other "side effects". We can revisit this in the context of the Darwin release, and we'll be fine with the square bullets at the second level in the meantime.

Oh no, I didn't mean to say you'll have to live with squares. We can still make bullets for all levels of lists. However, if Cancer.gov would suddenly decide to use hearts as bullets it would not automatically be reflected on our QC reports.
Keep that in mind for the next major Cancer.gov release. :-)

By the way, if you want to take a quick look at how the "new" QC reports would look go ahead and run a RS for patient on QA.

I just did 🙂 I can see we'll need to do some clean-up work.

Circles would be great, but this isn't a big deal if this is difficult to do.

OK, bullets are discs on all levels now.

Sorry, I'm in a rush to get out.

I'll have to check if the bullets are displaying correctly now for all QC reports: patient, HP, BU, R/S, QD.

I looked at all QC reports and they are behaving the same. However, I do see a couple of problems we'll have to decide if we can live with until Darwin (or longer) or need to have changed as part of Curie.

• The adjustment to the bulleted lists displays the TOC list with bullets now. This is a change from the previous behavior but the TOC on Cancer.gov is displayed with bullets and the TOC in Word is displayed with bullets as well.

• On Cancer.gov a list item has padding above and below and so does a list (UL). The QC report had to adjust the padding based on the compact attribute, so padding didn't exist around list items. In lists with paragraphs, however, there is padding below a para causing list items to sometimes display with and mostly without padding.

I believe the second issue should be left for the larger Darwin story since it would require changes of the current display for para, heading, and other elements.

I see what you mean about the "padding" - I think that is a problem. Things are now pretty squished together, but I know you are planning to stop by so we can look at that.

I don't think the bullets look very good in the TOC, but I've asked everyone to weigh in on that so I'll let you know.

One other thing we've noticed - in the screening and prevention summaries, the study design, internal validity, etc. items in the overview section now have bullets in the QC report (bold/underline and RLSO) but they do not have bullets on Cancer.gov. See CDR 62779 (Breast Cancer Prevention HP) as an example.

The spacing looks good! As does the TOC. Thank you. I think the only issue that remains is the extra bullets beside things that shouldn't have them (like the Breast Cancer Prevention example). However, if this is difficult to track down, we can push this to Darwin.

No, this is not difficult. I just have to find all of the placed where the class to suppress the bullets is used.

The lists that should display without bullets have now been fixed as well.

Thank you! This looks great. Verified on QA.

Please verify the new look and feel the difference. :-)

Please close the ticket if the changes look good on PROD.

The bullets are displaying far to the left of the text on PROD when I view the summary in MS Word. This is not fixed by running the macro. I will add a couple of screenshots.

One more observation: Longs headings that wrap onto the next line are not indented to the appropriate level. (Referring to the Table of Contents)

I know the solution to this problem: Let's use WordPad. :-)

Seriously, I checked this on STAGE and I see the same ugly behavior on STAGE as you have captured here from PROD. I do not believe we actually overlooked this problem, so the only explanation that I have without doing much research on the problem is to assume that either IE or Word have been updated since we QC'ed this on STAGE.

I will do a little more research and check would we modified that could have caused this change in behavior.

I've tried QA, too - same thing. I am trying to copy/paste from another browser now.

Copying from FF seems to be working for me, copying from IE or Chrome does not.

I tried both Chrome and Firefox. The copied/pasted QC report from Chrome looked the same as it did coming from IE (except, for some reason, the font size is larger than specified by the macro). The copied/pasted QC report from Firefox looked great in terms of the bulleted lists, but the table gridlines were missing and comments appeared as normal text (not bold/italicized text). So, that didn't help much.

Volker, do you have any thoughts as to a possible work-around for this for the time being? It is affecting both the RLSO and B/U reports. Is there a way to reconfigure the way bulleted lists are formed in Word? I seem to be able to do that by bullet, but not for the full document (or even for a full list if there are varying levels of bullets). Thanks.

I will spend some time on this today. We will first need to find out if the problem is Word or the QC report itself. Obviously, the old formatting of the QC reports did not cause this problem, so it should be possible to adjust the CSS and implement a work-around. We may have to go back to the old CSS.

The good news is that this is handled by a filter change and we're not dependent on CBIIT to install the changes!

Thanks, Volker! That is very good news! 🙂

volker/Volker

At least you didn't write Volcker. Everybody at the PTA seems to be doing that recently. :-)

it appears that you're getting slightly better handling of the itemized lists if you're saving the HTML QC report as an HTML document and then open it in Word.
The entire list is still shifted too far to the left with the bullet or number outside of the text area but the spacing between the bullet and the list item looks better.

I've only tested this in IE and FF and they both behave identically as far as I can tell.

Interesting. That does look better, but I agree it's still too far to the left. And the increase/decrease indent options don't work to move them over. 🙁

We might have to revert back to using the previous filter, if that isn't too much work or too fragile, while we continue to troubleshoot on another tier. We're in the thick of Board meeting prep now for all of the September meetings so those QC reports are getting a work out!

OK, I see the urgency now. I'll try to retrieve the old versions of the styles from subversion tomorrow to replace on PROD while we're trying to continue our investigation on STAGE or lower.

Thanks, Volker!

Just noticed another problem with the new QC report filters. Links within tables are not underlined. We print our summaries in B&W for the Board meetings, so it needs to be clear that a link is there. I added a screenshot.

The citation links seem to be underlined, so it's not a general issue with links but certain types of links.
What element is the Gail/BCRAT?

It's an external ref. Summary fragment refs are also not underlined in tables.

Robin, I am guessing you are running QC reports and copying those into Word all the time, right?
My question is, did you copy/paste QC reports to Word since the new Word macros had been installed and before the filters were copied to PROD?

Yes, I have. I'll upload an example of one that I ran last week.

I've looked at all of the differences of the Curie filter changes and updated the most likely candidate to the previous version on DEV. The filter is

• R13193: CDR335169 - Module: STYLE Default

The lists look good again. Please take a look and let me know if you'd like me to restore this version on PROD.

This version doesn't have any space between references. Sorry. Is there another candidate? 🙂

I have now reverted all filters related to formatting summaries to the version prior to Curie but the spaces between the reference sections are still missing.
I will have to check if any filter changes may be in the trunk of subversion - filter changes in production but not part of Curie.

If we are talking about a workaround until we know what's going on with Word the extra space for the references can easily be achieved given that there are not as many reference sections in a summary as there are lists.

It's actually all lists that are single-spaced, not just references. It was most noticeable in the reference lists.

You didn't say that but it makes more sense. We've removed the extra spaces after the non-compact list items because - as done in NVCG - the spacing for lists is controlled by CSS now.

I believe I found the style causing the problems with the bullets but I'd like to test a little more. It's currently updated on DEV.

Robin, what's a summary with a link in the table that I could use to test?

I just mocked up an example on DEV - CDR517309. There are several tables of Genetic Resources at the bottom. The first table includes an External Ref, a summary Frag ref, and a glossary term ref in proposed insertion markup and all are underlined on the bold/underline QC report.

When we copy/paste the B/U into Word the bullets are still way off to the lft. When we copy/paste the RLSO into Word, the bullets look better.

I reverted the filters on DEV to the Curie release and made one change to the style that caused the problem with the bullets. The problem is a style for the lists (not the list items):

 ul, ol          { margin-top: 0;
                   margin-left:  1em;
                   padding-left: 1em;
                 }

Including margin-left causes the problem with the bullets in Word but reduces the list item indentation in HTML.
Including padding-left only works OK in Word but removes indentation for lists in HTML (see screenshot-4).
Without margin-left and padding-left the HTML indentation is increased (see screenshot-5).
So, I think the workaround would be to drop both, margin-left and padding-left and ignore the accept the larger indentation for lists for the moment.

I don't think this will address the issue with the links not being underlined, Robin, but all links in tables in your test document display with the underline, so I would need to see this on PROD.

Yes, I think we should try removing those two items, too. Thanks.

When we copy/paste the B/U into Word the bullets are still way off to the lft. When we copy/paste the RLSO into Word, the bullets look better.

There are three style filters:

• a default for all reports

• a style filter for B/U only

• a style filter for R/S only

If it only looks good for one of the two reports then I probably still have to copy the same changes to the other style filter.
I'll check it out.

A few staff at CIAT are testing. This is what Kirsten Noticed:

"Just one thing I noticed in DEV while testing the QC reports. The B/U QC reports look fine, but in the RLSO reports, the summaryfragrefs are not showing up in blue (see below). They are underlined, but not blue. "

I will attach the screenshot shortly.

Which summary is that, William?

Never mind, I was able to enlarge the image and get the CDR-ID.

the summaryfragrefs are not showing up in blue

This is odd. According to the subversion logs this style itself hasn't changed in 4 years.
Do we want to change it now without additional testing?

I suggest we go ahead and make the change for the bullets on PROD for now and come back to these type of findings later - possibly captured under a new ticket.

I just looked at a printout of a RLSO QC report Victoria had from last week that the summaryfragrefs were black there, too, so this does not appear to be related to Curie (but a good find, nonetheless!). I agree that we should proceed with the bullet/list fixes for now and save that fix for the next time.

The style has been changed on PROD with the filter update

• R13453: CDR335169 - Module: STYLE Default

Hooray! I think we have a winner. Thank you, Volker!

I see three issues remaining

• Some links within tables aren't displayed with an underline (see screenshot-3)

• SummaryFragmentRefs within tables aren't displayed with the proper color (existing bug)

• Indentation for lists may not be exactly what we want (problem with copy/paste to MS-Word)

Should we add tickets for these three (or first two?) items in order to close out Curie or does any of these need to be resolved as part of this ticket?

I created a new ticket OCECDR-3970 for this issue:

This is odd. According to the subversion logs this style itself hasn't changed in 4 years.
Do we want to change it now without additional testing?
I suggest we go ahead and make the change for the bullets on PROD for now and come back to these type of findings later - possibly captured under a new ticket.

I'm considering this verified on PROD and closing this issue.