CDR Tickets

Added several watchers to this new ticket.

• Volker will look into how misc docs and modules are being handled in the filters to answer at least the following questions but probably others too:

  • Can misc docs be included within a summary module?
    Yes, misc docs can be included within modules.  However, please be aware that a MiscellaneousDocLink is only allowed within a SummarySection.

  • Will the misc doc text be displayed in the QC reports?
    Yes

  • what features/elements are NOT available in misc docs that we currently employ in the summaries?
    TBD

Comparing the elements of the Misc.Document and Summary schemas they are very similar.  The summary schema includes the following additional elements: 

• Comment

• EmbeddedVideo

• ResponseToComment

• SectMetaData

The Misc. Doc schema uses a Section element where the Summary uses a SummarySection element.

A few observations while trying out to work with Misc.Docs:

• Content included from a Misc. Docs must be in a publishable version

• Nested Section elements within the Misc. Doc are not preserved when included into a Summary.  The obvious workaround here is to have nesting setup within the Summary itself if that is needed.

• Related to the last point: The title of a section within the Misc. Doc is not carried over to the Summary.  If a title is needed it has to be created as the Title element of the SummarySection inside the summary.

• If we decide to use Misc. Docs we will need to change the MiscellaneousDocumentType element and create additional values for the LOV.

I discovered one issue so far with the Misc. Doc approach:  Citation links aren't handled properly by the filters.  Since misc. docs where only used for patient documents, we did not resolve Citation links within misc. docs and as a result those links are dropped from the content.  It will require a filter change to make citations inside misc docs work, too.

Hi ~volker,

I have added two examples. The standard text for special considerations document is the basic template that can be used in all of the pediatric health professional summaries. The second document is for the rare cancer summaries. Another option would be to use the standard text for special considerations for all of the pediatric summaries and use the text in the rare cancers document (comment marking the location) where a second document could be created that would only address the rare cancer summaries. Please let me know if my thoughts are unclear.

~volker has been taking the lead on the analysis for this ticket, so I assigned it to him.

I've created the two misc. documents on the DEV server with the Special Consideration text (CDR806249 and CDR806250) and added the first one to CDR62698 (Osteosarcoma and Undifferentiated Pleomorphic Sarcoma of Bone Treatment).  

For this example, I've added a SummarySection that includes only the MiscellaneousDocLink element.  The Section title will be taken from the misc. document.  You could also decide to add the section title to the summary but then you should not add a title to the imported Section to avoid duplication.  With this setup both, the QC report as well as the PP report look good to me.

Please note that I've left the original section as is so you can compare the old and new text directly within this document.

Which document would you like me to use for a test version with the rare cancers Special Consideration text, ~sershenca ?

Hi ~volker

Could we use the standard text for rare cancers document? Thanks!

Yes, we can certainly do that but what would be a "rare cancers document" that I should use for testing? I'm not too familiar with your summaries.

Could you give me a Summary title or CDR-ID?

The Section title will be taken from the misc. document.

It looks like this is not generally the case.  While I was testing with the summary CDR799716 (Childhood Pancreatic Cancer Treatment) I noticed that the title from the misc. document was not picked up for this document although it was picked up for CDR62698.  The difference between both documents is that one lists the Special Consideration section as a top-level SummarySection while the other one does not.  Therefore, in order to display the title for the SummarySection the title should be part of the Summary document and not of the misc. document unless we add some filter changes to handle the titles for these Special Consideration documents differently from the rest.

I think these are two samples of the rare cancers:

CDR 800109 Childhood Melanoma Treatment

CDR 799362 Childhood Tracheobronchial Tumors Treatment

While working with the MiscellaneousDocLink I'm realizing that I would prefer a little different display of this element. 

We have Media and Video links displayed with a border and module links with a green background.  This makes is very easy to quickly find a place in the document with these elements. I would like to suggest doing something similar with the MiscellaneousDocLink.

I asked Kirsten to review the changes on DEV and her main concern was that tables, images, citations, or links in the Misc Docs do not show up in the summary QC report or pub preview (they do show up in the QC report for the misc doc). So, it looks like at least QC reports and pub preview will need to be modified.

Crystal had asked about the status of this issue. Could we discuss this in the meeting next week and decide if we want to put this into Pauling?

The next step is for ~oseipokuw and ~sershenca to determine which element we need to support, with ~juther and ~vshields to do a first pass to make a preliminary determination.

Victoria, Crystal, and I met today to discuss this and came up with the following list of elements that will be needed in the Misc Doc schema in order to use this document type for the HP Special Considerations text.

Citation Link

Comment

Deletion

External Ref

Insertion

Itemized List

LOE Ref

Ordered List

Response to Comment

Summary Fragment Link ❓ - probably not needed as I don't think this is used in any of our summaries, but included just in case - we can discuss this one

Summary Fragment Ref

Summary Ref

Summary Section (includes Sect Metadata, Diagnosis, Section Type, Title, Alt Title, Key Point, Para)

Given this list of elements to be included for the Special Consideration text we probably should abandon the idea of using the Misc. Documents for this task.  If we're adding the SummarySection to the Misc Doc schema we might want to rethink using a SummaryModule instead because the modified schema would show (almost) all of the complexity of a Summary schema.  In fact, the Misc. Doc schema would be the only schema containing both, the Section element as well as the SummarySection element.

~volker  You were right, Kirsten had already setup two Misc Docs on DEV. 

0000806249
0000806250

However, she included other elements (table, media doc) for test purposes.

Filter changes have been implemented to accommodate all of the elements included in the two misc. documents (CDR806249, CDR806250).  I did not check, however, all elements in ~juther's list.  In addition to the filter changes, I modified the CSS for the Summary document type for the MiscDocLink element to stand out a bit and display similar to the SummaryModuleLink.  Also, the tag colors for the Misc. Doc has been modified to use the same tag colors as for the Summary documents.

The following files and filters have been modified:

• CDR000157.xml - Vendor Filter: Summary
  https://github.com/NCIOCPL/cdr-server/commit/cac437c

• CDR335424.xml - Denormalization Filter: Summary
  https://github.com/NCIOCPL/cdr-server/commit/cac437c

• Display/Summary.css

• Rules/Summary.ctm

• Rules/MiscellaneousDocument.ctm

This is ready for review on DEV.

Kirsten has done an initial review of the changes on DEV and is happy with it. I will provide more details or any further changes after meeting with her next week.

Kirsten and I met yesterday to test this out. We will have to do another round of testing in order to provide feedback.

I was able to meet with Kirsten this afternoon and we confirmed that everything is working as it should in the CDR.

I asked Crystal to take a look on DEV and she has confirmed that everything is working as expected.

I will go ahead and push the changes to QA in order to run diff reports.

I will need to take another look at the filters.  The diff for the summary QC reports showed that the SummaryRef element isn't handled correctly when it's part of the MiscDocument.

There was a bug in the way a SummaryRef was handled in a denormalization filter for the QC reports.  This bug has been fixed and the diff reports are now coming out clean.  This filter has been updated:

• CDR618263.xml - Denormalization Filter: Secondary Links
  https://github.com/NCIOCPL/cdr-server/commit/772795e

This is ready for review on QA.

While Bob was running his global changes for the browser title, he found another issue with the filter for the Misc. Doc for the Special Consideration.  For the PDQ partner output, the title of the Misc. Doc was included in the output.  In some cases, this caused two titles to exist for a section and therefore failed the validation.  I have modified the following filter to strip that second title:

• CDR335424.xml - Denormalization Filter: Summary
  https://github.com/NCIOCPL/cdr-server/commit/a981a13

This is ready for testing on DEV/QA.

I've ran diff reports on QA and everything looks good.  Let's make sure we prepare a couple summaries that use the Misc. Doc for Special Consideration and also run a publishing job for these.

The following summaries have been modified to include the Special Considerations misc. docs. They are ready for publishing/hotfix on QA. 

CDR0000799416
CDR0000799767
CDR0000799716
CDR0000062841
CDR0000062808
CDR0000062872

The publishing job failed due to a validation error which is a result of the changes related to the new browser title.  I will have to wait for testing of the browser title changes to finish before I can run a test publishing job for special considerations on QA.

~oseipokuw , the modified filters and style files have been replaced on QA but running a publishing job with these documents isn't possible at this time since the attribute KeepAtRefresh makes these documents invalid.  However, you're able to run the QC and PP reports.

This is ready for review on QA.

Thanks Volker! All the changes are showing correctly in the QC report with the exception of

1. Referenced Table Numbers in the Miscellaneous Document

2. Approved Markup of list Items in the Miscellaneous Document

Please see CDR0000799767 (Summary) and CDR0000812031 (Misc. Doc.) on QA.

~oseipokuw , could you be a bit more specific of what's not showing correctly?  When you're talking about the referenced table numbers, is it the table number that you don't see on the QC report (I don't see any problem here) or is it the display of the TableNumber element (a.k.a. the CSS) that you're referring to in the Misc. document itself?

For ListItems I believe you're referring to the Summary QC report where you would expect the item "Fertility specialists" to be displayed in insertion markup, right?

For ListItems I believe you're referring to the Summary QC report where you would expect the item "Fertility specialists" to be displayed in insertion markup, right?

That is right. 

~oseipokuw , could you be a bit more specific of what's not showing correctly?  When you're talking about the referenced table numbers, is it the table number that you don't see on the QC report (I don't see any problem here) or is it the display of the TableNumber element (a.k.a. the CSS) that you're referring to in the Misc. document itself?

Please see attached screenshots. The table is in the misc doc and it is referenced in the misc doc. but the QC report does not display it. I assume you're saying this is expected not to work, right?

Thanks, William!  Now I see.  You were looking at the ReferencedTabeNumber above the table.  I was looking at the TableNumber element instead.

I will take a look and see what's involved to make these two issues work.

I was able to address the table number issue by expanding the following filter:

• CDR723519 - Denormalization Filter: Summary Resolve Table and Figure Number

I'm still looking into the insertion markup issue.

The issue with the ListItem element within markup not being displayed is a lot more complicated than the issue with the table number and should be discussed.

The problem is the specific way the list item is being inserted.  The entire list item has been marked up with Insertion tags instead of just the text:

<Insertion><ListItem>Some Text</ListItem></Insertion>

The above isn't displaying the text correctly but if only the text will be marked up, this could be used as a workaround:

<ListItem><Insertion>Some Text</Insertion></ListItem>

The reason for this is the following:

One of our groups of summary filters scans a document for Insertion/Deletion markup and pushes the markup tags to be closest to the text itself.  These filters would, for instance, modify an inserted paragraph listed from

<Insertion><Para>This is a small paragraph</Para></Insertion>

to

<Para><Insertion>This is a small paragraph</Insertion></Para>

This process ensures that Insertion/Deletion tags will always be around text and not include element tags.  The problem here is that this group of filters runs early in the filter process at a time when the MiscellenousDocLink hasn't been denormalized yet.  This means, if we want to fix this issue we will have to modify the sequence of filters we're running from 

• Denormalize Modules

• Denormalize Video

• Process Insertion/Deletion

• Denormalize MiscDocLinks

• Other stuff

to something like the following

• Denormalize Modules

• Denormalize MiscDocLinks{}{}

• Denormalize Video

• Process Insertion/Deletion

• Other stuff

A change like this is certainly possible but we have to keep in mind that such a change might affect other documents or document types using MiscellaneousDocLinks.  Considering that this issue affects the editorial process of 2 documents I'm wondering if the effort to rearrange the summary filter set - along with the required filter adjustments and required tests - is worth it.

Thanks, ~volker  I think the workaround should be OK.

The following filters have been updated to rewrite the cdr:id attribute values for the denormalized MiscellaneousDocLink and insert numbers for the tables.

• CDR0000339576.xml - Module: InLine Markup Formatter

• CDR0000723519.xml - Denormalization Filter: Summary Resolve Table and Figure Number

• CDR0000335424.xml - Denormalization Filter: Summary

This is ready for review on DEV.

It looks like Table numbers are displayed correctly but figure numbers are not.  The misc doc has an image  which is referenced within the Misc. Doc but the number is not displayed in the QC report. 

CDR0000812692 - Misc

CDR0000799767 - Summary

The following filters have been modified to fix the display of figure numbers in the QC reports:

• CDR723519.xml - Denormalization Filter: Summary Resolve Table and Figure Number

• CDR000154.xml - Miscellaneous Document Report Filter

This is ready for review on DEV.

Looks good on DEV. Please install changes on QA. Thanks!

To keep filter names together in one comment, these are the filters that have been updated:

• CDR723519

• CDR339576

• CDR335424

• CDR618263

• CDR000157

These filters have been copied to QA.

Looks good on QA. Please install on the upper tiers. Thanks!

The filters have been copied to STAGE and PROD.

Please verify and close this ticket.

Verified on PROD. Thanks!