CDR Tickets

Issue Number 3404
Summary [Summaries] Modifications to Publishing Filter to include Module content
Created 2011-08-11 14:36:08
Issue Type Improvement
Submitted By Juthe, Robin (NIH/NCI) [E]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2012-09-18 17:38:44
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.107732
Description

BZISSUE::5097
BZDATETIME::2011-08-11 14:36:08
BZCREATOR::Robin Juthe
BZASSIGNEE::Volker Englisch
BZQACONTACT::Margaret Beckwith

As discussed in today's meeting, we will need to modify publishing software to allow the publication of module content. Modules may be publishing as stand-alone documents (handled via an attribute of ModuleOnly with a value of Yes if this is the only case) and/or as a component of a larger summary document. See OCECDR-3332 for background information.

It was suggested that we first refresh Mahler.

Volker, please adjust the component if necessary. Thanks.

Comment entered 2011-08-25 10:00:22 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-08-25 10:00:22
BZCOMMENTOR::Volker Englisch
BZCOMMENT::1

If a module is included into another summary will it always be the "entire" summary that will be included or would these summaries be used similar to misc. documents where maybe just a single SummarySection out of multiple sections available will be included?

Comment entered 2011-09-14 17:50:52 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-09-14 17:50:52
BZCOMMENTOR::Volker Englisch
BZCOMMENT::2

I created a new filter to pull in the text of a summary module. At the moment, the SummaryModuleLink is defined to link to a summary, not to a SummarySection, so I'm including all of the SummarySections from the specified module.

I do have a couple of questions on what to enter in the SummaryMetaData when the summary has been identified as ModuleOnly, like what to enter for the mandatory SummaryURL element or the MainTopics?

Once I used the SummaryModuleLink I also noticed that the element wasn't noticable at all in the tags-off view, so I made some minor modifications to the CSS for this element.

The new filter is
CDR712005 - Denormalization Filter: Summary Module

The modified CSS file is
Summary.css

I also updated the filter set
Denormalization Summary Set
to include the new denormalization filter.

I've tested the new filter with a small sample which I created myself. Maybe we could create a more complicated/realistic sample for me to test with?

Comment entered 2011-09-29 10:58:37 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2011-09-29 10:58:37
BZCOMMENTOR::Robin Juthe
BZCOMMENT::3

I created a sample summary module doc on Mahler, but I had several questions in doing so.

I didn't know how to populate the following fields:
Summary URL
Summary Title
Alt Title

I also have some concerns about summary frag refs and table numbering. Anyway, it's definitely helpful to have real data to work with to identify these issues! The document is CDR696908. I haven't linked it in any summaries yet, since I think we need to address these other issues first.

Comment entered 2011-10-19 17:39:16 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-10-19 17:39:16
BZCOMMENTOR::Volker Englisch
BZCOMMENT::4

In order to create a publishable version of the Main document (I've used CDR696899 - Testing Summary with Module) the module document needs to be publishable. You won't be able to make it publishable unless the mandatory elements are specified even if it's bogus data.
I also had to delete a few citations since these don't exist on MAHLER but then I was able to create a QC report and run the PP report. However, the numbering of the cdr:id values still needs some work because the cdr:id has been properly renamed for the module that I created but not the one Robin created.

Comment entered 2011-10-20 16:53:39 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-10-20 16:53:39
BZCOMMENTOR::Volker Englisch
BZCOMMENT::5

I've created the new filter that is denormalizing the new SummaryModuleLink element and takes care of renumbering/renaming the cdr:id attributes of elements of the module to ensure that the IDs are unique across the processed summary document. The new filter is called

CDR0000712005.xml - R10235: Denormalization Filter: Summary Module

and is has been included in the denormalization filter set on MAHLER.

For testing purposes I've used the main document CDR696899 - Testing Summary with Module which contains two SummarySections and two SummaryModuleLinks.

This is ready for review on MAHLER.

Comment entered 2011-10-20 17:35:38 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-10-20 17:35:38
BZCOMMENTOR::Volker Englisch
BZCOMMENT::6

(In reply to comment #3)
> I created a sample summary module doc on Mahler

Robin, would you mind looking at this document again and looking at the SummaryFragmentRef elements pointing to tables? I am thinking that you probably want to link to table 10 within the document which would require to modify the CDR-ID.

Comment entered 2011-10-21 12:38:07 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2011-10-21 12:38:07
BZCOMMENTOR::Robin Juthe
BZCOMMENT::7

(In reply to comment #6)
> (In reply to comment #3)
> > I created a sample summary module doc on Mahler
> Robin, would you mind looking at this document again and looking at the
> SummaryFragmentRef elements pointing to tables? I am thinking that you
> probably want to link to table 10 within the document which would require to
> modify the CDR-ID.

I modified the summary fragment ref so it now links to table 10 in this document.

Comment entered 2011-12-21 15:52:58 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-12-21 15:52:58
BZCOMMENTOR::Volker Englisch
BZCOMMENT::8

In order to test the dynamic Table and Figure numbering Robin created a test document which is great because it includes all sorts of links - within a summary, from module to summary, from summary to module, etc. - and it allows me to resolve all of the problems for a given scenario.

The numbering schema implemented earlier had worked fine with my simple test document but failed to correctly number the tables and figures with Robin's complex test document, so I had to rework the numbering.

I also had to rework the new filter denormalizing module information since I did not correctly account for module summaries included on a sub-section level.

Currently, there is one more scenario of links that I will have to address. In addition, we will need to adjust the QC reports because a SummaryFragmentRef to a document that is not the current document currently causes a new QC report for that summary to be created. However, when the link goes to a location within an included module we don't want to treat that module as a different summary and rather want to jump to the location within the document instead of calling the QC report for the module.

Comment entered 2011-12-28 15:05:17 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-12-28 15:05:17
BZCOMMENTOR::Volker Englisch
BZCOMMENT::9

Please also see the question in OCECDR-3428#c28.

Comment entered 2011-12-28 15:08:04 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-12-28 15:08:04
BZCOMMENTOR::Volker Englisch
BZCOMMENT::10

Correction:

Please also see the question in OCECDR-3428 comment 28.

Comment entered 2012-01-20 17:47:30 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-01-20 17:47:30
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

This issue depends on OCECDR-3428 which is currently on FRANCK for testing.

Comment entered 2012-01-26 17:11:03 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-01-26 17:11:03
BZCOMMENTOR::Volker Englisch
BZCOMMENT::12

I've copied the CSS files to FRANCK for testing:
Summary.css
Summary_structure.css

Comment entered 2012-01-27 22:17:13 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-01-27 22:17:13
BZCOMMENTOR::Volker Englisch
BZCOMMENT::13

Updating multiple related bugs at once:

The following filters have been updated on BACH:

  • CDR000157 - Vendor Filter: Summary - R10300

  • CDR271370 - Module: Vendor Filter Templates - R10264

  • CDR712005 - Denormalization Filter: Summary Module - R10299

  • CDR723649 - Denormalization Filter: Summary Module IDs - R10299

  • CDR723518 - Denormalization Filter: Summary Count Tables and Figures

  • R10299

  • CDR723519 - Denormalization Filter: Summary Resolve Table and Figure Number

  • R10308

  • CDR335424 - Denormalization Filter: Summary - R10307

  • CDR409593 - Copy XML for Summary Report - R10310

  • CDR609947 - Vendor Filter: Convert CG to Public Data - R10309

The DTD has been updated:

  • pdq.dtd - R10122

  • pdqCG.dtd - R10306

The CSS has been updated:

  • Summary.css - R10312

  • Summary_structure.css - R10312

  • Summary.ctm - R10311

The filter set 'Denormalization Summary Set' has been updated to include the following members:

  • Denormalization Filter: Summary Add Board Member - Spanish

  • Denormalization Filter: Summary Module

  • Denormalization Filter: Summary Module IDs

  • Denormalization Filter: Summary

  • Denormalization Filter: Summary Citation Formatting

  • Denormalization Filter: Summary Reference List

  • Denormalization Filter: Summary Reference De-Dup

  • Denormalization Filter: Summary Reference Numbering

  • Denormalization Filter: Summary InLine Numbering

  • Denormalization Filter: Summary Citation Wrapper Add

  • Denormalization Filter: Summary Citation Wrapper Sort

  • Denormalization Filter: Summary Count Tables and Figures

  • Denormalization Filter: Summary Resolve Table and Figure Number

Comment entered 2012-03-01 09:50:54 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-03-01 09:50:54
BZCOMMENTOR::Robin Juthe
BZCOMMENT::14

Some of the comments I just posted in a related issue (OCECDR-3428) pertain to this issue too.

[1. Text/tables within insertion elements inside a module are displayed in Publish Preview
when the module is included as a part of another document.]

[2. Proposed changes inside a module are displaying on QC reports of the main document
regardless of whether the option to show proposed markup is selected.]

In addition, we are unsure how to populate the URL field when a module document has the attribute of ModuleOnly=Yes.

We are not planning to use the PurposeText element when ModuleOnly=Yes. When the module is published as a stand-alone document, we expect that the "About This PDQ Summary" section will be displayed. Is that right? If so, we will need to populate the Purpose Text element in those cases.

Comment entered 2012-03-02 18:38:19 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-03-02 18:38:19
BZCOMMENTOR::Volker Englisch
BZCOMMENT::15

(In reply to comment #14)
> [1. Text/tables within insertion elements inside a module are...
> [2. Proposed changes inside a module are displaying on QC reports ...

These two issues have been fixed on MAHLER (and copied to FRANCK).

> In addition, we are unsure how to populate the URL field when a module
> document has the attribute of ModuleOnly=Yes.

As discussed at our status meeting, since the text will not be used when the document is being created as a module only it doesn't matter at all what to enter into these fields. However, the fields cannot be empty in order to satisfy the schema/DTD because the elements are mandatory.
Obviously, if a module is created as a stand-alone document the content for these elements will need to be created just like for any other summary.

> When
> the module is published as a stand-alone document, we expect that the "About
> This PDQ Summary" section will be displayed. Is that right?

Yes.

Comment entered 2012-03-16 16:04:57 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-03-16 16:04:57
BZCOMMENTOR::Volker Englisch
BZCOMMENT::16

The following filters have been copied to FRANCK and BACH:
CDR271370 - Module: Vendor Filter Templates - R10347
CDR000157 - Vendor Filter: Summary - R10347

I've also updated the filter sets on BACH to make the adjustment for the QC reports to display insertion/deletion markup for content within summary modules.
The following filter sets have been created or adjusted:

  • Denormalization Summary Module Set (new)

  • Denormalization Summary Set

  • QC QD Summary Patient Set

  • QC QD Summary Patient Set (Bold/Underline)

  • QC QD Summary Set

  • QC QD Summary Set (Bold/Underline)

  • QC Summary Patient Set

  • QC Summary Patient Set (Bold/Underline)

  • QC Summary Set

  • QC Summary Set (Bold/Underline)

  • Vendor QC Summary Set

  • Vendor Summary Set

Please verify the changes on Cancer.gov after tonight's publishing job.

Comment entered 2012-03-20 20:20:28 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-03-20 20:20:28
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::17

The following documents are ready for publishing on Franck:
62855 - Summary
62863 - Summary
714430 - Module

Comment entered 2012-03-21 17:08:55 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-03-21 17:08:55
BZCOMMENTOR::Volker Englisch
BZCOMMENT::18

I ran a hot-fix for both of the summaries on FRANCK but the processing failed on Gatekeeper with the message:
Unable to find Summary document with CDRID=714430.

This error message is due to a SummaryFragmentRef to a table within the module. Obviously, Gatekeeper doesn't recognize the href="CDR0000714430#_873" attribute as a location within the document.
The filter will need to convert the module's CDR-ID to that of the summary.

Comment entered 2012-04-19 19:17:53 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-04-19 19:17:53
BZCOMMENTOR::Volker Englisch
BZCOMMENT::19

I fixed the linking problem of the modules and was able to run a publishing job on the QA server once the filter had been updated:
CDR000157 - Vendor Filter: Summary

I used Small Intestine Cancer Treatment as a test summary, added a table to the main summary and one to the module and added four links:

  • linking from the main document to the table in the main document

  • linking from the main document to the table in the module

  • linking from the module to the table in the module

  • linking from the module to the table in the main document

All links are working as expected. You're welcome to check it out yourself on
http://wwwqa.cancer.gov/cancertopics/pdq/treatment/smallintestine/HealthProfessional

Comment entered 2012-04-30 15:00:39 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-04-30 15:00:39
BZCOMMENTOR::Volker Englisch
BZCOMMENT::20

Should we prepare a few more test documents on FRANCK to go through the publishing process for Summary Modules?

If there haven't been any other problems with the new modules this should be ready to be promoted as far as I can tell.

Comment entered 2012-05-07 15:07:37 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-07 15:07:37
BZCOMMENTOR::Volker Englisch
BZCOMMENT::21

(In reply to comment #20)
> Should we prepare a few more test documents on FRANCK to go through the
> publishing process for Summary Modules?

We've discussed at the last status meeting that CIAT will prepare a few more "module summaries" and test the implementation.

Comment entered 2012-05-08 19:29:04 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-05-08 19:29:04
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::22

(In reply to comment #21)
> (In reply to comment #20)
> > Should we prepare a few more test documents on FRANCK to go through the
> > publishing process for Summary Modules?
>
> We've discussed at the last status meeting that CIAT will prepare a few more
> "module summaries" and test the implementation.

The following summary and module are ready for testing:

The test module is 714430 (Peutz-Jeghers syndrome).
The test summary is 62890 (Genetics of Medullary Thyroid Cancer).

We discussed using a shorter summary. It has a link to the module and it contains tables, MediaLinks and Markup. The Module also contains a link to the Summary (SummaryRef) and it includes tables and markup.

Robin, are these all the things you want tested?

Comment entered 2012-05-09 09:47:25 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-05-09 09:47:25
BZCOMMENTOR::Robin Juthe
BZCOMMENT::23

(In reply to comment #22)
> (In reply to comment #21)
> > (In reply to comment #20)
> > > Should we prepare a few more test documents on FRANCK to go through the
> > > publishing process for Summary Modules?
> >
> > We've discussed at the last status meeting that CIAT will prepare a few more
> > "module summaries" and test the implementation.
> The following summary and module are ready for testing:
> The test module is 714430 (Peutz-Jeghers syndrome).
> The test summary is 62890 (Genetics of Medullary Thyroid Cancer).
> We discussed using a shorter summary. It has a link to the module and it
> contains tables, MediaLinks and Markup. The Module also contains a link to the
> Summary (SummaryRef) and it includes tables and markup.
> Robin, are these all the things you want tested?

Could you please add a few summary fragment refs from the summary document to the module and also add the new table/figure number elements? That should be all. Thank you!

Comment entered 2012-05-09 11:35:39 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-05-09 11:35:39
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::24

(In reply to comment #23)
> Could you please add a few summary fragment refs from the summary document to
> the module and also add the new table/figure number elements? That should be
> all. Thank you!

This is done. We've added Summary fragment refs as you suggested. The table and figure numbers already had the new number elements including the references to them so we are ready to go.

Comment entered 2012-05-10 15:41:20 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-05-10 15:41:20
BZCOMMENTOR::Robin Juthe
BZCOMMENT::25

(In reply to comment #24)
> (In reply to comment #23)
> > Could you please add a few summary fragment refs from the summary document to
> > the module and also add the new table/figure number elements? That should be
> > all. Thank you!
> This is done. We've added Summary fragment refs as you suggested. The table and
> figure numbers already had the new number elements including the references to
> them so we are ready to go.

Here's an update on my QC of the modules.

I created a new test module document. 714430 (Peutz-Jeghers) is the same module we've been testing with all along and it now has too many links to other documents for me to be able to tell what is working properly and what should be working and is not. I am using the summary document William prepared for testing.

Here are the CDR IDs of the test documents I'm using:

Summary - CDR 62890
Module - CDR 714473

I noticed a couple of things:

1. In the RLSO QC report of the summary document, text that is in approved insertion elements in the module is NOT displaying. Text that is in either approved or proposed deletion tags is displaying fine.

2. In the bold/underline QC report of the summary document, text that is in either insertion (I tested approved insertion tags) or deletion (I tested proposed deletion tags) is not displaying.

3. ReferencedTableNumber and ReferenceFigureNumber elements in the module are not filled in with a number when generating either the RLSO or B/U QC report or Publish Preview of the summary document.

4. All links (from main->main, module>module, main>module, and module->main) appear to be working fine. Hooray! It is worth noting (as I think we've already discussed) that links from the main document to the module open up a new QC report for the module only, rather than jump to that spot on the page. In publish preview, on the other hand, the document jumps to that spot on the page.

Comment entered 2012-05-10 16:38:17 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-10 16:38:17
BZCOMMENTOR::Volker Englisch
BZCOMMENT::26

Just to be clear: On which server are we doing this testing?

Comment entered 2012-05-10 16:54:35 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-05-10 16:54:35
BZCOMMENTOR::Robin Juthe
BZCOMMENT::27

(In reply to comment #26)
> Just to be clear: On which server are we doing this testing?

On Franck.

Comment entered 2012-05-11 12:13:09 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-11 12:13:09
BZCOMMENTOR::Volker Englisch
BZCOMMENT::28

(In reply to comment #25)
> 1. In the RLSO QC report of the summary document, text that is in approved
> insertion elements in the module is NOT displaying.

I cannot confirm this.
Table 3 (the third table of the module) is in insertion tags and is displayed for me when I'm running the RLSO report.

> 2. In the bold/underline QC report of the summary document, text that is in
> either insertion (I tested approved insertion tags) or deletion (I tested
> proposed deletion tags) is not displaying.

Again, I do see the inserted text displayed. Deleted text is not displayed per design.

> 3. ReferencedTableNumber and ReferenceFigureNumber elements in the module are
> not filled in with a number when generating either the RLSO or B/U QC report
> or Publish Preview of the summary document.

I see that there is one table number and one figure number not displayed. This is due to a missing and an invalid target. All other numbers do display for me in RLSO, BU, and PP.

> 4. All links (from main->main, module>module, main->module, and
> module-->main) appear to be working fine. Hooray! It is worth noting (as I
> think we've already discussed) that links from the main document to the module
> open up a new QC report for the module only, rather than jump to that spot on
> the page. In publish preview, on the other hand, the document jumps to that
> spot on the page.

This might be a side-effect from fixing the links for PP. I'll take a look at this but it certainly doesn't seem to be a major issue.

Comment entered 2012-05-11 12:14:51 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-11 12:14:51
BZCOMMENTOR::Volker Englisch
BZCOMMENT::29

One additional note: Many of the figure and table numbers are displaying double periods "Figure 1. .". I suggest to fix these as well as the incorrect table and figure links before I run a test publishing on this document.

Comment entered 2012-05-21 17:34:49 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-21 17:34:49
BZCOMMENTOR::Volker Englisch
BZCOMMENT::30

(In reply to comment #25)
> 1. In the RLSO QC report of the summary document, text that is in approved
> insertion elements in the module is NOT displaying. Text that is in either
> approved or proposed deletion tags is displaying fine.

As discussed at last Thursday's meeting the issue here was that Robin had used Insertion/Deletion markup around an entire SummarySection within a module which was not handled by my filter. This has been fixed now.

> 4. All links (from main->main, module>module, main->module, and
> module-->main) appear to be working fine. Hooray! It is worth noting (as I
> think we've already discussed) that links from the main document to the module
> open up a new QC report for the module only

This linking problem has been fixed now. All "internal" links including those from a module to the main doc will stay within the document and not run a new QC report for the module. The following filter has been updated:
CDR339576 - Module: InLine Markup Formatter - R10403
In order to test this change please ensure to test all links between modules and main docs as well as non-module external links to HP and Patient summaries.

Also, please note that some of the things Robin pointed out were wrong due to data entry errors as far as I could tell (missing table and figure number). You may want to correct the summary content first before re-testing.

This is ready for testing on MAHLER and FRANCK.

Comment entered 2012-05-24 12:00:55 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-05-24 12:00:55
BZCOMMENTOR::Robin Juthe
BZCOMMENT::31

(In reply to comment #30)
> In order to test this change please ensure to test all links between modules
> and main docs as well as non-module external links to HP and Patient summaries.
> Also, please note that some of the things Robin pointed out were wrong due to
> data entry errors as far as I could tell (missing table and figure number).
> You may want to correct the summary content first before re-testing.
> This is ready for testing on MAHLER and FRANCK.

The links between modules and main docs are working well. I still need to test external links to the HP and patient summaries - good point.

I'm still not seeing some things in markup that I think should be displaying, though. For example, in the B/U QC report of the summary (62890), the "More Information" section (in approved markup) does not display. Same goes for the RLSO QC report. I'm also not seeing a list that is in proposed deletion tags (with an Advisory Board attribute) in the B/U QC report of the summary.

I tried to correct the table/figure numbers, but I'm still doing something wrong. I think it may be best if William or someone at CIAT fixes them - William, would you mind taking a look at those and correcting them? Please check the table/figure number and referenced number elements in both the summary (62890) and the module (714473).

Thanks!

Comment entered 2012-05-24 18:43:00 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-24 18:43:00
BZCOMMENTOR::Volker Englisch
BZCOMMENT::32

(In reply to comment #31)
> I'm still not seeing some things in markup that I think should be displaying,
> though. For example, in the B/U QC report of the summary (62890), the "More
> Information" section (in approved markup) does not display. Same goes for the
> RLSO QC report.

I had said that the reports were ready on FRANCK and MAHLER but I guess at that point I had only copied the changes to fix the linking problems. When I checked more closely I realized that the filter to fix the Insertion/Deletion of entire sections in Modules had not been copied yet.
This is now fixed.

> I'm also not seeing a list that is in proposed deletion tags
> (with an Advisory Board attribute) in the B/U QC report of the summary.

From what I'm seeing at this point is that regular proposed advisory-board text does display correctly (i.e. text in a Para element) but the list gets dropped. I have to check a bit more but it's not unlikely that Robin found a bug in the filter that's been there for a while and might not be related to the module changes, namely that proposed lists with an advisory-board flag aren't handled properly.
I will continue working on this problem tomorrow.

Comment entered 2012-05-25 15:30:43 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-05-25 15:30:43
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::33

(In reply to comment #31)
> William, would you mind taking a look at those and correcting them? Please
> check the table/figure number and referenced number elements in both the
> summary (62890) and the module (714473).
>
> Thanks!

Working on Franck:

I did check and corrected the referenced table and figure numbers and as far as I can tell, they are showing correctly. However, it doesn't look like CDR0000714473 is marked as a module. Let me know if I should mark it as a module and also, if I missed any of the errors Volker pointed out, let me know.

Comment entered 2012-05-25 16:23:39 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-25 16:23:39
BZCOMMENTOR::Volker Englisch
BZCOMMENT::34

It doesn't need to be marked as a module (or ModuleOnly) if it can be published by itself.

Comment entered 2012-05-25 16:40:54 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-25 16:40:54
BZCOMMENTOR::Volker Englisch
BZCOMMENT::35

(In reply to comment #32)
> I have to check a bit more but it's not unlikely that Robin found a bug in the
> filter that's been there for a while and might not be related to the module
> changes

This turned out to be correct. The QC filters suppress the display of deleted text in the BU reports unless the deletion is proposed (the fact that this was advisory-board text was irrelevant). This was handled correctly for regular text (Para elements) but not for ItemizedLists and has now been fixed.

We may want to test these changes extensively since any of the markup-combinations could be affected by this change:
Insertion/Deletion, RS/BU, HP/Patient, Paras/Lists, approved/proposed/published, advisory/editorial boards and all of their 100+ combinations.

Comment entered 2012-05-25 16:55:52 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-05-25 16:55:52
BZCOMMENTOR::Volker Englisch
BZCOMMENT::36

(In reply to comment #33)
> I did check and corrected the referenced table and figure numbers

  • Table 4 still displays an extra period.

  • The last used ReferencedTableNumber element is pointing to a wrong target.

Comment entered 2012-05-25 17:13:42 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-05-25 17:13:42
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::37

(In reply to comment #36)
> (In reply to comment #33)
> > I did check and corrected the referenced table and figure numbers
>
> - Table 4 still displays an extra period.
> - The last used ReferencedTableNumber element is pointing to a wrong target.

They should be fine now.

Comment entered 2012-05-31 11:59:28 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-05-31 11:59:28
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::38

We are continuing to test the changes on Franck. I will enter a comment if we come across any problems or when we are done with testing.

Comment entered 2012-06-04 11:44:18 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-04 11:44:18
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::39

1. Table and figure numbers are omitted when using a SummaryFragmentRef to link to a table or figure from another summary. The links work correctly by going to the right table or figure when you click on it in the QC reports but the number of the table or figure isn't displayed. That is, if you want to say "See Table 4 in the Prostate Cancer summary", for example, where "Table 4" is the direct link to the table in the other summary. In the above example, the link to Table 4 in the Prostate summary works fine but only "Table" is displayed without the table number.

[See <SummaryFragramentRef> Table <ReferencedTableNumber> # </ReferencedTableNumber> </SummaryFragmentRef> in the Prostate Cancer summary]

2. When you insert markup directly around a TableNumber or FigureNumber element, it disables the numbering. Examples:

<Insertion><TableNumber>#</TableNumber></Insertion> doesn't work.
However,
<Insertion>Table<TableNumber>#</TableNumber></Insertion> does work.

Comment entered 2012-06-06 15:08:28 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-06 15:08:28
BZCOMMENTOR::Volker Englisch
BZCOMMENT::40

(In reply to comment #39)
> 1. Table and figure numbers are omitted when using a SummaryFragmentRef to
> link to a table or figure from another summary.

The templates to automatically number the figures and tables are only designed to automatically number the current document (including any modules). The filters are not written in a way to also number the tables and figures of an "external summary". If you want the link to the table of an external summary to specify "Table 4" you would have to manually enter "Table 4".

> [See <SummaryFragramentRef> Table <ReferencedTableNumber> #
> </ReferencedTableNumber> </SummaryFragmentRef> in the Prostate Cancer summary]

Which prostate cancer summary should I be looking at and on which server? I didn't see a sample in the treatment summaries.

> 2. When you insert markup directly around a TableNumber or FigureNumber
> element, it disables the numbering.

I don't see that. I created the situation you described on FRANCK with CDR62902 and the table numbers display correctly for all tables.

Comment entered 2012-06-06 15:47:05 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-06 15:47:05
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::41

(In reply to comment #40)
> filters are not written in a way to also number the tables and figures of an
> "external summary". If you want the link to the table of an external summary
> to specify "Table 4" you would have to manually enter "Table 4".
Thanks!

> > [See <SummaryFragramentRef> Table <ReferencedTableNumber> #
> > </ReferencedTableNumber> </SummaryFragmentRef> in the Prostate Cancer summary]
> Which prostate cancer summary should I be looking at and on which server? I
> didn't see a sample in the treatment summaries.

It looks like this is no longer necessary since as you already said above, it won't work.

> > 2. When you insert markup directly around a TableNumber or FigureNumber
> > element, it disables the numbering.
> I don't see that. I created the situation you described on FRANCK with
> CDR62902 and the table numbers display correctly for all tables.

I have just added insertion markup over the TableNumber elements of two of the tables in the test document on Frank - CDR0000062890: The one titled "Clinical Description"
and the other table titled "American Thyroid Association Medullary Thyroid Cancer Risk Stratification and Management Guidelines"

You can see references to the table before the tables are displayed; In the Clinical Description table, the reference to the table is found in the paragraph right before the table. It has the fragment id CDR0000062890#_743.

The reference to the "American Thyroid Association Medullary Thyroid Cancer Risk Stratification and Management Guidelines" table is found in the first paragraph of the same section where the table is found under the section titled "Genotype-Phenotype Correlations and Risk Stratification". It has the following fragment id :CDR0000062890#_731

If you run the B/U QC report for the test summary, you should see that the link is created but no table number is displayed for both references.

Comment entered 2012-06-06 17:45:05 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-06 17:45:05
BZCOMMENTOR::Volker Englisch
BZCOMMENT::42

This is an interesting case.

Let's look at the first example, the table in the "Clinical Description" paragraph. I'm wondering why you would want to put insertion markup around the TableNumber element (and only the TableNumber element). This element will not be edited (I think) once the summary has been updated to use the TableNumber elements. What you do markup is the SummaryFragmentRef with the ReferencedTableNumber and that's displaying correctly if the TableNumber isn't included in markup.
Of course, if there is a reason for placing the TableNumber in markup I'll update the filter to have the table number reference displayed for those cases as well.

Comment entered 2012-06-06 17:52:39 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-06 17:52:39
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::43

(In reply to comment #42)
> This is an interesting case.
> Let's look at the first example, the table in the "Clinical Description"
> paragraph. I'm wondering why you would want to put insertion markup around the
> TableNumber element (and only the TableNumber element).

I used Insertion markup only as an example. Deletion markup may do the same. If you're asking whether there would be a reason to markup just the element, I think they could be a reason to do that.

> Of course, if there is a reason for placing the TableNumber in markup I'll
> update the filter to have the table number reference displayed for those cases
> as well.

If this is possible, I will suggest that you do update the filter.

Comment entered 2012-06-06 18:04:24 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-06 18:04:24
BZCOMMENTOR::Volker Englisch
BZCOMMENT::44

(In reply to comment #43)
> If this is possible, I will suggest that you do update the filter.

It should be possible.

Would you need/prefer the ReferencedTableNumber to be displayed in markup if the corresponding TableNumber is in markup? If so, how would you want the situation handled when the ReferencedTableNumber is also in markup? Display it the same as if it were not in markup?

Comment entered 2012-06-07 09:51:18 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-07 09:51:18
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::45

(In reply to comment #44)
> (In reply to comment #43)
> > If this is possible, I will suggest that you do update the filter.
>
> It should be possible.
>
> Would you need/prefer the ReferencedTableNumber to be displayed in markup if
> the corresponding TableNumber is in markup?

No. It is okay not to do anything to the references. Users can handle the References manually, if need be.

Comment entered 2012-06-11 17:02:16 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-11 17:02:16
BZCOMMENTOR::Volker Englisch
BZCOMMENT::46

I've made changes to the filters to display insertion/deletion markup for the automated table number and figure number elements in the QC reports.

Please note that I have only modified the filters to display the figure and table numbers in regular Insertion/Deletion markup (Redline/Strikeout). I have not made changes to also display the numbers properly for proposed or advisory board markup. To do so will take a lot more effort but we decided at last Thursday's meeting that we only wanted to implement the markup for these elements (TableNumber, FigureNumber, ReferencedTableNumber, ReferencedFigureNumber) if it's not too much of an effort.

I've used these two documents for my testing:
CDR700000 - includes MediaLinks
CDR62902 - includes Tables

This is ready for testing on MAHLER.

Comment entered 2012-06-14 10:02:44 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-06-14 10:02:44
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::47

(In reply to comment #46)

>
> I've used these two documents for my testing:
> CDR700000 - includes MediaLinks
> CDR62902 - includes Tables
>
> This is ready for testing on MAHLER.

Verified on Mahler. Thanks!

Comment entered 2012-06-14 14:00:23 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-06-14 14:00:23
BZCOMMENTOR::Robin Juthe
BZCOMMENT::48

(In reply to comment #47)
> (In reply to comment #46)
> >
> > I've used these two documents for my testing:
> > CDR700000 - includes MediaLinks
> > CDR62902 - includes Tables
> >
> > This is ready for testing on MAHLER.
> Verified on Mahler. Thanks!

As discussed today, this is ready to be promoted after release 6.5.

Comment entered 2012-06-20 15:13:04 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-06-20 15:13:04
BZCOMMENTOR::Robin Juthe
BZCOMMENT::49

(In reply to comment #48)
> (In reply to comment #47)
> > (In reply to comment #46)
> > >
> > > I've used these two documents for my testing:
> > > CDR700000 - includes MediaLinks
> > > CDR62902 - includes Tables
> > >
> > > This is ready for testing on MAHLER.
> > Verified on Mahler. Thanks!
> As discussed today, this is ready to be promoted after release 6.5.

I am currently getting a python script error on Mahler that seems to be related to the modules. I'm pasting the error message I received when running the B/U QC report of CDR 62890 below. I also received an error message running a RLSO report of 517309. Thanks!

A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.

D:\Inetpub\wwwroot\cgi-bin\cdr\QcReport.py in ()
314 # Map for finding the filters for a given document type.

315 #----------------------------------------------------------------------

=> 316 filters = cdr.FILTERS

317

318 #----------------------------------------------------------------------

filters undefined, cdr = <module 'cdr' from 'D:\cdr\Lib\Python\cdr.pyc'>, cdr.FILTERS undefined

<type 'exceptions.AttributeError'>: 'module' object has no attribute 'FILTERS'
args = ("'module' object has no attribute 'FILTERS'",)
message = "'module' object has no attribute 'FILTERS'"

d:\cdr\Log\tmpuanoyv.html contains the description of this error.

Comment entered 2012-06-20 15:32:36 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-20 15:32:36
BZCOMMENTOR::Volker Englisch
BZCOMMENT::50

(In reply to comment #49)
> I am currently getting a python script error on Mahler that seems to be
> related to the modules.

It's actually related to Alan's fix of OCECDR-3518 from last night. When he fixed that bug and copied the change to MAHLER he had overwritten my changes that are necessary for the module.

I've re-fixed those changes. The report should be working again.

Comment entered 2012-06-20 16:04:17 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-06-20 16:04:17
BZCOMMENTOR::Robin Juthe
BZCOMMENT::51

(In reply to comment #50)
> (In reply to comment #49)
> > I am currently getting a python script error on Mahler that seems to be
> > related to the modules.
> It's actually related to Alan's fix of OCECDR-3518 from last night. When he fixed
> that bug and copied the change to MAHLER he had overwritten my changes that are
> necessary for the module.
> I've re-fixed those changes. The report should be working again.

Thanks! The reports are working fine.

Comment entered 2012-07-05 15:59:17 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-05 15:59:17
BZCOMMENTOR::Volker Englisch
BZCOMMENT::52

I talked to Blair regarding the problems with SummaryRef links he saw on Gatekeeper and it appears that there is still a bug in my filter code that's overly jealous in terms of marking a SummaryRef as the target within a module.

(Hopefully small) Adjustments to the filter are needed.

Comment entered 2012-07-06 17:47:13 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-06 17:47:13
BZCOMMENTOR::Volker Englisch
BZCOMMENT::53

The following filters has been updated to fix the linking issue identified by Blair:
CDR000157 - Vendor Filter: Summary - R10466

All of the SummaryRef elements that were not part of a Summary Module had been incorrectly modified.

Due to this change I would suggest two things:
a) Test to ensure the links are still working correctly (Main -> Module,
Main -> Main, Module -> Module, Module -> Main, SummaryRef)
I'm pretty sure everything is working OK but I thought that to be true
before these changes, too.
b) Include a Summary with Module (if we haven't already) in the set of
summaries that are being tested on wwwCA.cancer.gov (with or without
PermaTargs).

This is ready for review on MAHLER.
I've used CDR62890 as a test document (copied from FRANCK).

Comment entered 2012-07-11 09:50:21 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-07-11 09:50:21
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::54

(In reply to comment #53)

> b) Include a Summary with Module (if we haven't already) in the set of
> summaries that are being tested on wwwCA.cancer.gov (with or without
> PermaTargs).

Are you referring to the tests being done in OCECDR-3444?

Comment entered 2012-07-11 12:20:47 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-11 12:20:47
BZCOMMENTOR::Volker Englisch
BZCOMMENT::55

Yes. Those were the summaries that had been published for the PermaTargs and showed the broken links.

Comment entered 2012-07-11 16:52:18 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-07-11 16:52:18
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::56

(In reply to comment #53)
> The following filters has been updated to fix the linking issue identified by
> Blair:
> CDR000157 - Vendor Filter: Summary - R10466
>
> All of the SummaryRef elements that were not part of a Summary Module had been
> incorrectly modified.
>
> Due to this change I would suggest two things:
> a) Test to ensure the links are still working correctly (Main -> Module,
> Main -> Main, Module -> Module, Module -> Main, SummaryRef)
> I'm pretty sure everything is working OK but I thought that to be true
> before these changes, too.

All the links are working as expected.
We used the following documents for testing:
Module: Stem Cell Transplantation: CDR 734882
Summary: Childhood Hodgkin Lymphoma: CDR 62933

When the Module document does not have a publishable version, the QC report for the main document does not run successfully. I am not sure if this is the intended behavior.

Comment entered 2012-07-11 17:01:39 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-11 17:01:39
BZCOMMENTOR::Volker Englisch
BZCOMMENT::57

(In reply to comment #56)
> When the Module document does not have a publishable version, the QC report for
> the main document does not run successfully. I am not sure if this is the
> intended behavior.

Yes, it is. When a summary gets published you would not want to include a summary module that is not ready for publishing. Therefore a publishable version must exist first in order to include it.

Comment entered 2012-07-12 10:32:08 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2012-07-12 10:32:08
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::58

(In reply to comment #57)
> (In reply to comment #56)
> > When the Module document does not have a publishable version, the QC report for
> > the main document does not run successfully. I am not sure if this is the
> > intended behavior.
> Yes, it is. When a summary gets published you would not want to include a
> summary module that is not ready for publishing. Therefore a publishable
> version must exist first in order to include it.

Done. I have included a summary module in one of the test documents - 62707.

Comment entered 2012-07-18 16:36:28 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-18 16:36:28
BZCOMMENTOR::Volker Englisch
BZCOMMENT::59

The summary has been published and loaded to Gatekeeper QA.

Comment entered 2012-07-23 17:00:44 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-07-23 17:00:44
BZCOMMENTOR::Volker Englisch
BZCOMMENT::60

The following filter has been copied to BACH:
CDR000157: Vendor Filter: Summary - R10470

The HP disclaimer text (OCECDR-3489) can now be previewed in PublishPreview.
For the module and deep-linking changes the documents will need to be prepared and for the screening and prevention changes the MobileURL element will need to be populated. The ExcludeDevices attribute could be checked in the XML.

Comment entered 2012-08-07 17:38:26 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-08-07 17:38:26
BZCOMMENTOR::Robin Juthe
BZCOMMENT::61

We are working on the first module document on BACH (Peutz-Jeghers syndrome), and I wanted to bring up an issue that we are dealing with: what to do with the changes section.

I think it would make the most sense to store the changes section in the module document itself, rather than have to maintain the changes to that document in separate places and so that it follows the module content when it is published as a stand-alone document.

Volker and I just talked about this and he suggested that we might use an attribute to suppress the changes section when we are including it within another document so that it doesn't show up in the middle of a summary.

Let's discuss this issue on Thursday.

Comment entered 2012-08-08 12:37:35 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-08-08 12:37:35
BZCOMMENTOR::Robin Juthe
BZCOMMENT::62

We are also having an issue with the module link in Publish-level insertion element. The DocTitle of the module displays in the document that contains the SummaryModuleLink element, but the text of the module does not display in the B/U QC report or in Pub Preview.

Once we accept the changes (i.e., the module link element is no longer in markup), the section displays fine.

Comment entered 2012-08-09 15:30:03 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-08-09 15:30:03
BZCOMMENTOR::Volker Englisch
BZCOMMENT::63

We were so wrapped up in getting the module to be included in other documents that we completely forgot that the modules will also have to be excluded from being published individually for the attribute value ModuleOnly=Yes.

We need to modify the publishing document to exclude these summaries from being picked up for publication.

Comment entered 2012-08-29 17:01:03 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-08-29 17:01:03
BZCOMMENTOR::Volker Englisch
BZCOMMENT::64

I've updated the publishing document (cdr178.xml) on MAHLER and FRANCK and since Bob had me refresh FRANCK today I also ran a Summary-Export publishing job on FRANCK.
If this job runs successfully I will update the publishing document on BACH for this weekend's publishing job.

Comment entered 2012-08-30 13:49:32 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-08-30 13:49:32
BZCOMMENTOR::Volker Englisch
BZCOMMENT::65

We just thought about the reason why this "module" issue hasn't been closed yet?

I still have to modify the publishing document on BACH to prevent Module-only summaries from being published. I successfully ran a test publishing job this morning on FRANCK after unblocking the Peutz summary module to confirm that these documents aren't being picked up anymore.
I will copy the document to BACH shortly.

Comment entered 2012-08-31 17:57:05 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-08-31 17:57:05
BZCOMMENTOR::Volker Englisch
BZCOMMENT::66

The publishing document (CDR178.xml) has been updated on BACH.

I will close the issue once the weekly publishing job finished successfully.

Comment entered 2012-09-14 14:41:50 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-09-14 14:41:50
BZCOMMENTOR::Volker Englisch
BZCOMMENT::67

I've unblocked the Peutz-Jeghers syndrome summary module.

We will check next week to make sure that it's not included in the list of summaries to be updated/published.

Comment entered 2012-09-17 11:32:14 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-09-17 11:32:14
BZCOMMENTOR::Volker Englisch
BZCOMMENT::68

I confirmed that the unblocked Peutz-Jeghers syndrome summary module did not get updated for publishing on Friday just as expected.

I believe this issue could be closed now.

Comment entered 2012-09-18 17:38:44 by Juthe, Robin (NIH/NCI) [E]

BZDATETIME::2012-09-18 17:38:44
BZCOMMENTOR::Robin Juthe
BZCOMMENT::69

(In reply to comment #68)
> I confirmed that the unblocked Peutz-Jeghers syndrome summary module did not
> get updated for publishing on Friday just as expected.
> I believe this issue could be closed now.

Great! I will give the go-ahead to the rest of ICRDB to begin working on modules. Thanks for all of your work on this!

Closing this issue.

Elapsed: 0:00:00.001885