CDR Tickets

Issue Number 4335
Summary Non publishable media caption published to Cancer.gov
Created 2017-11-07 10:17:56
Issue Type Bug
Submitted By Osei-Poku, William (NIH/NCI) [C]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2017-11-07 16:04:12
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.216594
Description

It appears that changes that have been marked up but not yet made publishable in the Media Docs Captions have made it to Cancer.gov. So far, it appears to affect Spanish documents since we have not seen any example of this in English content. These are changes in versions that have not been made publishable yet in the captions for media docs. The unpublished changes are displaying in Cancer.gov summaries with the images.

Please check media doc: 785847
summary doc: 584552

link to summary on Cancer.gov:

https://www.cancer.gov/espanol/tipos/tiroides/paciente/tratamiento-tiroides-pdq#section/all

Comment entered 2017-11-07 11:40:36 by Englisch, Volker (NIH/NCI) [C]

I'm wondering if this is a problem that started happening but was working correctly before or if we're now starting to use revision markup in media documents.

Comment entered 2017-11-07 12:35:33 by Englisch, Volker (NIH/NCI) [C]

Didn't we talk about this issue before or was it something closely related?
The reason why the marked-up text for Media captions is displayed is a result of the order in which our filters are applied. This is the order of our filters:

  1. Insert all summary module links in our document

  2. Resolve the insertion/deletion markup

  3. Insert all Media documents

  4. other filters

As you can see, when the media docs are included in the summary document the markup has already been resolved in the parent document (and the modules). Anny additional markup will be displayed as regular text.
I think the way to handle these changes is by not creating a version.

If this is a big enough issue we may consider changing the order in which our documents are filtered but this would be a big change and could potentially affect all summary reports. Another alternative to that approach could be to run the Revision Markup Filter a second time to resolve additional markup as the one included via the Media documents.

Comment entered 2017-11-07 12:54:28 by Osei-Poku, William (NIH/NCI) [C]

We have been using markup in media docs for a long time now.

Comment entered 2017-11-07 12:59:18 by Osei-Poku, William (NIH/NCI) [C]

Yes, I remember we talked about this in relationship with another topic. The part I don't understand is why the filters are picking up text from unpublishable versions of the media document. I would think that it would only attempt to pick up publishable versions of the document. Is there a way to make the filters pick up only publishable versions ?

Comment entered 2017-11-07 13:10:39 by Englisch, Volker (NIH/NCI) [C]

I see! I thought the issue was the markup but your question is about picking up a different version.

We did - not too long ago - modify our filters to only select publishable versions for misc. documents. We could make that same change for media documents as well.

Comment entered 2017-11-07 13:49:22 by Osei-Poku, William (NIH/NCI) [C]

Sure. That will work. Thanks!

Comment entered 2017-11-07 14:32:40 by Osei-Poku, William (NIH/NCI) [C]

Will this be release dependent?

Comment entered 2017-11-07 14:40:31 by Englisch, Volker (NIH/NCI) [C]

No, this would be a filter change which doesn't need to be included in a release.

Comment entered 2017-11-07 14:44:59 by Osei-Poku, William (NIH/NCI) [C]

Great. Thanks!

Comment entered 2017-11-07 14:46:31 by Osei-Poku, William (NIH/NCI) [C]

I have updated the title and the original post to reflect the fact that this affects all media docs and also, it is the captions that are affected and not the content descriptions. Sorry for the confusion.

Comment entered 2017-11-07 16:03:12 by Englisch, Volker (NIH/NCI) [C]

The following filter has been updated on DEV:

  • CDR410709 - Module: Multimedia Denormalization

The change has been versioned in a local branch: [cdr4335 fa6e147]

Comment entered 2017-11-07 17:33:12 by Osei-Poku, William (NIH/NCI) [C]

PP is not working very well on DEV so should I make changes so we can hot-fix to test?

Comment entered 2017-11-07 18:49:38 by Osei-Poku, William (NIH/NCI) [C]

I have updated the following docs for possible hot-fixing on DEV:

Media doc: CDR0000719086
Summary doc: CDR0000258038
Summary doc: CDR0000258029

Comment entered 2017-11-08 10:27:48 by Englisch, Volker (NIH/NCI) [C]

This may be off topic but could you please explain what you mean with PP is not working very well on DEV?
Running a hot-fix has to go through the same processing as a publish preview report, so it's not going to be any faster and if there is a problem we need a ticket so we can fix it.

Comment entered 2017-11-08 11:28:43 by Osei-Poku, William (NIH/NCI) [C]

I have attached the blank page I got when I ran PP. That is all I see, no text on the page.

Comment entered 2017-11-08 12:54:29 by Englisch, Volker (NIH/NCI) [C]

PP is not working very well on DEV

I think it's fair to say that PP is not working for summaries. This is something we need to know especially in light of being in the process of upgrading our database to SQL Server 2016. Please don't ignore these type of issues by trying to work around it.

Could you please add a new ticket for the PP problem?

The hot-fix to PINK completed.
https://www-pink-dev.cancer.gov/types/thyroid/patient/thyroid-treatment-pdq

Comment entered 2017-11-08 13:11:15 by Englisch, Volker (NIH/NCI) [C]

Sorry, I did not notice that the initial push to Gatekeeper failed for the summaries. Those documents have been updated on PINK now.

Comment entered 2017-11-08 13:53:39 by Osei-Poku, William (NIH/NCI) [C]

I can't confirm that the changes have made it to pink yet. I entered "[Test]" after a paragraph in each of the summaries but I can't find it on PINK.

Comment entered 2017-11-08 14:26:52 by Englisch, Volker (NIH/NCI) [C]

One reason why a PP report is better suited for testing than a hot-fix is that you won't have to deal with Percussion publishing issues. I had to manually run a Percussion publishing event. You should now see your changes.

Comment entered 2017-11-08 15:47:16 by Osei-Poku, William (NIH/NCI) [C]

It worked. Thanks! Please run the hot-fixes again for me.

Media doc: CDR0000719086
Summary doc: CDR0000258038
Summary doc: CDR0000258029

Comment entered 2017-11-08 17:39:24 by Englisch, Volker (NIH/NCI) [C]

Since the PublishPreview report is working again on DEV please go ahead and test the results using the report.

Comment entered 2017-11-08 19:20:11 by Osei-Poku, William (NIH/NCI) [C]

I have verified the changes on DEV. Please proceed to apply the changes on QA.

Comment entered 2017-11-15 14:39:19 by Englisch, Volker (NIH/NCI) [C]

, the filter has been updated on QA.

I ran a diff report and found 4 documents with differences - all as part of the MediaLink caption:

  • CDR256718.xml

  • CDR258038.xml

  • CDR584552.xml

  • CDR62913.xml

Since QA had just been updated from PROD yesterday I'm assuming these are the same summaries currently displayed on PROD with problems.

Comment entered 2017-11-15 14:42:13 by Osei-Poku, William (NIH/NCI) [C]

Yes, I see at least one that I reported in my initial post.

Comment entered 2017-11-15 15:04:22 by Englisch, Volker (NIH/NCI) [C]

The filter change has been updated on STAGE. Please verify before moving the change to PROD.

Comment entered 2017-11-15 15:30:41 by Osei-Poku, William (NIH/NCI) [C]

I can't get PP to run. I am getting

Error in PubPreview: HTTP error: 500 (Internal Server Error) (occurred at Wed Nov 15 15:26:33 2017)

Comment entered 2017-11-15 17:07:36 by Englisch, Volker (NIH/NCI) [C]

I've been told that Percussion STAGE is currently being refreshed and won't be available until later tomorrow.
Since you checked the changes on DEV and QA already and this was a fairly minor change I've copied the filter to PROD.
Please verify and close this ticket if everything looks OK.

Comment entered 2017-11-16 13:50:39 by Osei-Poku, William (NIH/NCI) [C]

As discussed in the CDR meeting/conference call, tests on PROD show that the text in the caption is only showing the published version of the media documents, which is what we want. However, it appears
the images displaying in PP are revised images that have not been made publishable yet. The revised images are in versions of the media document that we expect to make publishable in the future.

Comment entered 2017-11-16 15:53:30 by Englisch, Volker (NIH/NCI) [C]

Yes, this is correct. The PublishPreview report uses the program GetCdrImage.py for pulling the image out of the CDR in order to display it on the report. This program, however, does not take into account the version of the Media document. It always pulls the latest blob which relates to the current working document.

You may want to submit a ticket to have the version of the captions match the version of the image when running a PP report.

Comment entered 2017-11-16 16:52:28 by Osei-Poku, William (NIH/NCI) [C]

I have created OCECDR-4339 to take care of it.

Comment entered 2017-11-27 10:23:21 by Osei-Poku, William (NIH/NCI) [C]

PP in PROD is showing both the old Caption and the new Caption. The new Caption has been appended to the old Caption. It is the same on Cancer.gov.

https://www.cancer.gov/types/thyroid/patient/thyroid-treatment-pdq#section/_27

Comment entered 2017-12-01 12:50:16 by Englisch, Volker (NIH/NCI) [C]

This appears to be the same problem we had with the scheduler, the filter change was overwritten by Fermi and needed to be re-applied. I've updated the modified filter on PROD again.

, please check if this is resolving the issue we discussed at yesterday's status meeting.

The filter updated is:

  • CDR410709 - Module: Multimedia Denormalization

https://github.com/NCIOCPL/cdr-server/pull/2/commits/fa6e147aa

Comment entered 2018-01-04 13:01:25 by Osei-Poku, William (NIH/NCI) [C]

The issue has been resolved. They are displaying correctly now. Thanks!

Attachments
File Name Posted User
PP on DEV.png 2017-11-08 11:27:56 Osei-Poku, William (NIH/NCI) [C]

Elapsed: 0:00:00.001332