CDR Tickets

Issue Number 4034
Summary [Summaries] Prevent Modules from Being Published Automatically
Created 2016-02-19 15:19:15
Issue Type Bug
Submitted By Juthe, Robin (NIH/NCI) [E]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2016-04-11 12:53:14
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.179276
Description

We need to prevent module-only summaries from being picked up in a hotfix or group publishing job and publishing to Cancer.gov.

Comment entered 2016-03-03 13:54:38 by Juthe, Robin (NIH/NCI) [E]

This would be handled by the publishing control document, which we can update without the assistance of CBIIT.

Comment entered 2016-03-31 17:18:22 by Englisch, Volker (NIH/NCI) [C]

This would be handled by the publishing control document

This is not entirely correct. The regular publishing jobs are using SQL queries from the publishing control document in order to select the document to be published. Those queries can be updated without the assistance of CBIIT.
For some task, however, namely the Hotfix publishing job, the document IDs aren't supplied as the result of a query but are entered individually. Preventing modules from being (accidentally) published via a Hotfix publishing job or a Republish job requires changes to our publishing system and this needs to be part of a release.

Comment entered 2016-04-01 13:20:59 by Englisch, Volker (NIH/NCI) [C]

The publishing control document has been updated on DEV:

  • R13834: Primary.xml

Comment entered 2016-04-01 16:05:34 by Englisch, Volker (NIH/NCI) [C]

The following program has been modified to prevent the publishing of modules:

  • RepublishDocs.py

I will have to discuss with Bob if we want to fail creating a job if a module has been included in the list of CDR-IDs or if we ignore the module and publish the remaining documents provided.

Comment entered 2016-04-04 14:52:42 by Englisch, Volker (NIH/NCI) [C]

I will discuss with Bob if the changes for the Hotfix-Export should be made in the server or elsewhere.

The SQL queries in the server (line 655 + 665 in CdrPublish.cpp will need to be modified to include the following exclusion:

...
AND v.id NOT IN (SELECT doc_id 
                   FROM query_term i
                  WHERE i.doc_id = v.id
                    AND path = '/Summary/@ModuleOnly'
                )
Comment entered 2016-04-11 12:49:10 by Englisch, Volker (NIH/NCI) [C]

I've modified the program publishing.py. This script already addressed the problem of audio recordings accidentally getting published as a hot-fix. I extended it to exclude modules:

  • R13878: publishing.py

This is ready for testing on DEV.

Comment entered 2016-04-11 12:52:08 by Englisch, Volker (NIH/NCI) [C]

The following program has been modified to prevent the publishing of modules as part of the republish interface:

  • R13821: RepublishDocs.py

Comment entered 2016-04-12 12:22:15 by Juthe, Robin (NIH/NCI) [E]

Hi Volker,

I just created a new module on DEV for testing but I am not authorized to publish it. Could you please either publish it or give me permission to do so? I promise not to break anything!
CDR778230 - Robin's Test Module

Thanks!

Comment entered 2016-04-18 17:43:08 by Osei-Poku, William (NIH/NCI) [C]

It looks like the sample document Robin mentioned above is now publishable. In case we need more modules for testing, I have created two new modules on DEV for additional testing. I also wanted to find out if for test purposes we need to vary the modules attributes? That is, currently we have all of them set as AvailableAsModule ="Yes" and ModuleOnly ="Yes". Is that okay? Here are the two new modules:
CDR0000778286
CDR0000778287

Comment entered 2016-04-18 17:59:14 by Englisch, Volker (NIH/NCI) [C]

Where do you see it published? I'm looking on Gatekeeper DEV and don't see the document.

Comment entered 2016-04-18 18:36:39 by Osei-Poku, William (NIH/NCI) [C]

Sorry for succeeding in confusing you. I said "publishable" :-). I assumed that Robin was unable to make the document publishable. I didn't think she meant "publishing" it to Cancer.gov.

Comment entered 2016-04-18 19:08:57 by Englisch, Volker (NIH/NCI) [C]

Yes, you did. I didn't read carefully enough.

If you want to do additional testing on DEV that is OK but I hadn't planned additional testing. Let me know if you'd like to do additional testing on DEV. Otherwise you may want to create those test modules on QA instead.

I also wanted to find out if for test purposes we need to vary the modules attributes?

No, the attributes are fine as is.

Comment entered 2016-04-19 10:07:17 by Osei-Poku, William (NIH/NCI) [C]

In that case we will test on QA at the appropriate time. Thanks!

Comment entered 2016-04-22 12:05:01 by Englisch, Volker (NIH/NCI) [C]

I had changes not yet versioned in my sandbox:

  • R13948: RepublishDocs.py

Comment entered 2016-04-27 16:18:35 by Osei-Poku, William (NIH/NCI) [C]

Here are two modules on QA for test publishing:

CDR0000779882
CDR0000779883

Comment entered 2016-04-29 13:26:14 by Englisch, Volker (NIH/NCI) [C]

Test #1 - Success
The new modules were not picked up for publishing as part of the nightly publishing job.

Comment entered 2016-04-29 13:27:48 by Englisch, Volker (NIH/NCI) [C]

Test #2 - Success
Attempt to hot-fix summary modules failed.

Comment entered 2016-04-29 14:21:45 by Englisch, Volker (NIH/NCI) [C]

Test #3 - Success
Attempt to Re-publish summary modules failed.

Comment entered 2016-04-29 14:31:47 by Englisch, Volker (NIH/NCI) [C]

I don't want to submit a publishing job because this will slow down the system to a crawl. We should let the weekend publishing job not pick up these documents and confirm on Monday.

Comment entered 2016-05-04 14:36:33 by Englisch, Volker (NIH/NCI) [C]

The two modules were not prat of the weekend publishing job.

Comment entered 2016-05-27 17:50:25 by Englisch, Volker (NIH/NCI) [C]

Can we close this ticket?

Comment entered 2016-06-15 09:58:46 by Juthe, Robin (NIH/NCI) [E]

Has it been verified on PROD? I don't think we've published any new modules since May 13. We can close it for now and reopen if it becomes a problem, though.

Comment entered 2016-06-15 10:15:58 by Englisch, Volker (NIH/NCI) [C]

We tested it on all other tiers. Of course, we don't really want to create a "fake" module just to test that it doesn't get published.
I agree, we should close it for now and reopen if needed.

Elapsed: 0:00:00.001397