PDQ Issues

Issue Number	4512
Summary	Fix bugs preventing hotfix/remove
Created	2018-08-07 15:16:52
Issue Type	Bug
Submitted By	Kline, Bob (NIH/NCI) [C]
Assigned To	Kline, Bob (NIH/NCI) [C]
Status	Closed
Resolved	2018-10-18 14:10:45
Resolution	Fixed
Path	/home/bkline/backups/jira/ocecdr/issue.230597

Description

The publishing script should only check the document type if the current publishing subset specifies a list of valid document types.

--- cdrpub.py   2018-08-07 14:58:34.408995300 -0400
+++ /d/cdr/lib/Python/cdrpub.py 2018-07-27 13:55:53.342819700 -0400
@@ -295,8 +295,9 @@
             for doc in self.job.docs:
                 if doc.id in self.processed:
                     continue
-                if doc.doctype.name not in spec.user_select_doctypes:
-                    continue
+                if spec.user_select_doctypes:
+                    if doc.doctype.name not in spec.user_select_doctypes:
+                        continue
                 self.docs.append("{}/{}".format(doc.id, doc.version))
                 self.processed.add(doc.id)
             if self.docs:

--- export-docs.py      2018-08-07 15:05:21.785235300 -0400
+++ /d/cdr/Publishing/export-docs.py    2018-07-27 14:44:03.405609700 -0400
@@ -131,7 +131,10 @@
                     messages = [error.message for error in errors]
                     directory = self.work_dir + "/InvalidDocs"
             xml = etree.tostring(filtered_doc.result_tree, encoding="utf-8")
-            xml = xml.replace(b"\r", b"").strip() + b"\n"
+            if xml is None:
+                xml = bytes(filtered_doc.result_tree)
+            else:
+                xml = xml.replace(b"\r", b"").strip() + b"\n"
             self.write_doc(xml, directory, filename)
             if errors:
                 raise Exception(messages)

Comment entered 2018-08-07 18:24:32 by Kline, Bob (NIH/NCI) [C]

~volker and ~oseipokuw: please don't run any hotfix/remove publishing jobs until I fix this (not likely, I know, since we have done any since some time last year, but just in case).

Comment entered 2018-08-08 08:31:59 by Kline, Bob (NIH/NCI) [C]

~volker: fix is installed on DEV. I tested earlier with removal of the CTGovProtocol documents, but I would rest easier if you did some testing as well. Thanks!

Comment entered 2018-08-28 09:40:55 by Englisch, Volker (NIH/NCI) [C]

I tested removing a blocked document in two ways:

blocking a document and publishing the document's document type
blocking a document and running a hotfix-remove for that document.
Both removal requests finished successfully on the CDR and pushed the remove request to Gatekeeper. The removal itself on Gatekeeper failed but this is due to an error on Gatekeeper preventing removal of some document types.

For the purpose of this ticket the test was successful.

Comment entered 2018-10-15 11:23:13 by Kline, Bob (NIH/NCI) [C]

~volker: I have applied the patch to QA. Do you want to test further, or should I proceed directly to the ticket for CBIIT to patch STAGE and PROD?

Comment entered 2018-10-15 11:57:08 by Englisch, Volker (NIH/NCI) [C]

You're good to go. I tested removing a document three times successfully:

Blocking a glossary term and running a HF-Remove
Blocking a summary and running a HF-Remove
Blocking a summary and running a Summary-Export

The remove request was submitted to Gatekeeper for all scenarios. The remove request failed on GK, however, this is due to a GK bug.

Comment entered 2018-10-15 13:28:26 by Kline, Bob (NIH/NCI) [C]

Opened ticket NCI-RITM0144624

Comment entered 2018-10-16 16:42:37 by Englisch, Volker (NIH/NCI) [C]

HF-Remove tested on STAGE successfully.

Comment entered 2018-10-18 14:10:45 by Kline, Bob (NIH/NCI) [C]

This is in production.

Elapsed: 0:00:00.001929

CDR Tickets