CDR Tickets

Issue Number 3727
Summary Bug in Publishing Verification on Lower Tiers
Created 2014-02-19 16:52:17
Issue Type Bug
Submitted By Englisch, Volker (NIH/NCI) [C]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2014-02-25 11:31:20
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.118614
Description

Aarti had asked me to run a publishing job to test one of the WCMS stories but the document had to be pushed to a server other than our default Gatekeeper server. The default server is gatekeeper-DT.qa.cancer.gov while the server to be used was gatekeeper.qa.cancer.gov.

After running these tests our publishing jobs started failing and Blair identified that I was asking the wrong server for the push job status.

It turns out that the problem of asking the wrong server for the GK job status could only be resolved by restarting the publishing service.

Comment entered 2014-02-19 17:09:47 by Englisch, Volker (NIH/NCI) [C]

Our default GK server is specified in our hosts file

\etc\cdrapphosts.rc

Whenever we need to push documents to a different server we can specify the new server name as a parameter (GKServer) in the Admin interface. The parameters are written to the table

pub_proc_parm

and the verification process queries this table to identify which server to connect to. If the GKServer parameter has been set the default hostname gets changed to this hostname that's found in the pub_proc_parm table for the given push job.

Note: This only happens on lower tiers as we never change the GK server target in production.

Following the tests (as requested to a different GK server) we're running regular publishing jobs. For the regular publishing jobs the default hostname is used again. Once the publishing job has pushed the documents to GK, the publishing service is taking over again in order to verify the status of the push job. Again, as before, the verification process checks the server name that's specified in the pub_proc_parm table (in case the push job went to a non-default server) but the GKServer value is not set meaning the push job has submitted the documents to the default server and therefore the default server will not be changed for this push job. !!! Here is the bug !!!
The default server has been changed earlier when we submitted a job to a non-default server and that non-default server has now become the default!!! The result is that the default server has to be restored by restarting the publishing service or all following publishing jobs will have to specify the GKServer parameter with the default server name.

This should get fixed.

Comment entered 2014-02-25 11:30:46 by Englisch, Volker (NIH/NCI) [C]

I've made changes to the program

PublishingService.py

to reset the cdr2gk.host variable to the default host if no host has been specified in the pub_proc_parm table for the job.

I'm currently unable to test the changes because the database on QA is being refreshed. I will test the changes once the refresh is complete.

Comment entered 2014-02-28 14:33:39 by Englisch, Volker (NIH/NCI) [C]

The program has been committed to subversion because it can only be tested on the QA tier:

  • R12397: PublishingService.py

Comment entered 2014-02-28 15:25:55 by Englisch, Volker (NIH/NCI) [C]

I installed the new program on QA, published a document on the default Gaterkeeper, then published a document on the secondary Gatekeeper server. Both jobs where correctly verified.
Then I published another document on the default Gatekeeper again and with the changes to the software the verification correctly connected to the default server.

This is ready to go to production.

Comment entered 2014-05-01 17:44:35 by Englisch, Volker (NIH/NCI) [C]

The program changes had been successfully tested on QA, the only tier on which it would currently have any effect. (There must be multiple Gatekeeper setups on a particular tier in order for the original problem to show up.)
Now that the software has been copied to production I'm going to close the issue. No further testing on PROD is necessary.

Elapsed: 0:00:00.001061