Issue Number | 4348 |
---|---|
Summary | Improve "Post Publishing" Process |
Created | 2017-12-12 14:30:45 |
Issue Type | Improvement |
Submitted By | Englisch, Volker (NIH/NCI) [C] |
Assigned To | Kline, Bob (NIH/NCI) [C] |
Status | Closed |
Resolved | 2019-08-01 07:14:39 |
Resolution | Fixed |
Path | /home/bkline/backups/jira/ocecdr/issue.218223 |
We identified several problems when a publishing job failed during the CG2Public step and we tried to rerun this step using the "Post Publishing" scheduled step.
Problem 1:
When the "Post Publishing" job is run it will only run successfully if
no other publishing job has run since the weekly publishing job ran
because the job is expecting the publishing data in the latest directory
with the latest Job-ID.
Solution 1:
Identify the latest weekly publishing job ID by selecting it from the
database.
Problem 2:
In "Live" mode the system expects the directory to be created in the
LicenseeDocs directory not to exist. It will fail to run and update the
files.
Solution 2:
Implement a step to clean out a directory created during a previous,
unsuccessful run.
Problem 3:
If the post-processing job fails during the CG2Public step the step to
update the FTP server will run regardless. This means that an incomplete
set of documents will be copied to the FTP server. Running the step
again at a later time won't create a diff report to last weekend's
output but to the latest FTP update job.
Solution 3:
Create checksums of each file and store in order to create the diff
report. This will allow the creation of diff reports between any
publishing job, not just the current and the most recent one.
Problem #2 has been addressed in Gauss.
~volker will take a look at problems #1 and #3.
Problem #1 has been addressed as part of Hawking (OCECDR-4347).
I have taken care of the last problem. Can we schedule a code walkthrough for some time tomorrow, ~volker?
https://github.com/NCIOCPL/cdr-publishing/blob/joule/Publishing/sftp-export-data.py
Sure, maybe after lunch?
Hmm. I tried (twice) to reply with a "thumbs-up" emoji, and both times Jira not only swallowed my reply, but also suppressed the display of your reply. Yours came back with a refresh, but not mine. Another Jira bug. Anyway, after lunch is fine.
Elapsed: 0:00:00.000670