CDR Tickets

BZDATETIME::2009-10-28 17:16:55
BZCOMMENTOR::Volker Englisch
BZCOMMENT::1

Solution of the problem from Bob:
------------------------------------
I had forgotten yesterday that HAVING only works with aggregate
columns. I think this is the right way to do the CTGovProtocol
selection query:

SELECT TOP 100000 v.id, v.num
FROM doc_version v
JOIN doc_type t
ON t.id = v.doc_type
JOIN (SELECT v.id, MAX(v.num) AS num
FROM doc_version v
JOIN active_doc a
ON a.id = v.id
WHERE v.publishable = 'Y'
AND v.val_status = 'V'
AND v.updated_dt <= '2009-10-27 11:21:58'
GROUP BY v.id) AS p
ON v.id = p.id
AND v.num = p.num
WHERE t.name = 'CTGovProtocol'
ORDER BY v.id

BZDATETIME::2010-02-04 14:23:20
BZCOMMENTOR::Bob Kline
BZCOMMENT::2

This needs to be addressed now, because it prevents us from solving the problem with protocols we tried to transfer and had bounce back to us.

BZDATETIME::2010-02-16 12:12:23
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::3

(In reply to comment #2)
> This needs to be addressed now, because it prevents us from solving the problem
> with protocols we tried to transfer and had bounce back to us.

From the CIAT/OCCM meeting this morning, Lakshmi and Margaret agreed that this issue needs to get a higher priority (P2) so that OCECDR-3074 can proceed quickly. I have therefore changed it to a P2.

BZDATETIME::2010-02-17 14:25:41
BZCOMMENTOR::Volker Englisch
BZCOMMENT::4

(In reply to comment #1)
> Solution of the problem from Bob:

Bob/Alan:
I am wondering if we should implement this change only for the CTGovProtocol and InScopeProtocol documents or - as a precaution - for all document types that are being published?

BZDATETIME::2010-02-17 17:40:22
BZCOMMENTOR::Volker Englisch
BZCOMMENT::5

I've updated the publishing document on FRANCK
CDR000178.xml - V46
to make the changes to the CTGovProtocol selection criteria proposed by Bob and tested the query with a few test cases of documents switching document types and being publishable/not publishable.

I've stored the publishing document on FRANCK and I'm currently re-running the publishing job that failed to properly process the protocols that need to be switched back to InScopeProtocols.
We will have the final result in the morning.

BZDATETIME::2010-02-18 06:41:33
BZCOMMENTOR::Bob Kline
BZCOMMENT::6

(In reply to comment #4)
> (In reply to comment #1)
> > Solution of the problem from Bob:
>
> Bob/Alan:
> I am wondering if we should implement this change only for the CTGovProtocol
> and InScopeProtocol documents or - as a precaution - for all document types
> that are being published?

I think the intention was to fix this across the board; "for instance" in the original report was meant to convey that the specific types were identified only as examples of the problem.

BZDATETIME::2010-02-18 10:14:37
BZCOMMENTOR::Volker Englisch
BZCOMMENT::7

(In reply to comment #6)
> I think the intention was to fix this across the board;

I modified the publishing document on FRANCK for the CTGovProtocols only yesterday and the publishing run successfully published CDR528955 as an InScopeProtocol rather than a CTGovProtocol.

You should be able to continue with OCECDR-3074 while I update the publishing document for the other document types.

BZDATETIME::2010-02-19 15:50:51
BZCOMMENTOR::Volker Englisch
BZCOMMENT::8

As discussed at our CDR status meeting yesterday we wanted to copy the updated publishing document to BACH so that the problem documents in OCECDR-3074 could be processed as part of the weekly publishing job.

The publishing document
CDR000178 - (cdr version 48)
has been copied to BACH.

BZDATETIME::2010-02-22 10:57:24
BZCOMMENTOR::Volker Englisch
BZCOMMENT::9

I had thought that the documents would be 'repaired' by running the weekend Export job but when I was checking the document I used for testing on FRANCK this wasn't the case.
It looks like the documents - at least the one I was looking at (CDR528955) - hadn't been prepared yet to be published as an InScopeProtocol. There doesn't exist a new version for the document with this document type.

Once the protocols are being prepared accordingly, publishing should process them as expected.

BZDATETIME::2010-02-22 11:58:49
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::10

The documents are all ready and I think we can do a hot fix.

CDR0000528955
CDR0000398096
CDR0000256928
CDR0000450765
CDR0000508653
CDR0000064256

BZDATETIME::2010-02-22 13:08:58
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

(In reply to comment #10)
> The documents are all ready and I think we can do a hot fix.

Are you aware that this hot-fix will require a manual run of the CTGovExport job and a manual upload of the data file?

BZDATETIME::2010-02-23 09:56:29
BZCOMMENTOR::Volker Englisch
BZCOMMENT::12

Since I did not get a response to my last comment I did not run the hot-fix yesterday and left everything to the regular nightly publishing job.

I checked the protocols on the list and they all have been included in the CTGovExport processing and pushed to CTGov.

BZDATETIME::2010-02-24 14:05:58
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::13

(In reply to comment #12)
> Since I did not get a response to my last comment I did not run the hot-fix
> yesterday and left everything to the regular nightly publishing job.
>

Thanks!

> I checked the protocols on the list and they all have been included in the
> CTGovExport processing and pushed to CTGov.

I verified on CT.gov and Cancer.gov and all the records are in there. The CT.gov records have the correct Last Update date of February 23, 2010. On Cancer.gov however, it retained the values in the DateLastModified field.

I am not the QA for this issue so I am not going to close it. I just wanted to post a note to say that I have reviewed the trials and did not see any problems.

BZDATETIME::2010-02-24 14:24:46
BZCOMMENTOR::Volker Englisch
BZCOMMENT::14

(In reply to comment #13)
> I am not the QA for this issue so I am not going to close it.

That was the right thing to do because of comment #6. The issue has not been completed yet (although the urgency has dropped somewhat).

BZDATETIME::2010-02-26 15:44:01
BZCOMMENTOR::Volker Englisch
BZCOMMENT::15

The publishing document has been updated to use the new queries for the remaining document types.

Currently running test publishing run on FRANCK.

BZDATETIME::2010-03-02 20:35:40
BZCOMMENTOR::Volker Englisch
BZCOMMENT::16

I ran a before and after weekly publishing job and saw no differences between the output files from both jobs (except for the missing PoliticalSubUnitShortName that was related to the changes in OCECDR-2954 and OCECDR-3076).

The publishing document
CDR000178 - (cdr version 49)
has been copied to BACH.

BZDATETIME::2010-03-04 15:06:55
BZCOMMENTOR::Volker Englisch
BZCOMMENT::17

Per discussion at today's status meeting we'll leave this issue open until the weekly publishing job ran successfully. Once the Friday job ran successfully we can go ahead and close the issue.

BZDATETIME::2010-03-08 13:12:31
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::18

We noticed that some summary documents were published on 03/03 and 03/04 even though they weren't supposed to be published until Friday 03/05, according to our publishing ‘rules’. However, 62817, 304518 and 62888 (there may be more) got published on either 03/03 (Wed) or 03/04 (Thurs) or both days for 62888, even though they were not hot-fixed, neither were they new summaries. I exchanged brief emails with Volker about this and he thinks this may need to be investigated under this issue.

BZDATETIME::2010-03-08 13:24:54
BZCOMMENTOR::Volker Englisch
BZCOMMENT::19

It appears that all summaries had been published on Wednesday (3/3) and Thursday (3/4) last week and these had been pushed to Cancer.gov:

Published 3/4
62738 Summary
62746 Summary
62748 Summary
62817 Summary
62935 Summary
256622 Summary
256626 Summary
256631 Summary
304518 Summary

Published 3/3
62678 Summary
62747 Summary
62869 Summary
62888 Summary
256629 Summary
256696 Summary

I'll have to double-check our selection criteria even though nothing should have changed in the interim publishing subset.

BZDATETIME::2010-03-08 15:32:44
BZCOMMENTOR::Volker Englisch
BZCOMMENT::20

The line that restricted the documents to only pick up the new Summaries must have gotten deleted in the SQL statement for the Interim-Export set when I made the previous changes.

This has been fixed and copied to production.

BZDATETIME::2010-03-15 14:56:08
BZCOMMENTOR::Volker Englisch
BZCOMMENT::21

A week of nightly publishing jobs and a weekly job finished successfully.

Closing issue.