CDR Tickets

BZDATETIME::2010-02-12 18:10:04
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

(In reply to comment #0)

> It looks like for these
> records, you need to manually delete the rows in the ctgov_import table.

Done.

> For the records below, they do not have corresponding CTGov documents. They are
> mostly withdrawn trials. I am not sure if they fall into the same category as
> trials listed above but it will be good to have some guidance on that.

I believe these should work the same way. I have deleted all 22 rows. Please check on Monday to ensure that new CTGovProtocol documents are created.

BZDATETIME::2010-02-18 09:48:17
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::2

(In reply to comment #1)
> (In reply to comment #0)
>
> > It looks like for these
> > records, you need to manually delete the rows in the ctgov_import table.
>
> Done.
>
> > For the records below, they do not have corresponding CTGov documents. They are
> > mostly withdrawn trials. I am not sure if they fall into the same category as
> > trials listed above but it will be good to have some guidance on that.
>
> I believe these should work the same way. I have deleted all 22 rows. Please
> check on Monday to ensure that new CTGovProtocol documents are created.

It appears only one of the trials was converted 561266. The rest do not show up on the report. I retrieved a few of the records and they remain inscope protocols.

BZDATETIME::2010-02-18 09:56:54
BZCOMMENTOR::Bob Kline
BZCOMMENT::3

(In reply to comment #2)

> It appears only one of the trials was converted 561266. The rest do not show up
> on the report. I retrieved a few of the records and they remain inscope
> protocols.

Looks like they're sitting in the queue waiting to be reviewed.

BZDATETIME::2010-02-18 10:08:47
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::4

(In reply to comment #3)
> (In reply to comment #2)
>
> > It appears only one of the trials was converted 561266. The rest do not show up
> > on the report. I retrieved a few of the records and they remain inscope
> > protocols.
>
> Looks like they're sitting in the queue waiting to be reviewed.

Hmmm. On which report will I find them? They usually show up in the Statistics Report - Import report the next day which would have been the 13th and the only one I see for that day is 561266. Besides, the InScope documents remain inscope documents instead of being converted CTGov documents for all the ones I checked.

BZDATETIME::2010-02-18 10:14:39
BZCOMMENTOR::Bob Kline
BZCOMMENT::5

They're included in the list of 32 protocols to be reviewed (CT.gov Protocols / Review New Protocols). As for why the CDR documents are still InScope: the download program just downloads; the import program is the one which creates the new CTGovProtocol version, and it's waiting for the review. You might not think of these protocols as "new" but since we deleted the rows in the ctgov_import table that's how the software sees them. It will use the existing documents, though, once they've been reviewed.

BZDATETIME::2010-02-18 12:46:21
BZCOMMENTOR::Bob Kline
BZCOMMENT::6

Bob will investigate to find out why these went down a different path than other transferred protocols.

BZDATETIME::2010-02-26 10:58:23
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::7

(In reply to comment #6)
> Bob will investigate to find out why these went down a different path than
> other transferred protocols.

Bob, I am just checking to see if you have a solution for this issue? When we reviewed and imported the records, they created duplicates instead of converting into CTgov documents.

BZDATETIME::2010-02-26 11:02:44
BZCOMMENTOR::Bob Kline
BZCOMMENT::8

At the moment, I'm wrestling with the Microsoft bug which is preventing us from storing mp3 files, since that task has a higher priority.

BZDATETIME::2010-02-26 12:47:58
BZCOMMENTOR::Lakshmi Grama
BZCOMMENT::9

William, is this impacting CTGOV/Inscope trial processing? If so, I would say this needs to be looked at as soon as possible.

BZDATETIME::2010-02-26 13:37:47
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::10

(In reply to comment #9)
> William, is this impacting CTGOV/Inscope trial processing? If so, I would say
> this needs to be looked at as soon as possible.

It is not holding up anything at this point except that some of them have been transferred in the PRS a long time ago but cancer.gov does not have the current versions for those that have been updated in CTGov since the transfer. So there are discrepancies between the versions on Cancer.gov and the ones on CTGov.

BZDATETIME::2010-03-01 11:38:16
BZCOMMENTOR::Lakshmi Grama
BZCOMMENT::11

What William describes indicates that this is high priority - adjusting priority since issue about media storage is now being lowered in priority.

BZDATETIME::2010-03-01 13:07:47
BZCOMMENTOR::Bob Kline
BZCOMMENT::12

(In reply to comment #6)
> Bob will investigate to find out why these went down a different path than
> other transferred protocols.

As far as I can tell, that happened because CIAT blocked the original InScopeProtocol documents before the transfer process completed (in contrast with the way things were handled when we tested and first deployed the solution to issue #4734). This prevented the download program from finding the value to insert into the cdr_id column of the newly created ctgov_import row (as I recall, we had to restrict our query to the active InScopeProtocol documents because we would otherwise sometimes find multiple InScopeProtocol documents with the NCT ID in the OtherID block). You can't block the InScopeProtocol documents until at least after the CT.gov download program has created the new ctgov_import row (if not waiting until the CT.gov import program has created the new CTGovProtocol document).

BZDATETIME::2010-03-01 13:37:33
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::13

(In reply to comment #12)
> (In reply to comment #6)
> > Bob will investigate to find out why these went down a different path than
> > other transferred protocols.
>
> As far as I can tell, that happened because CIAT blocked the original
> InScopeProtocol documents before the transfer process completed (in contrast
> with the way things were handled when we tested and first deployed the solution
> to issue #4734).

You're right. I checked a few and all of them had been blocked in the past. What we should have done was to unblock them.

Next Steps for CIAT:

1. Unblock the inscope documents
2. Block the corresponding ctgov documents
3. Inform you when completed.

Are the above steps the appropriate steps to take?

BZDATETIME::2010-03-01 13:58:45
BZCOMMENTOR::Bob Kline
BZCOMMENT::14

(In reply to comment #13)

> Next Steps for CIAT:
>
> 1. Unblock the inscope documents
> 2. Block the corresponding ctgov documents
> 3. Inform you when completed.
>
> Are the above steps the appropriate steps to take?

I think so. Then I'll need to clear out the ctgov_import rows again, after which the download program will create new rows, this time with the cdr_id columns set appropriately (we hope). Then after CIAT clears them in the review process they should be imported using the original CDR IDs. Let's pause before the step where CIAT clears them for import to make sure the cdr_id column is set correctly for the new rows. I believe you said not all of the documents originally identified for this issue had this problem. Please make sure and let me know which rows I need to delete.

BZDATETIME::2010-03-01 14:37:47
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::15

(In reply to comment #14)
> (In reply to comment #13)
>
> > Next Steps for CIAT:
> >
> > 1. Unblock the inscope documents
> > 2. Block the corresponding ctgov documents
> > 3. Inform you when completed.
> >
%

BZDATETIME::2010-03-01 14:41:56
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::16

(In reply to comment #14)
> (In reply to comment #13)
>
> > Next Steps for CIAT:
> >
> > 1. Unblock the inscope documents
> > 2. Block the corresponding ctgov documents
> > 3. Inform you when completed.
> >
%

BZDATETIME::2010-03-01 14:45:16
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::17

(In reply to comment #14)
> (In reply to comment #13)
>
> > Next Steps for CIAT:
> >
> > 1. Unblock the inscope documents
> > 2. Block the corresponding ctgov documents
> > 3. Inform you when completed.
> >
> > Are the above steps the appropriate steps to take?
>
> I think so. Then I'll need to clear out the ctgov_import rows again, after
> which the download program will create new rows, this time with the cdr_id
> columns set appropriately (we hope). Then after CIAT clears them in the review
> process they should be imported using the original CDR IDs. Let's paus

BZDATETIME::2010-03-01 16:24:51
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::18

Sorry for comments 15-17. I was having connectivity problems.

(In reply to comment #14)
> (In reply to comment #13)
>
> > Next Steps for CIAT:
> >
> > 1. Unblock the inscope documents
> > 2. Block the corresponding ctgov documents
> > 3. Inform you when completed.
> >
> > Are the above steps the appropriate steps to take?
>
> I think so. Then I'll need to clear out the ctgov_import rows again, after
> which the download program will create new rows, this time with the cdr_id
> columns set appropriately (we hope). Then after CIAT clears them in the review
> process they should be imported using the original CDR IDs. Let's pause >before

If what created the problem (of not displaying on the right report) had to do with the blocking of the documents, then I assume the 'converted' trials will not end up on the regular ctgov review page but will rather end up on the "statistics report - import" report, right?

> the step where CIAT clears them for import to make sure the cdr_id column is
> set correctly for the new rows. I believe you said not all of the documents
> originally identified for this issue had this problem. Please make sure and
> let me know which rows I need to delete.

Yes. only one of the them did not have this issue -

561266 NCT00194779 452123

(In reply to comment #13)
> (In reply to comment #12)
> > (In reply to comment #6)
> > > Bob will investigate to find out why these went down a different path than
> > > other transferred protocols.
> >
> > As far as I can tell, that happened because CIAT blocked the original
> > InScopeProtocol documents before the transfer process completed (in contrast
> > with the way things were handled when we tested and first deployed the solution
> > to issue #4734).
>
> You're right. I checked a few and all of them had been blocked in the past.
> What we should have done was to unblock them.
>
> Next Steps for CIAT:
>
> 1. Unblock the inscope documents

This is done

> 2. Block the corresponding ctgov documents

This is done.
> 3. Inform you when completed.
>

BZDATETIME::2010-03-02 11:42:31
BZCOMMENTOR::Bob Kline
BZCOMMENT::19

I have cleared out the rows from the import tables again. Next step is for the download job to recreate the rows. Please hold off on the review approval process for import until I have had a chance to confirm that the cdr_id column is populated correctly. I will let you know when I have done that.

BZDATETIME::2010-03-03 14:45:51
BZCOMMENTOR::Bob Kline
BZCOMMENT::20

Last night's (or rather, this morning's) download job didn't ever finish, so I ran it by hand with the same set of trials the download job pulled down, but hadn't finished queuing up in the ctgov_import table. If you don't block the InScopeProtocol documents the disposition is set to "Import requested" so there's no review step involved. Looks like all 14 of the trials in the first set have the correct CDR IDs associated with them. All but one of the trials in the second set were bypassed, because they have a status of "Withdrawn" which we're supposed to skip. Tomorrow morning's import job should import the 14 queued trials, using the CDR IDs for the corresponding InScopeProtocol documents.

BZDATETIME::2010-03-04 10:03:12
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::21

(In reply to comment #20)
> Last night's (or rather, this morning's) download job didn't ever finish, so I
> ran it by hand with the same set of trials the download job pulled down, but
> hadn't finished queuing up in the ctgov_import table. If you don't block the
> InScopeProtocol documents the disposition is set to "Import requested" so
> there's no review step involved. Looks like all 14 of the trials in the first
> set have the correct CDR IDs associated with them. All but one of the trials
> in the second set were bypassed, because they have a status of "Withdrawn"
> which we're supposed to skip. Tomorrow morning's import job should import the
> 14 queued trials, using the CDR IDs for the corresponding InScopeProtocol
> documents.

Verified. All imported correctly except for the ones with withdrawn status.

We will block the duplicate records that were created earlier and send them to you so that they can be deleted.

Issue closed. Thank you!