CDR Tickets

Issue Number 3192
Summary Identify and fix ctgov trials not being updated
Created 2010-07-19 12:11:14
Issue Type Improvement
Submitted By Osei-Poku, William (NIH/NCI) [C]
Assigned To Kline, Bob (NIH/NCI) [C]
Status Closed
Resolved 2010-09-30 11:33:21
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.107520
Description

BZISSUE::4879
BZDATETIME::2010-07-19 12:11:14
BZCREATOR::William Osei-Poku
BZASSIGNEE::Bob Kline
BZQACONTACT::William Osei-Poku

(Alan, since Bob is on vacation, I am assigning this to you :-) ).

Here is the initial email from Lakshmi
________________________________________
From: Grama, Lakshmi (NIH/NCI) [E]
Sent: Friday, July 16, 2010 5:05 PM
To: Osei-Poku, William (NIH/NCI) [C]; Kline, Robert (NCI)
Cc: Beckwith, Margaret (NIH/NCI) [E]
Subject: Please check why we don't have updated record for this trial

On CTGOV this trial was last updated in March
http://clinicaltrials.gov/show/NCT00705393

But it seems like our version is from January
http://www.cancer.gov/search/viewclinicaltrials.aspx?version=healthprofessional&cdrid=598330

Lakshmi

Comment entered 2010-07-19 12:13:59 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-07-19 12:13:59
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::1

After researching further into this problem, I found out that this similar to the NIH CC trials that we recently discussed. NL M transferred the trial to the responsible party before we converted the trial.

This means Bob (or Alan? Since Bob is on vacation) will have to manually update the ctgov_import table to direct updates from NCT01068327 to CDR0000598330. We will also have to remove the duplicate designation for NCT01068327 in the ctgov_import table.

Our CDR record actually has the obsolete NCT ID. When the InScope document converted, it converted with the now obsolete NCT ID, which was probably the active NCT ID at that time.

So it looks like this is what needs to be done:
CDR0000598330 currently has NCT NCT00705393 as the active NCT ID. But NCT00705393 is obsolete on clinicaltrials.gov. The active NCT ID is NCT01068327 but it is marked as a duplicate in the ctgov_import table. We need to direct all updates from NCT01068327 to CDR0000598330 and also remove the duplicate designation for NCT01068327 from the ctgov_import table.

Comment entered 2010-07-21 11:32:42 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-07-21 11:32:42
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::2

I changed the QA from Lakshmi to me.

Comment entered 2010-07-26 14:47:43 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-07-26 14:47:43
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::3

Re-assigning to Bob.

As discussed in the emails, there may be other trials that are not being updated like this trial. Perhaps we should discuss in the next meeting how to identify those trials.

Comment entered 2010-07-27 07:35:22 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-07-27 07:35:22
BZCOMMENTOR::Bob Kline
BZCOMMENT::4

CDR598330 is now being updated from NCT01068327.

Comment entered 2010-07-27 12:20:44 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-07-27 12:20:44
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::5

(In reply to comment #4)
> CDR598330 is now being updated from NCT01068327.

Verified. Thanks!

I will leave this issue open until after the discussion in comment# 3.

Comment entered 2010-07-27 13:45:22 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-07-27 13:45:22
BZCOMMENTOR::Bob Kline
BZCOMMENT::6

You might want to change the title of the issue if you want to broaden its scope.

Comment entered 2010-07-27 15:51:44 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-07-27 15:51:44
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::7

(In reply to comment #6)
> You might want to change the title of the issue if you want to broaden its
> scope.

Done. Thanks!

Comment entered 2010-07-27 16:25:53 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-07-27 16:25:53
BZCOMMENTOR::Bob Kline
BZCOMMENT::8

There are 295 trials in the ctgov_import table on Bach with disposition of 'imported' for which NLM is no longer sending us the trial information. I have attached the list of those trials, along with the CDR IDs of the CTGovProtocol documents into which we at one time were importing those trials.

Comment entered 2010-07-27 16:25:53 by Kline, Bob (NIH/NCI) [C]

Attachment OrphanedCTGovProtocols.txt has been added with description: Trials NLM is no longer sending us

Comment entered 2010-07-27 16:46:41 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-07-27 16:46:41
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::9

(In reply to comment #8)
> Created attachment 1962 [details]
> Trials NLM is no longer sending us
>
> There are 295 trials in the ctgov_import table on Bach with disposition of
> 'imported' for which NLM is no longer sending us the trial information. I have
> attached the list of those trials, along with the CDR IDs of the CTGovProtocol
> documents into which we at one time were importing those trials.
Thanks!

Could you also include the OverallStatus status of the trials? If you can provide both the status from NLM and also what is currently the OverallStatus of the CTGovProtocol, and sort them by status, that will be helpful. Also, if you can indicate whether the CTGovProtocol is blocked from publication or not, that will be helpful.

Comment entered 2010-07-28 13:39:48 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-07-28 13:39:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::10

(In reply to comment #9)

> Could you also include the OverallStatus status of the trials? If you can
> provide both the status from NLM and also what is currently the OverallStatus
> of the CTGovProtocol, and sort them by status, that will be helpful. Also, if
> you can indicate whether the CTGovProtocol is blocked from publication or not,
> that will be helpful.

Requested modifications implemented:

http://bach.nci.nih.gov/cgi-bin/cdr/FindOrphanedCTGovProtocols.py

Comment entered 2010-08-24 12:29:29 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-08-24 12:29:29
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::11

The attached spreadsheet contains the list of trials which do not have active NCT IDs. I provided a column for the Active NCT IDs. So for all the trials in this spreadsheet, the XML files that should be updating them should come from the active NCT IDs.

Comment entered 2010-08-24 12:29:29 by Osei-Poku, William (NIH/NCI) [C]

Attachment Active NCT IDs.xlsx has been added with description: Active NCT IDs

Comment entered 2010-08-24 13:34:49 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-08-24 13:34:49
BZCOMMENTOR::Bob Kline
BZCOMMENT::12

Should the NCT IDs found in the first column be marked as duplicates in the ctgov_import table?

Comment entered 2010-08-24 15:07:31 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-08-24 15:07:31
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::13

(In reply to comment #12)
> Should the NCT IDs found in the first column be marked as duplicates in the
> ctgov_import table?

Yes please.

Comment entered 2010-08-25 10:30:56 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-08-25 10:30:56
BZCOMMENTOR::Bob Kline
BZCOMMENT::14

The rows in the ctgov_import table have been adjusted, and the import job was run (on Bach):

!6760 Wed Aug 25 10:28:03 2010: Updated CDR257197 from NCT00038064
!6760 Wed Aug 25 10:28:09 2010: Updated CDR647638 from NCT00186576
!6760 Wed Aug 25 10:28:15 2010: Updated CDR558850 from NCT00352638
!6760 Wed Aug 25 10:28:24 2010: Updated CDR652493 from NCT00358501
!6760 Wed Aug 25 10:28:35 2010: Updated CDR472034 from NCT00389571
!6760 Wed Aug 25 10:28:39 2010: Updated CDR353367 from NCT00601380
!6760 Wed Aug 25 10:28:43 2010: Updated CDR512878 from NCT00608478
!6760 Wed Aug 25 10:28:45 2010: Updated CDR352391 from NCT00609076
!6760 Wed Aug 25 10:28:47 2010: Updated CDR587839 from NCT00612560
!6760 Wed Aug 25 10:28:49 2010: Updated CDR582858 from NCT00635284
!6760 Wed Aug 25 10:28:51 2010: Updated CDR599821 from NCT00923117
!6760 Wed Aug 25 10:28:53 2010: Updated CDR600867 from NCT00971789
!6760 Wed Aug 25 10:28:56 2010: Updated CDR583024 from NCT00972205
!6760 Wed Aug 25 10:28:58 2010: Updated CDR652221 from NCT00987753
!6760 Wed Aug 25 10:28:59 2010: Updated CDR652225 from NCT01004731
!6760 Wed Aug 25 10:29:01 2010: Updated CDR660166 from NCT01062503
!6760 Wed Aug 25 10:29:04 2010: Updated CDR625162 from NCT01095094

Comment entered 2010-08-25 13:38:35 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-08-25 13:38:35
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::15

For record NCT00216307 CDR451694, The correct/active NCT ID is NCT00243854. However, it has a duplicate record - 577786 with the same NCT ID of NCT00243854, but it is blocked from publication and yet it is the one that has been receiving the most recent updates. Could you direct all updates from NCT00243854 to 451694 instead of 577786? NCT00216307 should be the duplicate NCT ID for the record.

Comment entered 2010-08-25 14:08:07 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-08-25 14:08:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::16

Fixed & imported.

Comment entered 2010-08-25 16:33:13 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-08-25 16:33:13
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::17

We will force download the attached list of trials. These are trials that have status discrepancies. That is, trials that NLM's status is different from PDQ status.

Comment entered 2010-08-25 16:33:13 by Osei-Poku, William (NIH/NCI) [C]

Attachment Trials to force download _ status discrepancy.xlsx has been added with description: Status discrepancies

Comment entered 2010-08-27 11:15:08 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-08-27 11:15:08
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::18

We started the force download with the following 4 trials:

CDR0000525927 NCT00396331 - PDQ Status is Active / CTGov Status is Completed
CDR0000529062 NCT00411749 - PDQ Status is Closed / CTGov status is Completed
CDR0000567503 NCT00515788 - PDQ Status is Active / CTGov status is Completed
CDR0000570496 NCT00527488 - PDQ Status is Active / CTGov status is Completed

Publishable versions were created but their statuses were not updated so we continue to have status discrepancies.

Comment entered 2010-08-27 11:36:14 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-08-27 11:36:14
BZCOMMENTOR::Bob Kline
BZCOMMENT::19

There was a problem in the force part of this morning's download job. I'm running it again, and will do the import again when the download is finished.

Comment entered 2010-08-27 13:18:14 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-08-27 13:18:14
BZCOMMENTOR::Bob Kline
BZCOMMENT::20

Download and import are done. I believe the statuses are correct now.

Comment entered 2010-08-31 10:44:36 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-08-31 10:44:36
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::21

(In reply to comment #20)
> Download and import are done. I believe the statuses are correct now.

Yes. The statuses are correct and the others we marked for force download also have the correct statuses. We are now working through the ones that do not have any obvious discrepancies either in status or indexing.

Comment entered 2010-09-02 11:19:54 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-09-02 11:19:54
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::22

Some of the trials continue to say that they have already been marked for forced download when we attempt to force download them. We attempted to force download the following trials 2 days ago and we got the same message we got today.

NCT00366145 - Already marked for forced download
NCT00411645 - Already marked for forced download
NCT00128713 - Already marked for forced download
NCT00463385 - Already marked for forced download
NCT00314951 - Already marked for forced download

Comment entered 2010-09-02 11:29:49 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-09-02 11:29:49
BZCOMMENTOR::Bob Kline
BZCOMMENT::23

(In reply to comment #22)
> Some of the trials continue to say that they have already been marked for
> forced download when we attempt to force download them. We attempted to force
> download the following trials 2 days ago and we got the same message we got
> today.
>
> NCT00366145 - Already marked for forced download
> NCT00411645 - Already marked for forced download
> NCT00128713 - Already marked for forced download
> NCT00463385 - Already marked for forced download
> NCT00314951 - Already marked for forced download

Well, they are marked for forced download. Not all of them are queued up for re-import, because what we got from NLM this morning was unchanged from what we got the previous time.

Comment entered 2010-09-02 11:53:39 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-09-02 11:53:39
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::24

(In reply to comment #23)
> (In reply to comment #22)
> Well, they are marked for forced download. Not all of them are queued up for
> re-import, because what we got from NLM this morning was unchanged from what we
> got the previous time.

Okay. Thanks! For those trials, I believe we can safely assume that we have the most current versions from NLM.

Comment entered 2010-09-24 11:58:33 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-09-24 11:58:33
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::25

We have completed the forced download of the remaining trials.

Please modify the report to not display trials that have been blocked from publication and add it to the report menu under Management Reports and call it "CTGov trials not receiving updates:
CIAT/OCCM Staff > CTGOV Protocols > Management Reports > CTGov trials not receiving updates

Also, create a user group with appropriate rights to run the report and modify the software so that I can assign users to the group.

Comment entered 2010-09-30 10:48:08 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-09-30 10:48:08
BZCOMMENTOR::Bob Kline
BZCOMMENT::26

Report and new group (CTGov Orphan Report) installed on Bach.

Comment entered 2010-09-30 11:33:21 by Osei-Poku, William (NIH/NCI) [C]

BZDATETIME::2010-09-30 11:33:21
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::27

(In reply to comment #26)
> Report and new group (CTGov Orphan Report) installed on Bach.

Verified on Bach. Thanks!
Issue closed.

Attachments
File Name Posted User
Active NCT IDs.xlsx 2010-08-24 12:29:29 Osei-Poku, William (NIH/NCI) [C]
OrphanedCTGovProtocols.txt 2010-07-27 16:25:53
Trials to force download _ status discrepancy.xlsx 2010-08-25 16:33:13 Osei-Poku, William (NIH/NCI) [C]

Elapsed: 0:00:00.001629