CDR Tickets

Attachment transfer of clinical center trials.xls has been added with description: clinical center trials

BZDATETIME::2010-06-30 15:55:55
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

What's the relationship between the trials for this issue and those for #4824?

BZDATETIME::2010-06-30 16:07:01
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::2

(In reply to comment #1)
> What's the relationship between the trials for this issue and those for #4824?

They both deal with transfer trials but in this case (OCECDR-3184), the Clinical Center sent a request to NLM to transfer these selected trials into their account but Lakshmi asked us to rather initiate the transfer process because when NLM initiates the transfer, they will create problems for us down the line.

BZDATETIME::2010-06-30 17:44:36
BZCOMMENTOR::Bob Kline
BZCOMMENT::3

What's the goal here? Do you want the orange trials to have the documents for the NCT IDs in column B to be imported using the CDR IDs in the orange comments, and for the rest of the trials the documents for the NCT IDs in column B to be imported using the CDR ID in column D (or rather, the first CDR ID column D, since one of the rows has two CDR IDs in column D)?

BZDATETIME::2010-06-30 18:24:53
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::4

(In reply to comment #3)
> What's the goal here? Do you want the orange trials to have the documents for
> the NCT IDs in column B to be imported using the CDR IDs in the orange
> comments, and for the rest of the trials the documents for the NCT IDs in
> column B to be imported using the CDR ID in column D (or rather, the first CDR
> ID column D, since one of the rows has two CDR IDs in column D)?

That is correct.

BZDATETIME::2010-07-01 08:35:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::5

(In reply to comment #4)
> (In reply to comment #3)
> > What's the goal here? Do you want the orange trials to have the documents for
> > the NCT IDs in column B to be imported using the CDR IDs in the orange
> > comments, and for the rest of the trials the documents for the NCT IDs in
> > column B to be imported using the CDR ID in column D (or rather, the first CDR
> > ID column D, since one of the rows has two CDR IDs in column D)?
>
> That is correct.

Then why do the non-orange comments say "Inscope to be blocked at import"? If you block the CDR IDs from column D, won't you be blocking the CTGovProtocol document we'd be importing into that slot? If you're trying to confuse me even more than I already am, you're too late: I'm already as confused about these duplicate/transferred trials as any human could possibly be. :-)

BZDATETIME::2010-07-01 08:38:29
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::6

(In reply to comment #5)

>
> Then why do the non-orange comments say "Inscope to be blocked at import"? If
> you block the CDR IDs from column D, won't you be blocking the CTGovProtocol
> document we'd be importing into that slot? If you're trying to confuse me even
> more than I already am, you're too late: I'm already as confused about these
> duplicate/transferred trials as any human could possibly be. :-)

Sorry, I should have taken all the comments out. They were not meant for you. They were provided by Lockheed when they prepared the spreadsheet and they are not accurate

BZDATETIME::2010-07-01 08:55:58
BZCOMMENTOR::Bob Kline
BZCOMMENT::7

(In reply to comment #0)

> 1. Most of the trials have been marked as duplicates in the CDR (highlighted
> yellow in the spreadsheet). That is, they are on the duplicate table.

Which duplicate table are you referring to?

BZDATETIME::2010-07-01 08:59:00
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::8

(In reply to comment #7)
> (In reply to comment #0)
>
> > 1. Most of the trials have been marked as duplicates in the CDR (highlighted
> > yellow in the spreadsheet). That is, they are on the duplicate table.
>
> Which duplicate table are you referring to?

I am referring to the records marked as duplicate report:

CIAT/OCCM Staff > CTGov Protocols > Records Marked Duplicate

BZDATETIME::2010-07-01 09:16:56
BZCOMMENTOR::Bob Kline
BZCOMMENT::9

Is it true that the term 'duplicate' is being applied variously to different situations? For example, do we sometimes mean NLM has made a mistake and assigned two separate NCT IDs to the same trial, and at other times mean that they created a document for the trial and so did PDQ, independently, and at still other times mean that two InScopeProtocol documents were created for the same trial? What are all of the possible scenarios in which we're using the term "duplicate" for trials?

BZDATETIME::2010-07-01 10:14:55
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::10

(In reply to comment #9)
> Is it true that the term 'duplicate' is being applied variously to different
> situations? For example, do we sometimes mean NLM has made a mistake and
> assigned two separate NCT IDs to the same trial, and at other times mean that
> they created a document for the trial and so did PDQ, independently, and at

It is not that NLM made a mistake and assigned two separate NCT IDs to the same trial. Rather, different institutions can register the same trial with slightly different titles, with NLM. Also users used to submit their trials to PDQ and also submit to NLM. Because we make changes to the titles to match PDQ specifications before registering with NLM, such duplicate registrations may not be caught at the time of registration. When the duplicate registration is identified after registration, one of the NCT IDs is either made obsolete or marked an alias by NLM.

> still other times mean that two InScopeProtocol documents were created for the
> same trial? What are all of the possible scenarios in which we're using the
> term "duplicate" for trials?

It is possible to also use ‘duplicate’ for two InScopeProtocol documents when we receive the same trial from different centers (or from the same institution) collaborating on a particular trial. Usually, this kind of duplicate would be caught at the time of processing the trial so another InScopeProtocol would not be created but there are cases where two InScopeProtocol documents were created.

Here are the possible scenarios:

1a. PDQ has a trial and a CTGov Protocol is imported or marked as a duplicate at the time of review.
1b. PDQ Registers a trial for a particular center in CTGov and the center also registers the same trial in CTGov. The trial may or may not have been imported into the CDR.

2. We import and publish a CTGov protocol and later come across the same trial under a different NCT ID.

3. PDQ has received a trial and another center submits the same trial to PDQ. Two InScopeProtocols may or may not have been created.

For the purposes of transfer protocols, when we use the word duplicate to refer to 1a and 1b above.

1. There are two records in the CDR - 1 CTGOV and the other InScope
2. There is an existing InScope trial, we received a new CTGov trial and identified it at the time of review as a duplicate. Under this particular case we will say that it is marked as a duplicate or it is in the ctgov duplicate report.

BZDATETIME::2010-07-01 16:03:01
BZCOMMENTOR::Bob Kline
BZCOMMENT::11

CIAT appears to have already jumped ahead on the transfer for at least one of the trials listed on the spreadsheet. Andrea Jackson added the new owner to the transfer block for CDR581356 yesterday, and this morning's download and import jobs resulted in a CTGovProtocol version being created in place for that document. CIAT should do the same for CDR485379 and CDR482415 (the next two rows on the spreadsheet), as well as CDR465482 (the first of the six trials listed at the end of the spreadsheet for which CTGovProtocol documents have been created and blocked. For those six trials (assuming these tests are successful), the plan is to leave the existing CTGovProtocol documents blocked, and let the download and import programs create a new CTGovProtocol document version in place over the InScopeProtocol document. Tomorrow we will take a look at the three documents and see if they are processed as we expect they will be, and then decide what to do next.

BZDATETIME::2010-07-01 17:53:35
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::12

(In reply to comment #11)

We have added the transfer blocks to the three documents. I will check and post a comment tomorrow morning.

BZDATETIME::2010-07-02 08:59:27
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::13

(In reply to comment #12)
> (In reply to comment #11)
>
> We have added the transfer blocks to the three documents. I will check and post
> a comment tomorrow morning.

I just checked and it looks like all of them transformed correctly. CDR465482 also transformed correctly.

It looks like it is OK for us to add the transfer blocks to all the others at this point?

BZDATETIME::2010-07-02 11:27:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::14

Yes, go ahead. Monitor the results carefully, please.

BZDATETIME::2010-07-08 17:00:06
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::15

We discussed the fact that these trials do not need to have ownership transferred and should be handled differently. Here are the steps we talked about:

1. Lockheed will put a comment in the in scope trial that it will be overwritten with the Clinical Center CTgov trial (with the original NCT ID).

2. Bob will update the CTgov import table to put the CDR ID of the in scope trial

3. CIAT will communicate to CT.gov about unblocking the Clinical Center version and blocking the NCI version

4. When the Clinical Center CTgov version overwrites the in scope version, CIAT will need to go into these and update them (copy and paste) and add a comment that these have been overwritten by the Clinical Center trial.

We will test this on 1 trial first to make sure it works as hoped.

Please feel free to add more detail to my comments or correct them if I missed something.

BZDATETIME::2010-07-09 07:57:19
BZCOMMENTOR::Bob Kline
BZCOMMENT::16

Once more, it seems that every time I take a step forward in understanding what's going on, I get pushed back two steps. My understanding from yesterday's meeting was that NLM has the NCT IDs for the Clinical Center's version of the trials blocked. But when I went to look closely at the two trials I picked (deciding to do a couple rather than just one) I saw that NLM's been sending those trials to us every day. Have I missed something else?

BZDATETIME::2010-07-09 12:55:01
BZCOMMENTOR::Lakshmi Grama
BZCOMMENT::17

Which two trials?

(In reply to comment #16)
> Once more, it seems that every time I take a step forward in understanding
> what's going on, I get pushed back two steps. My understanding from
> yesterday's meeting was that NLM has the NCT IDs for the Clinical Center's
> version of the trials blocked. But when I went to look closely at the two
> trials I picked (deciding to do a couple rather than just one) I saw that NLM's
> been sending those trials to us every day. Have I missed something else?

BZDATETIME::2010-07-09 13:44:35
BZCOMMENTOR::Bob Kline
BZCOMMENT::18

NCT00273910 and NCT00302159.

BZDATETIME::2010-07-09 14:04:50
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::19

I also went back and checked CDR485379 and CDR482415, the trials we used in our test cases. I checked clinicaltrials.gov and these two trials are actually owned by the NIH CC. By ownership, I mean the CC is the information provider and NCI is the Sponsor of the trials:
http://www.clinicaltrials.gov/ct2/show/NCT00326495?term=NCT00343772&rank=1
http://www.clinicaltrials.gov/ct2/show/NCT00321555?term=NCT00337311&rank=1

This appears to be the case for all the other 10 records (in the spreadsheet) I checked. I will be checking all the records to determine if they all follow the same pattern.

What is even more puzzling is that, the InScope protocols for all of these records are not blocked so technically, they should be on clinicaltrials.gov. However, it appears that clinicaltrials.gov has rather blocked our records and published the CC’s records. I am making this assumption because, typically, any inscope record we have not explicitly blocked from publication or blocked from ctgov, is registered and updated in clinicaltrials.gov but it appears not to be the case for these documents in the spreadsheet.

What I see from this situation is that, since the tests we did with the 3 records worked by just adding the ctgov transfer blocks, if all of these trials (after we have checked), fall into the same situation, it may mean that they will be transferred normally without any further programmatic intervention.

I will post another comment after I have checked to see who the information provider is on clinicaltrials.gov for all of the records.

BZDATETIME::2010-07-09 14:13:58
BZCOMMENTOR::Lakshmi Grama
BZCOMMENT::20

As you say, CTGOV may have jumped the gun despite our asking them to wait. They want to turn these around really quickly!

BZDATETIME::2010-07-09 14:32:11
BZCOMMENTOR::Bob Kline
BZCOMMENT::21

Do you want me to plug the InScope CDR ID for a couple of these into the row for the corresponding Clinical Center NCT ID, then manually stuff the XML we got from NLM this morning for the two trials into the XML column, set the disposition to 'import requested' and run another import job by hand? If I sound as if I know what's going on or what will happen, don't believe it. :-)

BZDATETIME::2010-07-09 15:55:29
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::22

(In reply to comment #21)
> Do you want me to plug the InScope CDR ID for a couple of these into the row
> for the corresponding Clinical Center NCT ID, then manually stuff the XML we
> got from NLM this morning for the two trials into the XML column, set the
> disposition to 'import requested' and run another import job by hand? If I
> sound as if I know what's going on or what will happen, don't believe it. :-)

It looks like we may have to look further into this before going forward because there is an issue with at least 2 of the trials we tested (482415 and 485379).

For 482415, the NCT ID in the inscope document is NCT00337311 and when it converted, the ctgov protocol rightly had the same NCT ID of NCT00337311. However, NCT00337311 is marked as the obsolete NCT ID on clinicaltrials.gov,

http://www.clinicaltrials.gov/ct2/show/NCT00321555?term=NCT00337311&rank=1

(which may mean that we will not receive any updates to the trial) and it looks like because of that about July 06, a trial with the active NCT ID of the trial showed up on the ctgov review page ( we have not imported or marked as a duplicate yet) l, which is - NCT00321555.

The second trial - CDR485379 may eventually end up with the same issue (of receiving another trial in the review page with the active NCT ID) because the active NCT ID on clinicaltrials.gov is not what is currently in the converted ctgov document. NCT00343772 is what is in the converted ctgov document and NCT00326495 is the active NCT ID on clinicaltrials.gov with NCT00343772 marked as obsolete.

Bob, do you have any suggestions of how to deal with the ones we have already converted (above)?

It looks like going forward; we may have to manually change the NCT IDs in the inscope documents to the active ones on ctgov before we do anything?

BZDATETIME::2010-07-12 11:17:33
BZCOMMENTOR::Bob Kline
BZCOMMENT::23

(In reply to comment #22)

> Bob, do you have any suggestions of how to deal with the ones we have already
> converted (above)?

Rube Goldberg would be proud. :-)

You want CDR485379 to be updated from NCT00326495 and CDR482415 to be updated from NCT00321555, right? If so, what would you say to having me manually set the cdr_id, title, xml, downloaded, changed, phase, and disposition columns for the two rows in ctgov_import for those NCT IDs, and mark the rows for NCT00343772 and NCT00337311 with disposition of duplicate, and then run an import job?

> It looks like going forward; we may have to manually change the NCT IDs in the
> inscope documents to the active ones on ctgov before we do anything?

Why?

BZDATETIME::2010-07-13 10:47:37
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::24

(In reply to comment #23)
> (In reply to comment #22)
>
> > Bob, do you have any suggestions of how to deal with the ones we have already
> > converted (above)?
>
> Rube Goldberg would be proud. :-)
>
> You want CDR485379 to be updated from NCT00326495 and CDR482415 to be updated
> from NCT00321555, right?

That is correct.

>If so, what would you say to having me manually set
> the cdr_id, title, xml, downloaded, changed, phase, and disposition columns for
> the two rows in ctgov_import for those NCT IDs, and mark the rows for
> NCT00343772 and NCT00337311 with disposition of duplicate, and then run an
> import job?
>
I believe this will solve the problem.

> > It looks like going forward; we may have to manually change the NCT IDs in the
> > inscope documents to the active ones on ctgov before we do anything?
>
> Why?

The reason is that the NCT IDs in the current inscope protocols are obsolete (I have not checked all of them yet) and I was concerned that, if we decide to go with the option of CIAT adding the CTGov Transfer blocks, the conversion will not work or it may work but may add the obsolete NCT IDs like in the two cases above.

It looks like the next step is to try your suggestion in comment #21 with 1 or 2 protocols.

BZDATETIME::2010-07-13 12:12:06
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::25

I checked the remainder of the trials in the spreadsheet and they all have the NCT IDs of the Clinical Center as the active NCT ID on clinicaltrials.gov. The NCT ID in the InScope protocol is the obsolete NCT ID. This is just a confirmation of that NLM has activated the Clinical Center's trials and blocked PDQ trials..

BZDATETIME::2010-07-13 18:04:06
BZCOMMENTOR::Bob Kline
BZCOMMENT::26

(In reply to comment #24)

> I believe this will solve the problem.

Please take a look at CDR485379 and CDR482415 on Franck; they should have data from NCT00326495 and NCT00321555 respectively. If they look OK I'll do the same thing on Bach tomorrow (or whenever you get back to me).

> > > It looks like going forward; we may have to manually change the NCT IDs in
> > > the inscope documents to the active ones on ctgov before we do anything?
> >
> > Why?
>
> The reason is that the NCT IDs in the current inscope protocols are obsolete (I
> have not checked all of them yet) and I was concerned that, if we decide to go
> with the option of CIAT adding the CTGov Transfer blocks, the conversion will
> not work or it may work but may add the obsolete NCT IDs like in the two cases
> above.

Can't you fix problems like that after the import?

BZDATETIME::2010-07-14 11:49:06
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::27

(In reply to comment #26)
> (In reply to comment #24)
>
> > I believe this will solve the problem.
>
> Please take a look at CDR485379 and CDR482415 on Franck; they should have data
> from NCT00326495 and NCT00321555 respectively. If they look OK I'll do the
> same thing on Bach tomorrow (or whenever you get back to me).
>

They both look good. Please proceed with the next step. Thanks!

> Can't you fix problems like that after the import?

Actually, there will not be the need for us to do anything if the trials are converted the same way as above.

BZDATETIME::2010-07-14 12:39:54
BZCOMMENTOR::Bob Kline
BZCOMMENT::28

(In reply to comment #27)

> They both look good. Please proceed with the next step. Thanks!

The rows for the the same two trials have been doctored on Bach, and I ran the import job. Please examine the results carefully.

BZDATETIME::2010-07-14 13:52:45
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::29

(In reply to comment #28)
> (In reply to comment #27)
>
> > They both look good. Please proceed with the next step. Thanks!
>
> The rows for the the same two trials have been doctored on Bach, and I ran the
> import job. Please examine the results carefully.

Verified.

Please don't run the import job the remaining trials yet. We need to add comments to the InScope documents which will be copied over into the ctgov version after the conversion.

I will let you know when to run the import job.

BZDATETIME::2010-07-26 10:54:03
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::30

(In reply to comment #29)
> (In reply to comment #28)

> Please don't run the import job the remaining trials yet. We need to add
> comments to the InScope documents which will be copied over into the ctgov
> version after the conversion.
>
> I will let you know when to run the import job.

Comments have been added to the remaining trials. You may proceed with the changes and import job.

BZDATETIME::2010-07-26 14:00:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::31

For the six trials at the bottom of the spreadsheet, the comments appear to imply that you want the NIHCC documents to continue to be imported as the existing CTGovProtocol documents. Is that true, or do you prefer that they be imported using the CDR IDs of the InScopeProtocol documents?

BZDATETIME::2010-07-26 14:07:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::32

For the first trial on the spreadsheet (08-C-0033), which of the two InScopeProtocol documents do you want to use as the target for import of NCT00720187?

BZDATETIME::2010-07-26 14:39:39
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::33

(In reply to comment #31)
> For the six trials at the bottom of the spreadsheet, the comments appear to
> imply that you want the NIHCC documents to continue to be imported as the
> existing CTGovProtocol documents. Is that true, or do you prefer that they be
> imported using the CDR IDs of the InScopeProtocol documents?

That is correct. Where we already have the correct ctgov document imported, we want to continue using those records. We only have to make sure that the trials are being updated by the ctgov import program. I was able to confirm that only two are being updated because they were recently updated. The two are:

350451 - NCT00073073 - Last updated 7/16/2010
371627 - NCT00082368 - Last updated 7/16/2010

The remaining 4 have not been updated by the import program for a while and even though they have the correct status compared to what is in ctgov, looking at the history of changes on clinicaltrials.gov, there appear to have been updates that we should have received. For the remaining 4, is it possible for you to also run an import job for them just to make sure we are getting all the updates? Or is there another way to know if we are not receiving updates we should be receiving?

BZDATETIME::2010-07-26 14:44:30
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::34

(In reply to comment #32)
> For the first trial on the spreadsheet (08-C-0033), which of the two
> InScopeProtocol documents do you want to use as the target for import of
> NCT00720187?

Please use 581356. However 581356 is now a ctgov document. However, it converted with the 'wrong' NCT # NCT00589290. The NCT ID to use for updating 581356 is NCT00720187.

BZDATETIME::2010-07-26 15:27:32
BZCOMMENTOR::Bob Kline
BZCOMMENT::35

I included the four trials at the bottom of the spreadsheet which you said weren't already importing (they had been marked as duplicates). I was unable to include 08-C-0033, because NLM is no longer sending us NCT00720187.

Attachment Request4871TrialsSet2.log has been added with description: Log from script to fix import mappings

BZDATETIME::2010-07-26 15:39:23
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::36

(In reply to comment #35)
> Created attachment 1960 [details]
> Log from script to fix import mappings
>
> I included the four trials at the bottom of the spreadsheet which you said
> weren't already importing (they had been marked as duplicates). I was unable
> to include 08-C-0033, because NLM is no longer sending us NCT00720187.

NLM marked NCT00720187 (the clinical center's NCT ID) as an alias instead of our NCT ID (NCT00589290). Meanwhile, I checked 581356 (08-C-003 , NCT00589290) and it is receiving updates so we are good on that one.

BZDATETIME::2010-07-27 09:12:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::37

Attachment Request4871WorkingFiles.zip has been added with description: Attaching these working files from "fix" runs so they'll be preserved

BZDATETIME::2010-07-27 12:17:35
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::38

I reviewed the list hoping see that all of them have transformed but some of them have not. They are still in the InScope state. Was I wrong in expecting that they have all been transformed?

BZDATETIME::2010-07-27 14:43:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::39

I believe what happened is that I didn't run the import job immediately after fixing the rows in the import table, so another download job happened in between the fix and the import. I'm not sure how that ended up with the trials being marked as duplicate again, but I think they should be CTGovProtocol documents now. Please review them to confirm.

BZDATETIME::2010-07-27 15:22:58
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::40

(In reply to comment #39)
> I believe what happened is that I didn't run the import job immediately after
> fixing the rows in the import table, so another download job happened in
> between the fix and the import. I'm not sure how that ended up with the trials
> being marked as duplicate again, but I think they should be CTGovProtocol
> documents now. Please review them to confirm.

Confirmed. They are all ctgov protocols now. Also, the 4 records at the end of the list have been updated. Thanks!

I am closing this issue now.