CDR Tickets

BZDATETIME::2010-11-22 10:36:36
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

My picture of how this will be done is as follows: we get the trial document from NLM as part of the pile they send us in response to our query; we get the CDR ID from the org_study_id in the document; we look at the document represented by the CDR ID and see that it has the new ImportTrialFrom element with the value 'CTRP' which tells us not to import the document from NLM, but instead to queue up the trial for import as a CTRPProtocol document, folding in NLM's NCT ID as part of the data. At this point I have two questions:

1. Is this the picture everyone else has?
2. What happens if the new RP changes the org_study_id?

BZDATETIME::2010-12-07 14:24:32
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::2

(In reply to comment #1)

> 1. Is this the picture everyone else has?
> 2. What happens if the new RP changes the org_study_id?

During initial discussions of this issue, there was mention of a unique CTRP id which will be used to identify the trials (perhaps in combination with the CDR ID). In the absence of the unique CTRP id it looks like using the NCT ID as the unique identifier will be a good way to identify the trials. For the coop group trials we are about to import from CTRP, we already know their NCT IDs and they are unlikely to be altered by the RP.

BZDATETIME::2010-12-14 16:10:21
BZCOMMENTOR::Bob Kline
BZCOMMENT::3

(In reply to comment #2)
> (In reply to comment #1)
>
> > 1. Is this the picture everyone else has?
> > 2. What happens if the new RP changes the org_study_id?
>
> During initial discussions of this issue, there was mention of a unique CTRP id
> which will be used to identify the trials (perhaps in combination with the CDR
> ID). In the absence of the unique CTRP id it looks like using the NCT ID as the
> unique identifier will be a good way to identify the trials. For the coop group
> trials we are about to import from CTRP, we already know their NCT IDs and they
> are unlikely to be altered by the RP.

I'm not sure how to use the instructions in this response. Are you saying that instead of getting the CDR ID from the org_study_id element and seeing if the document for that ID has an ImportTrialFrom element with the value 'CTRP' I should look for the NCT ID in the OtherID/IDString element to identify the document in which I should look for ImportTrialFrom = 'CTRP'?

BZDATETIME::2010-12-14 16:24:21
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::4

(In reply to comment #3)
> I'm not sure how to use the instructions in this response. Are you saying that
> instead of getting the CDR ID from the org_study_id element and seeing if the
> document for that ID has an ImportTrialFrom element with the value 'CTRP' I
> should look for the NCT ID in the OtherID/IDString element to identify the
> document in which I should look for ImportTrialFrom = 'CTRP'?

I assumed (in accordance with question 2) that the org_study_id has been altered by the RP and there is no ‘valid’ CDR ID (in the org_study_id field) to work with in the first place. The only other ID that cannot be altered by the RP and for which we can have an exact match in the CDR is the NCT ID.

BZDATETIME::2010-12-14 16:51:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::5

OK, here's the revised proposal for how the software will work:

For each document we get from NLM for which we don't already have a row mapping the trial to a CDR ID whose document is already mapped for receiving the imported trial information we will get the NCT ID from the file name for the downloaded trial document and look for CDR Documents with that NCT ID as the value of the OtherID/IDString element having a sibling IDType element with the value 'ClinicalTrials.gov ID'. If any of those documents has an ImportTrialFrom element with the value 'CTRP' we will avoid queuing up the trial for import in the ctgov_import table, and will instead add the trial to the ctrp_import table. Does this logic meet the requirements?

BZDATETIME::2010-12-14 17:21:29
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::6

(In reply to comment #5)
> OK, here's the revised proposal for how the software will work:
> For each document we get from NLM for which we don't already have a row mapping
> the trial to a CDR ID whose document is already mapped for receiving the
> imported trial information we will get the NCT ID from the file name for the
> downloaded trial document and look for CDR Documents with that NCT ID as the
> value of the OtherID/IDString element having a sibling IDType element with the
> value 'ClinicalTrials.gov ID'. If any of those documents has an
> ImportTrialFrom element with the value 'CTRP' we will avoid queuing up the
> trial for import in the ctgov_import table, and will instead add the trial to
> the ctrp_import table. Does this logic meet the requirements?

Yes. It does.

BZDATETIME::2010-12-20 07:09:38
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::7

Added CTRP tag.

BZDATETIME::2011-09-16 10:25:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::8

(In reply to comment #5)

> For each document we get from NLM for which we don't already have a row mapping
> the trial to a CDR ID whose document is already mapped for receiving the
> imported trial information we will get the NCT ID from the file name for the
> downloaded trial document and look for CDR Documents with that NCT ID as the
> value of the OtherID/IDString element having a sibling IDType element with the
> value 'ClinicalTrials.gov ID'. If any of those documents has an
> ImportTrialFrom element with the value 'CTRP' we will avoid queuing up the
> trial for import in the ctgov_import table, and will instead add the trial to
> the ctrp_import table.

The last part of this description implies that the primary key for the ctrp_import table will be the NCT ID for the trial, not the CTRP registry identifier. Is this appropriate?

BZDATETIME::2011-09-16 16:12:15
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::9

(In reply to comment #8)

> The last part of this description implies that the primary key for the
> ctrp_import table will be the NCT ID for the trial, not the CTRP registry
> identifier. Is this appropriate?

Since NEW trials from CTRP may not have NCT IDs it seems using the CTRP registry identifier as the primary key would be appropriate.

BZDATETIME::2011-09-16 16:40:32
BZCOMMENTOR::Bob Kline
BZCOMMENT::10

(In reply to comment #9)

> Since NEW trials from CTRP may not have NCT IDs it seems using the CTRP
> registry identifier as the primary key would be appropriate.

So we should refrain putting a row in the ctrp_import table until we have both a CTRP ID as well as an NCT ID for the trial?

BZDATETIME::2011-09-16 17:16:06
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::11

(In reply to comment #10)
> (In reply to comment #9)
>
> > Since NEW trials from CTRP may not have NCT IDs it seems using the CTRP
> > registry identifier as the primary key would be appropriate.
>
> So we should refrain putting a row in the ctrp_import table until we have both
> a CTRP ID as well as an NCT ID for the trial?

Yes. For the existing coop group/RSS trials we should be fine since all of them would have both the NCT IDs and the ctrp registry identifiers. But the new trials would depend on the other CTRP-CTGov developments we talked about last Friday.

BZDATETIME::2011-09-20 10:07:56
BZCOMMENTOR::Bob Kline
BZCOMMENT::12

(In reply to comment #5)

> If any of those documents has an ImportTrialFrom element with
> the value 'CTRP' we will avoid queuing up the trial for import
> in the ctgov_import table, and will instead add the trial to
> the ctrp_import table.

I won't be able to add rows to the ctrp_import table as part of the CT.gov download program, because I may not have the CTRP ID at that point, and we've decided we're going to use the CTRP ID as the primary key for the ctrp_import table. I will be able to avoid queuing up NLM's document for import, though. Here's the logic I've added to the CT.gov download program:

IF THE NCT ID IS ALREADY IN THE CTRP-IMPORT TABLE:
LOG THE TRIAL AS SKIPPED BECAUSE WE'RE IMPORTING IT FROM CTRP
OTHERWISE, IF THE TRIAL'S ORG-STUDY-ID IS A CDR ID:
IF THE CDR DOCUMENT FOR THAT ID IS MARKED AS IMPORT-FROM-CTRP:
LOG THE TRIAL AS SKIPPED BECAUSE WE'RE IMPORTING IT FROM CTRP
OTHERWISE:
CONTINUE WITH EXISTING PROCESSING LOGIC

The CTRP download program will not use the ImportTrialFrom element to identify trials to queue up for import: that element exists sole for the purpose of preventing import of NLM's document for the trial.

I have added the ImportTrialFrom to the CTGovProtocol schema, so that CIAT can mark trials which have already been transferred, to prevent further import of those trials from NLM.

If any of this is wrong, please let me know.

BZDATETIME::2011-09-21 14:15:51
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::13

(In reply to comment #12)
> (In reply to comment #5)
> If any of this is wrong, please let me know.

I have reviewed it and it looks good to me. I don't see any problems with it. I think it will take care of the existing RSS trials because we will get all of them back with NCT and CTRP IDs. We still need to see how the CTRP - CTGov platform is implemented in order to know how to tackle new RSS trials.

BZDATETIME::2011-10-04 09:09:13
BZCOMMENTOR::Bob Kline
BZCOMMENT::14

I believe this task has been rendered obsolete by the most recent decision to import the CTRP trials as CTGovProtocol documents. Is this right?