CDR Tickets

BZDATETIME::2009-11-03 15:39:52
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

(In reply to comment #0)
> Please modify the CTGov import software to copy the <ProtocolSpecialCategory>
> block for re-imported trials (if it exists in the trial).
>
> Also, preserve this block when the trial is updated.

This appears to be in conflict with the requirements for request #4690. Or, more precisely: if we follow the instructions for this request and #4690 as written, then each time we import a trial which has CDR32457 or CDR34517 in the lists of sites, the software will keep all of the instances of ProtocolSpecialCategory which are already present in the CTGovProtocol document and add another one with a SpecialCategory element containing "NIH Clinical Center trial"; I'll be very surprised if that's what you really want. Please think through each of the possible conditions which the software can encounter and provide explicit instructions for what you expect to happen. For example:

• Brand new trial

• Transferred trial

• Existing trial with Magnuson with no ProtocolSpecialCategory blocks

• Trial with Magnuson with 1 or more ProtocolSpecialCategory blocks,
  at least one of which already has "NIH Clinical Center trial"

• Trial with Magnuson with 1 or more ProtocolSpecialCategory blocks,
  none of which already has "NIH Clinical Center trial"

• Trial without Magnuson, with 1 or more ProtocolSpecialCategory blocks,
  at least one of which has "NIH Clinical Center trial"

BZDATETIME::2009-11-03 15:40:29
BZCOMMENTOR::Bob Kline
BZCOMMENT::2

Fixing status.

BZDATETIME::2009-11-03 15:41:01
BZCOMMENTOR::Bob Kline
BZCOMMENT::3

Hit "resolved" by mistake earlier instead of "assigned."

BZDATETIME::2009-11-04 06:02:55
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::4

(In reply to comment #1)
> (In reply to comment #0)
> > Please modify the CTGov import software to copy the <ProtocolSpecialCategory>
> > block for re-imported trials (if it exists in the trial).
> >
> > Also, preserve this block when the trial is updated.
>
>
You’re right. That is not what we want. For already transferred trials, we want to copy the information over as a onetime event and allow the rule in OCECDR-3014 to apply anytime the trial is updated.
Since the NIH Clinical Center trial is determined by the participation of Warren Magnuson in a trial, when Warren Magnuson is no longer participating in a trial, that information must change and this may happen quickly. That is, we can assign the Special Category for a trial today and by next week if Warren Magnuson site is closed, the trial does not qualify to have a Special Category anymore. (This makes my comment in the initial post to preserve the block wrong). This is because if we preserve the block but a trial is updated by closing or removing the Warren Magnuson site, we would have the wrong information and also for the reason you provide above.

> * Brand new trial

For brand new CTGov trials, OCECDR-3014 applies. That is, apply the rule: if the Magnuson site exists, assign the special category of NIH Clinical Center Trial.

> * Transferred trial

For InScopeProtocol, we assigned the Special Category of NIH Clinical Center trial only when the site status is active or Temporarily closed. When the site is closed, we manually remove the NIH Clinical Center trial special category. If the protocol is approved (overall status) or if the site is not active, we do not assign special category. However, for older trials there may be a few where the Warren Magnuson site is Temporarily Closed and yet does not have the Special Category.

Going by the existing rules, if the transferred trial had the block in the InScopeProtocol and the site information is active or Temporarily closed, copy the block to the converted trial. If it does not have the block and yet there is a Warren site that is active or Temporarily closed, then assign the special category of NIH Clinical Center trial. If there are trials that do not follow our existing rules, they are likely user errors. If you report them, we can fix them manually.

> * Existing trial with Magnuson with no ProtocolSpecialCategory blocks

If there are existing trials with the Magnuson site active or Temporarily Closed, OCECDR-3014 applies. Assign Special Category. If the site has any other status apart from Active or Temporarily Closed, do nothing.

> * Trial with Magnuson with 1 or more ProtocolSpecialCategory blocks,
> at least one of which already has "NIH Clinical Center trial"

The same rule applies. If there is a Warren Magnuson site that is Active or Temporarily Closed, copy the NIH Clinical Center trial block only.

I am inclined to think that we are only interested in the “NIH Clinical Trials Center trials” special category at this point. Lakshmi and Margaret can clarify this information. While the other Special Categories are important, they cannot be ‘automatically’ assigned by the software by looking at the lead org information or the site information for example. It would have to be manually maintained. (We may want to copy existing information in the InScopeProtocol documents for trials that already have other special categories like the NCI High Priority trial and the NCI Web site featured trial but even in those cases, we may want to track the issue in a separate issue.)

> * Trial with Magnuson with 1 or more ProtocolSpecialCategory blocks,
> none of which already has "NIH Clinical Center trial"

If the Magnuson site is Active or Temporarily Closed, then assign a new Special Category, otherwise do nothing.

> * Trial without Magnuson, with 1 or more ProtocolSpecialCategory blocks,
> at least one of which has "NIH Clinical Center trial"

I would think that in this case, it would be user error. Do nothing but report errors for us to fix manually. The NIH Clinical Center trial special category is determined solely by participation of the Warren sites.

BZDATETIME::2009-11-05 09:56:31
BZCOMMENTOR::Bob Kline
BZCOMMENT::5

(In reply to comment #4)

> You’re right. That is not what we want. For already transferred trials, we
> want to copy the information over as a onetime event and allow the rule in Bug
> #4690 to apply anytime the trial is updated.

That sounds like a global change, not something which should be handled by the import software. If so, please create a separate issue for the global change, specify what you need to be done, and assign it to Alan.

> > * Transferred trial
>
> For InScopeProtocol, we assigned the Special Category of NIH Clinical Center
> trial only when the site status is active or Temporarily closed. When the site
> is closed, we manually remove the NIH Clinical Center trial special category.
> If the protocol is approved (overall status) or if the site is not active, we
> do not assign special category. However, for older trials there may be a few
> where the Warren Magnuson site is Temporarily Closed and yet does not have the
> Special Category.
>
> Going by the existing rules, if the transferred trial had the block in the
> InScopeProtocol and the site information is active or Temporarily closed, copy
> the block to the converted trial. If it does not have the block and yet there
> is a Warren site that is active or Temporarily closed, then assign the special
> category of NIH Clinical Center trial. If there are trials that do not follow
> our existing rules, they are likely user errors. If you report them, we can
> fix them manually.

I can think of two cases in which CIAT would not have followed the rules you just described:

1. Magnuson present, active or temporarily closed, special category missing
2. Magnuson not present with the right status, but NIHCCT specified anyway

You said for #1 you want the software to fix the problem, but then you said you wanted the software to report both cases and have CIAT fix the problems manually. Which do you really want?

> > * Existing trial with Magnuson with no ProtocolSpecialCategory blocks
>
> If there are existing trials with the Magnuson site active or Temporarily
> Closed, OCECDR-3014 applies. Assign Special Category. If the site has any
> other status apart from Active or Temporarily Closed, do nothing.

That would appear to be saying we shouldn't even report the discrepancy in this case.

> > * Trial with Magnuson with 1 or more ProtocolSpecialCategory blocks,
> > at least one of which already has "NIH Clinical Center trial"
>
> The same rule applies. If there is a Warren Magnuson site that is Active or
> Temporarily Closed, copy the NIH Clinical Center trial block only.

Really? Drop all the other blocks?

> I am inclined to think that we are only interested in the “NIH Clinical Trials
> Center trials” special category at this point. Lakshmi and Margaret can
> clarify this information. While the other Special Categories are important,
> they cannot be ‘automatically’ assigned by the software by looking at the lead
> org information or the site information for example. It would have to be
> manually maintained. (We may want to copy existing information in the
> InScopeProtocol documents for trials that already have other special
> categories like the NCI High Priority trial and the NCI Web site featured
> trial but even in those cases, we may want to track the issue in a
> separate issue.)

I'm confused by the language above. We seem to be conflating discussions of copying existing information with instructions for inserting programatically derived information, and mixing up requirements for handling trials being imported at the point of ownership transfer (when an InScopeProtocol document is being overlaid with a CTGovProtocol version) with those governing ongoing import of CTGovProtocol documents.

If you really want to ditch everything but the "NIH Clinical Trials Center trials" special category, then why bother looking at the InScopeProtocol document at all? I would propose two steps, enormously simplifying the requirements for these two "special category" issues:

1. Run a global change to insert the special category where appropriate
in the existing CTGovProtocol documents.

2. Modify the import software to insert it where appropriate and drop it
otherwise.

"Where appropriate" would mean at least one appearance of Magnuson present with the right status. Lakshmi needs to answer William's question in #4690 about whether to use just CDR34517 or both CDR34517 and CDR32457.

BZDATETIME::2009-11-05 14:48:03
BZCOMMENTOR::Bob Kline
BZCOMMENT::6

We discussed this issue, as well as #4690, at today's status meeting, and decided to simplify the requirements and reorganize the tasks for the work needed for dealing with ProtocolSpecialCategory blocks in CTGovProtocol documents.

This issue covers the modifications needed in the CT.gov import software. Issue #4690 now covers the global change which will be needed to populate existing CTGovProtocol documents with ProtocolSpecialCategory blocks, as well as transferring information from InScopeProtocol documents which have been replaced by CTGovProtocol documents. That issue depends on this one, in the sense that the global change should not be run until the changes made for this issue have been put into production.

Here is the logic to be implemented for this issue in the CT.gov import software:

FOR EACH TRIAL:
IF THE TRIAL CONTAINS AN ACTIVE[1] SITE FOR MAGNUSON[2]:
LET NIHCC = TRUE
OTHERWISE:
LET NIHCC = FALSE
IF WE ALREADY HAVE A PROTOCOL DOCUMENT FOR THIS TRIAL (INSCOPE OR CTGOV):
LET PSC = SET OF PROTOCOLSPECIALCATEGORY ELEMENTS IN THAT DOCUMENT
OTHERWISE:
LET PSC = AN EMPTY SET OF PROTOCOLSPECIALCATEGORY ELEMENTS
IF NIHCC IS TRUE:
IF PSC DOES NOT ALREADY CONTAIN AN "NIH CLINICAL CENTER TRIAL" BLOCK:
ADD SUCH A BLOCK TO PSC (WITH COMMENT "INSERTED BY IMPORT PROGRAM")
OTHERWISE:
REMOVE ANY "NIH CLINICAL CENTER TRIAL" BLOCKS FROM PSC
INSERT PSC INTO THE IMPORTED CTGOVPROTOCOL DOCUMENT

[1] "Active" or "Approved-not yet active" for purposes of this logic.
[2] Lakshmi will tell us whether to look for just CDR34517 or both CDR34517
and CDR32457.

Lakshmi/William:

Does this capture correctly what you need to have happen for this issue?

BZDATETIME::2009-11-05 17:30:24
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::7

(In reply to comment #6)

>
> [1] "Active" or "Approved-not yet active" for purposes of this logic.
> [2] Lakshmi will tell us whether to look for just CDR34517 or both CDR34517
> and CDR32457.
>
> Lakshmi/William:
>
> Does this capture correctly what you need to have happen for this issue?

I think it does capture it very well.

Also, Lakshmi said to look for both CDR34517 and CDR32457.

I can confirm from a cancer.gov search that when NIHCC is selected, trials that have either of these sites are displayed:

For example, I selected only NIH Clinical Center trial on cancer.gov and I got the following trial which lists: CDR32457

http://www.cancer.gov/search/ViewClinicalTrials.aspx?cdrid=624423&version=HealthProfessional&protocolsearchid=6957848

…and the following trials also which lists: CDR34517

http://www.cancer.gov/search/ViewClinicalTrials.aspx?cdrid=433265&version=HealthProfessional&protocolsearchid=6957848

BZDATETIME::2009-11-09 11:55:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::8

(In reply to comment #7)

> > Lakshmi/William:
> >
> > Does this capture correctly what you need to have happen for this issue?
>
> I think it does capture it very well.

I'm going to assume Lakshmi approves as well, and that if she sees any problems with the proposed logic she'll stop me before I get too far. Therefore, I'm going to proceed with implementing the logic as laid out in comment #6.

BZDATETIME::2009-11-10 07:55:11
BZCOMMENTOR::Bob Kline
BZCOMMENT::9

I have implemented the logic laid out in comment #6 above. How would you like to handle testing (in light of the fact that we already have tests under way for several other issues in this area)?

BZDATETIME::2009-11-10 15:33:46
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::10

(In reply to comment #9)
> I have implemented the logic laid out in comment #6 above. How would you like
> to handle testing (in light of the fact that we already have tests under way
> for several other issues in this area)?

When Franck is refreshed, I will start testing these changes.

1. I will try and identify some Warren Magnuson trials we do not have in the CDR and force import them.

2. I will identify a few Warren Magnuson trials in the CDR that you can re-import to see (Including #1 above).

It may be necessary to manipulate some of the import data to test the other scenarios. For example, when NLM drops a Warren Maganuson site (at re-import) that we have already assigned NIHCCT (through an initial import).

BZDATETIME::2009-11-10 16:54:00
BZCOMMENTOR::Bob Kline
BZCOMMENT::11

(In reply to comment #7)
> (In reply to comment #6)
>
> >
> > [1] "Active" or "Approved-not yet active" for purposes of this logic.
> > [2] Lakshmi will tell us whether to look for just CDR34517 or both CDR34517
> > and CDR32457.
> >
> > Lakshmi/William:
> >
> > Does this capture correctly what you need to have happen for this issue?
>
> I think it does capture it very well.

It looks like there is a discrepancy in the comments for this issue regarding which statuses for the Magnuson sites should trigger the inclusion of the "NIH Clinical Center trial" special category block. In the pseudo-code (immediately above), which I believe came from the discussions with Lakshmi (and which you just signed off on) we are saying the statuses we're looking for are "Active" and "Approved-not yet active"; however, elsewhere in the issue the statuses to be used are identified as "Active" and "Temporarily closed"; can we get confirmation as to which status values we should be looking for?

BZDATETIME::2009-11-10 17:53:28
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::12

(In reply to comment #11)
> (In reply to comment #7)
> > (In reply to comment #6)

>
> It looks like there is a discrepancy in the comments for this issue regarding
> which statuses for the Magnuson sites should trigger the inclusion of the "NIH
> Clinical Center trial" special category block. In the pseudo-code (immediately
> above), which I believe came from the discussions with Lakshmi (and which you
> just signed off on) we are saying the statuses we're looking for are "Active"
> and "Approved-not yet active"; however, elsewhere in the issue the statuses to
> be used are identified as "Active" and "Temporarily closed"; can we get
> confirmation as to which status values we should be looking for?

I think the discrepancy stems from mapping the InScopeProtocol status values to the CTGovProtocol status values. In the InScopeProtocol document, we assign NIHCCT special category when the site is either Active or Temporarily closed. So I guess whatever values these two statuses map to in the CTGovProtocol or the values that NLM send us corresponding to the two statuses, should be the ones we use to assign the special category. I know for sure that we do get sites that have Active statuses but I am not sure about temporarily closed statuses.

In the InScopeProtocol document, we do not assign a NIHCCT when the Warren Magnuson site is Approved-not yet active. The site needs to be ready to accept patients or should be temporarily unable to do so before we assign the ProtocolSpecialCategory of NIHCCT.

BZDATETIME::2009-11-10 18:54:37
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::13

I am not sure if this will be of any importance to you: here is a list that we refer to in comparing our status values with that of NLM's.

PDQ / NLM

1. Approved-not yet active / Not yet recruiting
2. Active / Recruiting
3. Enrolling by invitation / Enrolling by invitation
4. Temporarily closed / Suspended
5. Closed / Active, not recruiting
6. Completed / Completed
7. Withdrawn / Terminated or Withdrawn

So going strictly by what we do manually in the InScopeProtocol documents, we will only assign NIHCCT for trials with site values in #2 and #4.

BZDATETIME::2009-11-11 13:22:26
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::14

I will be testing this alongside OCECDR-3012 with some of the following trials when they are converted:

CDR0000298988
CDR0000298984
CDR0000339611
CDR0000378183
CDR0000456444
CDR0000521522

BZDATETIME::2009-11-12 09:39:44
BZCOMMENTOR::Bob Kline
BZCOMMENT::15

New download/import test jobs run this morning. Here are the version numbers created for the documents you identified:

298984 16
298988 31
339611 33
378183 21
456444 25
521522 27

BZDATETIME::2009-11-12 11:12:18
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::16

Please import the following trials for testing the Special Category changes. I had to look for trials with Warren Magnuson already on clinicaltrials.gov.
CDR0000559847
CDR0000349338
CDR0000334841
CDR0000500489
CDR0000068357
CDR0000538554

I also marked the following trials for force import on Franck. I am not sure what to expect from these ones because they are all completed. I am finding it difficult to find trials on clinicaltrials.gov that have Warren Magnuson that we did not register.

NCT00001589
NCT00004978
NCT00005004
NCT00001714
NCT00001515

BZDATETIME::2009-11-13 08:37:52
BZCOMMENTOR::Bob Kline
BZCOMMENT::17

(In reply to comment #16)
> Please import the following trials for testing the Special Category changes. I
> had to look for trials with Warren Magnuson already on clinicaltrials.gov.

I ran a new download and import test on Franck.

> CDR0000559847
> CDR0000349338
> CDR0000334841
> CDR0000500489
> CDR0000068357
> CDR0000538554

Most of these (but not all) got imported.

>
> I also marked the following trials for force import on Franck. I am not sure
> what to expect from these ones because they are all completed. I am finding it
> difficult to find trials on clinicaltrials.gov that have Warren Magnuson that
> we did not register.
>
> NCT00001589
> NCT00004978
> NCT00005004
> NCT00001714
> NCT00001515

Only one of these showed up for import (NCT00001515).

BZDATETIME::2009-11-13 11:32:25
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::18

(In reply to comment #17)
> (In reply to comment #16)
> > Please import the following trials for testing the Special Category changes. I
> > had to look for trials with Warren Magnuson already on clinicaltrials.gov.
>
> I ran a new download and import test on Franck.
>
> > CDR0000559847
> > CDR0000349338
> > CDR0000334841
> > CDR0000500489
> > CDR0000068357
> > CDR0000538554
>
> Most of these (but not all) got imported.
>
> >
> > I also marked the following trials for force import on Franck. I am not sure
> > what to expect from these ones because they are all completed. I am finding it
> > difficult to find trials on clinicaltrials.gov that have Warren Magnuson that
> > we did not register.
> >
> > NCT00001589
> > NCT00004978
> > NCT00005004
> > NCT00001714
> > NCT00001515
>
> Only one of these showed up for import (NCT00001515).

Warren Magnuson site is active on all of the following trials but the Protocol Special Category block did not display in any of the records.

CDR0000538554
CDR0000538554
CDR0000068357
CDR0000500489
CDR0000334841
CDR0000659811

BZDATETIME::2009-11-13 12:30:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::19

(In reply to comment #18)

> Warren Magnuson site is active on all of the following trials but the Protocol
> Special Category block did not display in any of the records.
>
> CDR0000538554
> CDR0000538554
> CDR0000068357
> CDR0000500489
> CDR0000334841
> CDR0000659811

The filters had not been updated on Franck. I fixed that and marked the documents for import again. This results in a couple of them getting the block (3384841 and 500489) but not all of them. For 538554 (which you had twice in your list above) I see a site with the clinical center named, but the corresponding key [1] in the mapping table isn't mapped to any CDR ID, so the Name element for that site didn't have a cdr:ref linking attribute.

[1] NIH - Warren Grant Magnuson Clinical Center|Bethesda|Maryland|20892-1182|United States

BZDATETIME::2009-11-13 13:16:40
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::20

(In reply to comment #19)
> (In reply to comment #18)
>
> > Warren Magnuson site is active on all of the following trials but the Protocol
> > Special Category block did not display in any of the records.
> >
> > CDR0000538554
> > CDR0000538554
> > CDR0000068357
> > CDR0000500489
> > CDR0000334841
> > CDR0000659811
>
> The filters had not been updated on Franck. I fixed that and marked the
> documents for import again. This results in a couple of them getting the block
> (3384841 and 500489)

Yes. These ones (334841 and 500489) display correctly.

but not all of them. For 538554 (which you had twice in
> your list above) I see a site with the clinical center named, but the
> corresponding key [1] in the mapping table isn't mapped to any CDR ID, so the
> Name element for that site didn't have a cdr:ref linking attribute.
>
> [1] NIH - Warren Grant Magnuson Clinical
> Center|Bethesda|Maryland|20892-1182|United States

I will map this one correctly on both Franck and Bach.

I have identified additional trials that can be imported for further testing.

CDR0000349338
CDR0000555674
CDR0000586110
CDR0000555102
CDR0000067459
CDR0000631737
CDR0000502207
CDR0000536009

I have also forced import the following trials I selected from NIH. They are trials I believe we have not received before so they will not meet our selection criteria.

NCT00630461
NCT00839124
NCT00766883
NCT00032513
NCT00977184
NCT00048347
NCT00695136
NCT00100217
NCT00077909
NCT00484861

BZDATETIME::2009-11-13 13:35:37
BZCOMMENTOR::Bob Kline
BZCOMMENT::21

(In reply to comment #20)

> I have also forced import the following trials ....

Don't forget to create the task to modify the interface for that tool to make it clear that you're forcing a download (assuming NLM has the document in their system), not an import. Assign it to Volker or to myself (whichever of us has the fewest critical tasks queued up).

BZDATETIME::2009-11-13 14:09:20
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::22

(In reply to comment #21)
> (In reply to comment #20)
>
> > I have also forced import the following trials ....
>
> Don't forget to create the task to modify the interface for that tool to make
> it clear that you're forcing a download (assuming NLM has the document in their
> system), not an import. Assign it to Volker or to myself (whichever of us has
> the fewest critical tasks queued up).

Created OCECDR-3024 for this issue.

Also, in the CDR meeting yesterday, Lakshmi suggested manipulating the XML files of some of the CTGov protocols so that I can test for the preservation of the blocks. I will identify some of the trials and add them to this issue.

BZDATETIME::2009-11-16 13:37:46
BZCOMMENTOR::Bob Kline
BZCOMMENT::23

(In reply to comment #20)

> I have identified additional trials that can be imported for further testing.
>
>
> CDR0000349338
> CDR0000555674
> CDR0000586110
> CDR0000555102
> CDR0000067459
> CDR0000631737
> CDR0000502207
> CDR0000536009

I have run a number of download/import jobs since comment #20, and only four of those CDR IDs are in the ctgov_import table (67459, 349338, 555674, and 631737), and all four have a disposition of 'duplicate'.

>
> I have also forced import the following trials I selected from NIH. They are
> trials I believe we have not received before so they will not meet our
> selection criteria.
>
> NCT00630461
> NCT00839124
> NCT00766883
> NCT00032513
> NCT00977184
> NCT00048347
> NCT00695136
> NCT00100217
> NCT00077909
> NCT00484861

All of these have been imported.

BZDATETIME::2009-11-16 14:13:03
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::24

(In reply to comment #23)
> (In reply to comment #20)
>
> > I have identified additional trials that can be imported for further testing.
> >
> >
> > CDR0000349338
> > CDR0000555674
> > CDR0000586110
> > CDR0000555102
> > CDR0000067459
> > CDR0000631737
> > CDR0000502207
> > CDR0000536009
>
> I have run a number of download/import jobs since comment #20, and only four of
> those CDR IDs are in the ctgov_import table (67459, 349338, 555674, and
> 631737), and all four have a disposition of 'duplicate'.
>
> >
> > I have also forced import the following trials I selected from NIH. They are
> > trials I believe we have not received before so they will not meet our
> > selection criteria.
> >
> > NCT00630461
> > NCT00839124
> > NCT00766883
> > NCT00032513
> > NCT00977184
> > NCT00048347
> > NCT00695136
> > NCT00100217
> > NCT00077909
> > NCT00484861
>
> All of these have been imported.

None of the above got the block but at least all of them except the first 2 have NIH - Warren Grant Magnuson Clinical Center as a location.

Meanwhile, on Friday, I corrected the mapping problem you identified in comment # 19.

BZDATETIME::2009-11-16 16:47:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::25

(In reply to comment #24)
> > >
> > > NCT00630461
> > > NCT00839124
> > > NCT00766883
> > > NCT00032513
> > > NCT00977184
> > > NCT00048347
> > > NCT00695136
> > > NCT00100217
> > > NCT00077909
> > > NCT00484861
> >
> > All of these have been imported.
>
> None of the above got the block but at least all of them except the first 2
> have NIH - Warren Grant Magnuson Clinical Center as a location.

The filter to insert the ProtocolSpecialCategory element assumed that it would find a PDQAdminInfo block, but of course there won't always be such a block. I have modified the filter to handle that case. I NULLed out the cdr_id column for the rows for these trials (so new CDR documents would be created), set the disposition column to 'import requested', and ran another import job. Here are the CDR IDs:

NCT00032513 660430
NCT00048347 660431
NCT00077909 660432
NCT00100217 660433
NCT00484861 660434
NCT00630461 660435
NCT00695136 660436
NCT00766883 660437
NCT00839124 660438
NCT00977184 660439

BZDATETIME::2009-11-17 10:15:13
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::26

(In reply to comment #25)

> the CDR IDs:
>
> NCT00032513 660430
> NCT00048347 660431
> NCT00077909 660432
> NCT00100217 660433
> NCT00484861 660434
> NCT00630461 660435
> NCT00695136 660436
> NCT00766883 660437
> NCT00839124 660438
> NCT00977184 660439

All of these look good. Thank you!

BZDATETIME::2009-11-17 10:19:00
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::27

(In reply to comment #24)
> (In reply to comment #23)
> > (In reply to comment #20)
> >
> > > I have identified additional trials that can be imported for further testing.
> > >
> > >
> > > CDR0000349338
> > > CDR0000555674
> > > CDR0000586110
> > > CDR0000555102
> > > CDR0000067459
> > > CDR0000631737
> > > CDR0000502207
> > > CDR0000536009
> >
> > I have run a number of download/import jobs since comment #20, and only four of
> > those CDR IDs are in the ctgov_import table (67459, 349338, 555674, and
> > 631737), and all four have a disposition of 'duplicate'.
> >
I tried using the CTGov Protocol to be Marked/Removed as Duplicate tool to remove the duplicates but I am getting the following python script error:
A problem occurred in a Python script.

d:\cdr\Log\tmptspari.html contains the description of this error. d:\python\lib\cgitb.py:173: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 value = pydoc.html.repr(getattr(evalue, name))

BZDATETIME::2009-11-17 11:10:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::28

Could you please give it another try? Thanks!

BZDATETIME::2009-11-17 11:54:22
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::29

(In reply to comment #28)
> Could you please give it another try? Thanks!

It appears to work fine. I did a force import for the following trials and I did not get any errors

67459, 349338, 555674, and 631737

BZDATETIME::2009-11-17 11:57:25
BZCOMMENTOR::Bob Kline
BZCOMMENT::30

(In reply to comment #29)
> (In reply to comment #28)
> > Could you please give it another try? Thanks!
>
> It appears to work fine. I did a force import for the following trials and I
> did not get any errors
>
> 67459, 349338, 555674, and 631737

I thought you were trying to turn off the 'duplicate' disposition, not set the 'force' flag. Please look back at the bottom of comment #27.

BZDATETIME::2009-11-17 12:20:40
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::31

(In reply to comment #30)
> (In reply to comment #29)
> > (In reply to comment #28)
> > > Could you please give it another try? Thanks!
> >
> > It appears to work fine. I did a force import for the following trials and I
> > did not get any errors
> >
> > 67459, 349338, 555674, and 631737
>
> I thought you were trying to turn off the 'duplicate' disposition, not set the
> 'force' flag. Please look back at the bottom of comment #27.

That is correct. I turned off the duplicate and forced import without errors. If turning off the duplicate did not work, I assumed it would not let me force import.

BZDATETIME::2009-11-17 12:48:49
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::32

(In reply to comment #22)
>
> Also, in the CDR meeting yesterday, Lakshmi suggested manipulating the XML
> files of some of the CTGov protocols so that I can test for the preservation of
> the blocks. I will identify some of the trials and add them to this issue.

With regards to the preservation of the blocks, please manipulate the XML files of the following trials:

Special Category:
660430
660432

Funding Info/PDQ ID Info/Transfer & Contact Blocks
66625
66635

Apart from the most recent forced import I did in comment # 29, this will be the final round of testing we need to do.

BZDATETIME::2009-11-17 13:57:58
BZCOMMENTOR::Bob Kline
BZCOMMENT::33

(In reply to comment #31)

> That is correct. I turned off the duplicate and forced import without errors.
> If turning off the duplicate did not work, I assumed it would not let me force
> import.

That's not true on Franck. You had us promote that change (request #4661) to Bach, but not to Franck. The disposition is still set to 'duplicate' for these trials.

BZDATETIME::2009-11-17 14:06:57
BZCOMMENTOR::Bob Kline
BZCOMMENT::34

(In reply to comment #32)

> With regards to the preservation of the blocks, please manipulate the XML files
> of the following trials:
>
> Special Category:
> 660430
> 660432
>
> Funding Info/PDQ ID Info/Transfer & Contact Blocks
> 66625
> 66635

Could you elaborate on the manipulation you would like us to perform on these documents?

BZDATETIME::2009-11-17 14:23:43
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::35

(In reply to comment #34)
> (In reply to comment #32)
>
> > With regards to the preservation of the blocks, please manipulate the XML files
> > of the following trials:
> >
> > Special Category:
> > 660430
> > 660432
> >
> > Funding Info/PDQ ID Info/Transfer & Contact Blocks
> > 66625
> > 66635
>
> Could you elaborate on the manipulation you would like us to perform on these
> documents?

Special Category:

1. Delete the WGM site for 660430. I will test to see if the NIHCCT element will be updated (removed).
2. Remove Exclusion Criteria data from 660432. I will test to see if the NIHCCT element remains unchanged after the removal of Exclusion Criteria

For Funding Info/PDQ ID Info/Transfer & Contact Blocks:

1. Any changes in the XML files for 66625 and 66635 that will trigger a re-import of the trials will be OK. Removal of inclusion criteria or adding a bullet to it etc.

BZDATETIME::2009-11-17 14:58:14
BZCOMMENTOR::Bob Kline
BZCOMMENT::36

NCT00003561, NCT00003569, NCT00032513, and NCT00077909 have been re-imported on Franck.

BZDATETIME::2009-11-17 15:18:15
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::37

(In reply to comment #33)
> (In reply to comment #31)
>
> > That is correct. I turned off the duplicate and forced import without errors.
> > If turning off the duplicate did not work, I assumed it would not let me force
> > import.
>
> That's not true on Franck. You had us promote that change (request #4661) to
> Bach, but not to Franck. The disposition is still set to 'duplicate' for these
> trials.

In place of the 'duplicate' trials, I have added transfer info blocks to the following trials. I selected a relatively large number of trials just in case some fail to get imported. Please import these trials. I will be testing for the presence of the NIHCCT special category in these trials.

63391
64802
65530
65863
66885
67084
67218
67224
67331
67556
67889
68200
68621
69133
69310
69395
69420
69429
69448
74606
78293

BZDATETIME::2009-11-17 15:33:14
BZCOMMENTOR::Bob Kline
BZCOMMENT::38

(In reply to comment #37)

> Please import these trials. ....

Done.

BZDATETIME::2009-11-17 15:59:05
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::39

(In reply to comment #36)
> NCT00003561, NCT00003569, NCT00032513, and NCT00077909 have been re-imported on
> Franck.

1. Looking at the version history, it doesn't look like there were any changes to NCT00077909 and NCT00032513. There was nothing to indicate that they have been re-imported.

2. NCT00003561 looks good. The Funding Info/PDQ ID Info/Transfer & Contact Blocks were all preserved

3. NCT00003569 - It does not look like there were any changes or may be it failed to re-import?

BZDATETIME::2009-11-17 16:09:13
BZCOMMENTOR::Bob Kline
BZCOMMENT::40

Take another look, please. I had modified the XML from the ctgov_import table, but forgot to put the modified XML back in the table. Should be OK now.

BZDATETIME::2009-11-17 16:14:54
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::41

(In reply to comment #38)
> (In reply to comment #37)
>
> > Please import these trials. ....
>
> Done.

None of the documents got transformed to the 'new' CTGov trial. All of them have NCT IDs so I was hoping a few (if not all) of them would get transformed.

BZDATETIME::2009-11-17 16:20:00
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::42

(In reply to comment #40)
> Take another look, please. I had modified the XML from the ctgov_import table,
> but forgot to put the modified XML back in the table. Should be OK now.

All of them look good except NCT00032513, which did not have have the WGM site deleted so the NIHCCT special category was also preserved.

BZDATETIME::2009-11-17 16:35:56
BZCOMMENTOR::Bob Kline
BZCOMMENT::43

(In reply to comment #41)
> (In reply to comment #38)
> > (In reply to comment #37)
> >
> > > Please import these trials. ....
> >
> > Done.
>
>
> None of the documents got transformed to the 'new' CTGov trial. All of them
> have NCT IDs so I was hoping a few (if not all) of them would get transformed.

Well, all I did when you asked me to import them was to mark the disposition column as 'import requested' for those rows and then kick off another run of the CT.gov import job. But none of the rows has any XML from NLM, and they all have the comment "Marked as duplicate by download job" (that would have been the job I ran this morning shortly after 8).

Tell me more about when you needed me to do for these.

BZDATETIME::2009-11-17 16:42:59
BZCOMMENTOR::Bob Kline
BZCOMMENT::44

Third time's a charm. The 'commit' was omitted from the utility used to replace the XML in the ctgov_import table. I think it's right now (the ProtocolSpecialCategory block has been stripped from the PDQAdminInfo element, which should now be empty).

BZDATETIME::2009-11-17 16:51:04
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::45

(In reply to comment #44)
> Third time's a charm. The 'commit' was omitted from the utility used to
> replace the XML in the ctgov_import table. I think it's right now (the
> ProtocolSpecialCategory block has been stripped from the PDQAdminInfo element,
> which should now be empty).

Confirmed!

BZDATETIME::2009-11-17 17:00:40
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::46

(In reply to comment #43)
> (In reply to comment #41)
> > (In reply to comment #38)
> > > (In reply to comment #37)
> > >
> > > > Please import these trials. ....
> > >
> > > Done.
> >
> >
> > None of the documents got transformed to the 'new' CTGov trial. All of them
> > have NCT IDs so I was hoping a few (if not all) of them would get transformed.
>
> Well, all I did when you asked me to import them was to mark the disposition
> column as 'import requested' for those rows and then kick off another run of
> the CT.gov import job. But none of the rows has any XML from NLM, and they all
> have the comment "Marked as duplicate by download job" (that would have been
> the job I ran this morning shortly after 8).
>
> Tell me more about when you needed me to do for these.

If you can force a download of these trials that would be great. I need to test for how newly transferred trials with Warren Magnuson would behave. I have tested a few of them with other changes. But this will be solely for testing the Special Category changes when InScope trials are transferred.

BZDATETIME::2009-11-17 17:47:10
BZCOMMENTOR::Bob Kline
BZCOMMENT::47

(In reply to comment #46)

> If you can force a download of these trials that would be great. I need to test
> for how newly transferred trials with Warren Magnuson would behave. I have
> tested a few of them with other changes. But this will be solely for testing
> the Special Category changes when InScope trials are transferred.

These trials were all listed in a file we got from Lakshmi at least 6 years ago, listing CDR IDs and NCT IDs for duplicates. I can pull down the XML for these trials by hand, stuff them into the ctgov_import table, set the disposition as 'import requested' and run the import job if you're confident that this pulling rabbits out of hats will really provide a reliable test of what you're trying to verify.

BZDATETIME::2009-11-18 09:00:13
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::48

(In reply to comment #47)
> (In reply to comment #46)
>
> > If you can force a download of these trials that would be great. I need to test
> > for how newly transferred trials with Warren Magnuson would behave. I have
> > tested a few of them with other changes. But this will be solely for testing
> > the Special Category changes when InScope trials are transferred.
>
> These trials were all listed in a file we got from Lakshmi at least 6 years
> ago, listing CDR IDs and NCT IDs for duplicates. I can pull down the XML for
> these trials by hand, stuff them into the ctgov_import table, set the
> disposition as 'import requested' and run the import job if you're confident
> that this pulling rabbits out of hats will really provide a reliable test of
> what you're trying to verify.

It looks like that will be too much work. I will find some of the recent trials and add the transfer blocks. Thanks!

BZDATETIME::2009-11-18 09:07:37
BZCOMMENTOR::Bob Kline
BZCOMMENT::49

(In reply to comment #48)

> It looks like that will be too much work. I will find some of the recent trials
> and add the transfer blocks. Thanks!

It's not a question of how much work it is. I can write scripts that will do the heavy lifting. The issue is whether you're confident that the test will be reliable.

BZDATETIME::2009-11-18 09:15:19
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::50

(In reply to comment #49)
> (In reply to comment #48)
>
> > It looks like that will be too much work. I will find some of the recent trials
> > and add the transfer blocks. Thanks!
>
> It's not a question of how much work it is. I can write scripts that will do
> the heavy lifting. The issue is whether you're confident that the test will be
> reliable.

Yes. These are all active protocols with active WGM sites and still we still own them on clinicaltrials.gov, I am confident they will be useful for testing when they are successfully downloaded.

BZDATETIME::2009-11-18 11:21:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::51

For each of these trials when I asked NLM for the trial document I got back XML with a different NCT ID (the NCT ID I requested, based on what we have in the ctgov_import table, was present in the nct_alias element). I can modify my scripts to use whatever document NLM gives us back, regardless of whether it's the one we asked for, as long as you are OK with using documents that don't match for this test.

BZDATETIME::2009-11-18 11:28:47
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::52

(In reply to comment #51)
> For each of these trials when I asked NLM for the trial document I got back XML
> with a different NCT ID (the NCT ID I requested, based on what we have in the
> ctgov_import table, was present in the nct_alias element). I can modify my
> scripts to use whatever document NLM gives us back, regardless of whether it's
> the one we asked for, as long as you are OK with using documents that don't
> match for this test.

Yes. Let's give that a try. As long as the documents contain Warren Magnuson sites, I should be fine.

BZDATETIME::2009-11-18 12:02:47
BZCOMMENTOR::Bob Kline
BZCOMMENT::53

OK. Here's the output from the script to prep the rows in ctgov_import:

plugged in XML from NCT00018993 for NCT00001379 (CDR63391)
plugged in XML from NCT00019149 for NCT00001506 (CDR64802)
plugged in XML from NCT00019305 for NCT00001575 (CDR65530)
plugged in XML from NCT00019370 for NCT00001586 (CDR65863)
plugged in XML from NCT00019617 for NCT00001238 (CDR66885)
plugged in XML from NCT00004044 for NCT00001813 (CDR67084)
plugged in XML from NCT00004007 for NCT00001163 (CDR67218)
plugged in XML from NCT00019799 for NCT00001823 (CDR67224)
plugged in XML from NCT00019942 for NCT00001832 (CDR67331)
plugged in XML from NCT00020020 for NCT00001941 (CDR67556)
plugged in XML from NCT00020163 for NCT00005655 (CDR67889)
plugged in XML from NCT00020280 for NCT00026689 (CDR68200)
plugged in XML from NCT00020592 for NCT00013533 (CDR68621)
plugged in XML from NCT00030108 for NCT00025961 (CDR69133)
plugged in XML from NCT00033670 for NCT00030992 (CDR69310)
plugged in XML from NCT00039533 for NCT00033137 (CDR69395)
plugged in XML from NCT00040924 for NCT00035373 (CDR69420)
plugged in XML from NCT00039598 for NCT00034424 (CDR69429)
plugged in XML from NCT00041158 for NCT00037817 (CDR69448)
plugged in XML from NCT00018915 for NCT00001186 (CDR74606)
plugged in XML from NCT00018980 for NCT00001337 (CDR78293)

The import job has been run.

BZDATETIME::2009-11-18 12:20:54
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::54

(In reply to comment #53)
> OK. Here's the output from the script to prep the rows in ctgov_import:
>

> The import job has been run.

This one worked perfectly. Thank you!

This is perhaps the last request for this issue. When copying the CTGovTransferInfo block, is it possible to include the CTGovOwnershipTransferDate empty element? Users will enter this date in the converted document and it will save a lot of time to have the empty elements already present, if possible.

BZDATETIME::2009-11-19 08:39:31
BZCOMMENTOR::Bob Kline
BZCOMMENT::55

(In reply to comment #54)

> This is perhaps the last request for this issue. When copying the
> CTGovTransferInfo block, is it possible to include the
> CTGovOwnershipTransferDate empty element? Users will enter this date in the
> converted document and it will save a lot of time to have the empty elements
> already present, if possible.

Do you really want the software to remove existing values that it finds when copying the block? Or did you mean that you want to copy the element as it is found when it is already present in the block being copied, adding an empty instance if the element is not already present?

BZDATETIME::2009-11-19 08:42:06
BZCOMMENTOR::Bob Kline
BZCOMMENT::56

(In reply to comment #55)
> (In reply to comment #54)
>
> > This is perhaps the last request for this issue. When copying the
> > CTGovTransferInfo block, is it possible to include the
> > CTGovOwnershipTransferDate empty element? Users will enter this date in the
> > converted document and it will save a lot of time to have the empty elements
> > already present, if possible.
>
> Do you really want the software to remove existing values that it finds when
> copying the block? Or did you mean that you want to copy the element as it is
> found when it is already present in the block being copied, adding an empty
> instance if the element is not already present?

And does "copying the CTGovTransferInfo block" in this context mean just the copy of the block from the InScopeProtocol document when the trial is first transferred (in contrast to the preserving of the block when subsequent versions of the CTGovProtocol document are made)?

BZDATETIME::2009-11-19 09:24:24
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::57

(In reply to comment #55)
> (In reply to comment #54)
>
> > This is perhaps the last request for this issue. When copying the
> > CTGovTransferInfo block, is it possible to include the
> > CTGovOwnershipTransferDate empty element? Users will enter this date in the
> > converted document and it will save a lot of time to have the empty elements
> > already present, if possible.
>
> Do you really want the software to remove existing values that it finds when
> copying the block? Or did you mean that you want to copy the element as it is
> found when it is already present in the block being copied, adding an empty
> instance if the element is not already present?

Please copy the element as you find it in the InScopeProtocol document. However, it is unlikely you will find any values in the document (Alan's global will find values). The last step in our new transfer process is to add the date and publish the document. In our previous process it was also the last process (to add the date value and block the document). So for documents that we will be adding the blocks to in Bach, we do not expect to find the date in the document before the documents are converted. However, to be safe, I will say copy any values that are in the element to the converted document.

BZDATETIME::2009-11-19 09:31:56
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::58

(In reply to comment #56)
> (In reply to comment #55)
> > (In reply to comment #54)
> >
> > > This is perhaps the last request for this issue. When copying the
> > > CTGovTransferInfo block, is it possible to include the
> > > CTGovOwnershipTransferDate empty element? Users will enter this date in the
> > > converted document and it will save a lot of time to have the empty elements
> > > already present, if possible.
> >
> > Do you really want the software to remove existing values that it finds when
> > copying the block? Or did you mean that you want to copy the element as it is
> > found when it is already present in the block being copied, adding an empty
> > instance if the element is not already present?
>
> And does "copying the CTGovTransferInfo block" in this context mean just the
> copy of the block from the InScopeProtocol document when the trial is first
> transferred (in contrast to the preserving of the block when subsequent
> versions of the CTGovProtocol document are made)?

What I meant was, when copying the block before conversion of the document, also copy the CTGovOwnershipTransferDate element and when the document is re-imported, the date should be preserved just like the Ownership Transfer block. Generally, when the document is re-imported, we do not expect to update the Date element (again) unless, we were unable to add the date and publish the document before the document was re-imported.

BZDATETIME::2009-11-19 16:09:25
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::59

You mentioned in the CDR meeting that the addition of the CTGovOwnershipTransferDate is completed. I have added transfer blocks to the following trials to be downloaded for testing. I added a date value to one of the trials for testing purposes. Thanks!

600355
601054
625222
634155
634501
653173

BZDATETIME::2009-11-19 16:40:57
BZCOMMENTOR::Bob Kline
BZCOMMENT::60

I'll run another download/import set tomorrow morning (and we'll hope that NLM sends us changes for at least a couple of the trials).

BZDATETIME::2009-11-20 06:15:53
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::61

(In reply to comment #60)
> I'll run another download/import set tomorrow morning (and we'll hope that NLM
> sends us changes for at least a couple of the trials).

If the download/import will include the set of trials in comment #59, that should be enough. I am only testing for the presence of the CTGovOwnershipTransferDate (from the InScopeProtocol document) element which depends on whether the document will be converted on not. As long as any of the documents in comment #59 are converted, I should be able to test the new change and I think that will not depend on whether NLM sends changes or not.

BZDATETIME::2009-11-20 09:15:06
BZCOMMENTOR::Bob Kline
BZCOMMENT::62

Ran another download/import test on Franck. Five of the six documents showed up (all except 634501, which matches NCT00465595, which is involved with CDR546817 as a duplicate).

BZDATETIME::2009-11-20 11:21:37
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::63

(In reply to comment #62)
> Ran another download/import test on Franck. Five of the six documents showed
> up (all except 634501, which matches NCT00465595, which is involved with
> CDR546817 as a duplicate).

I reviewed the list and it appears only the one I entered the date had the element copied. We will also want the element to be copied even when there is no value in it.

BZDATETIME::2009-11-20 13:08:00
BZCOMMENTOR::Bob Kline
BZCOMMENT::64

(In reply to comment #63)

> I reviewed the list and it appears only the one I entered the date had the
> element copied. We will also want the element to be copied even when there is
> no value in it.

Well, as it turns out the element was there (at least in the one I checked) already, in the InScopeProtocol, but since all it had in it was the XMetaL processing instruction for the prompt, and since the flag was turned on to validate the document when it was stored, the element was discarded by the server (as it is required to do when validating documents). The only way you can prevent the element from being discarded when the CTGovProtocol is saved is to have us turn off validation when we save the document. Do you want us to do that?

BZDATETIME::2009-11-20 18:30:51
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::65

(In reply to comment #64)
> (In reply to comment #63)
>
> > I reviewed the list and it appears only the one I entered the date had the
> > element copied. We will also want the element to be copied even when there is
> > no value in it.
>
> Well, as it turns out the element was there (at least in the one I checked)
> already, in the InScopeProtocol, but since all it had in it was the XMetaL
> processing instruction for the prompt, and since the flag was turned on to
> validate the document when it was stored, the element was discarded by the
> server (as it is required to do when validating documents). The only way you
> can prevent the element from being discarded when the CTGovProtocol is saved is
> to have us turn off validation when we save the document. Do you want us to do
> that?

I think it will be OK to turn off validation for these protocols since we are going to review the documents, validate them before we publish them. So turning off validation should not be a problem. Also, I did check a few newly downloaded trials and it looks like we do not validate them. If this is the case, I think it will be OK to turn off validation.

However, it will be good for Lakshmi to give the OK on this one. I will send her a quick email on Monday morning.

BZDATETIME::2009-11-23 12:50:22
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::66

(In reply to comment #65)

> However, it will be good for Lakshmi to give the OK on this one. I will send
> her a quick email on Monday morning.

Here is Lakshmi's response:

I am Ok with turning off validation, just so long as you all know what to do if there are other validation issues!

Bob:
I think it is OK to promote this issue at this point. If you agree, I will post the other bugs numbers that must be promoted with this one.

BZDATETIME::2009-11-23 15:48:54
BZCOMMENTOR::Bob Kline
BZCOMMENT::67

(In reply to comment #66)

> I am Ok with turning off validation, just so long as you all know what to do if
> there are other validation issues!

I have modified the import program to turn off the validation of the documents for newly transferred trials.

> I think it is OK to promote this issue at this point. If you agree, I will post
> the other bugs numbers that must be promoted with this one.

Sure.

BZDATETIME::2009-11-23 15:57:35
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::68

(In reply to comment #67)
> (In reply to comment #66)

>
> > I think it is OK to promote this issue at this point. If you agree, I will post
> > the other bugs numbers that must be promoted with this one.
>
> Sure.

Here are the bugs that need to be promoted with this OCECDR-3013

1. OCECDR-3012 - Modify CT.gov schema to include Protocol Special Category
2. OCECDR-2988 - Modify import software to add Processing Details Block to newly imported trials
3. OCECDR-3009 - Modify import software to preserve blocks from InScopeProtocol documents
4. OCECDR-2991 - Copy CTGov Ownership Transfer Info and Contact Log blocks into newly converted document
5. OCECDR-3008 - Copy Funding information block into converted trials
6. OCECDR-3018 - Modify CSS and Template to display funding information
7. OCECDR-3019 - Modify CSS and Template to display transfer blocks

BZDATETIME::2009-11-23 16:17:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::69

(In reply to comment #68)

> Here are the bugs that need to be promoted with this OCECDR-3013
>
> 1. OCECDR-3012 - Modify CT.gov schema to include Protocol Special Category
> 2. OCECDR-2988 - Modify import software to add Processing Details Block to newly
> imported trials
> 3. OCECDR-3009 - Modify import software to preserve blocks from InScopeProtocol
> documents
> 4. OCECDR-2991 - Copy CTGov Ownership Transfer Info and Contact Log blocks into
> newly converted document
> 5. OCECDR-3008 - Copy Funding information block into converted trials
> 6. OCECDR-3018 - Modify CSS and Template to display funding information
> 7. OCECDR-3019 - Modify CSS and Template to display transfer blocks

Alan:

If I promote the software for these CT.gov import change now (the schema has already been promoted), will that cause any problems for your global change software?

BZDATETIME::2009-11-23 16:19:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::70

(In reply to comment #69)
> (In reply to comment #68)
>
> > Here are the bugs that need to be promoted with this OCECDR-3013
> >
> > 1. OCECDR-3012 - Modify CT.gov schema to include Protocol Special Category
> > 2. OCECDR-2988 - Modify import software to add Processing Details Block to newly
> > imported trials
> > 3. OCECDR-3009 - Modify import software to preserve blocks from InScopeProtocol
> > documents
> > 4. OCECDR-2991 - Copy CTGov Ownership Transfer Info and Contact Log blocks into
> > newly converted document
> > 5. OCECDR-3008 - Copy Funding information block into converted trials
> > 6. OCECDR-3018 - Modify CSS and Template to display funding information
> > 7. OCECDR-3019 - Modify CSS and Template to display transfer blocks
>
> Alan:
>
> If I promote the software for these CT.gov import change now (the schema has
> already been promoted), will that cause any problems for your global change
> software?

Volker:

You should have been included as a CC at least as far back as comment #68, since part of that comment carries instructions for you to promote some of your work on other issues than this one.

BZDATETIME::2009-11-23 16:47:25
BZCOMMENTOR::Bob Kline
BZCOMMENT::71

Changes installed on Bach; please monitor the download/import jobs carefully over the next few days.

BZDATETIME::2009-11-23 17:16:54
BZCOMMENTOR::Volker Englisch
BZCOMMENT::72

(In reply to comment #70)
> You should have been included as a CC at least as far back as comment #68,
> since part of that comment carries instructions for you to promote some of your
> work on other issues than this one.

Actually, I have a similar comment on my tasks but the issues are not linked.
So, I'm guessing the CSS is now ready to go, right, William?

BZDATETIME::2009-11-23 17:18:51
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::73

(In reply to comment #72)
> (In reply to comment #70)
> > You should have been included as a CC at least as far back as comment #68,
> > since part of that comment carries instructions for you to promote some of your
> > work on other issues than this one.
>
> Actually, I have a similar comment on my tasks but the issues are not linked.
> So, I'm guessing the CSS is now ready to go, right, William?

Yes. That is correct.

BZDATETIME::2009-11-25 11:37:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::74

[Copied over from issue #4690]

(In reply to comment #43)

> We need to have a special category when the overall status is Temporarily
> closed. That is, if we have a status at the site level or not.

I think I missed that in the requirements for issue #4689. It certainly didn't
show up in the statement of logic presented in
http://verdi.nci.nih.gov/tracker/show_bug.cgi?id=4689#c6 (as I read it). Do we
need to re-visit that issue?

BZDATETIME::2009-11-25 11:59:54
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::75

(In reply to comment #74)
> [Copied over from issue #4690]
>
> (In reply to comment #43)
>
> > We need to have a special category when the overall status is Temporarily
> > closed. That is, if we have a status at the site level or not.
>
> I think I missed that in the requirements for issue #4689. It certainly didn't
> show up in the statement of logic presented in
> http://verdi.nci.nih.gov/tracker/show_bug.cgi?id=4689#c6 (as I read it). Do we
> need to re-visit that issue?

It may not be necessary to revisit this if Lakshmi agrees that we should not include the special category for trials that are temporarily closed and also do not have a site status. However, I am not sure if you need to make any changes to the import software.

BZDATETIME::2009-12-03 15:34:45
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::76

(In reply to comment #75)
> (In reply to comment #74)
> > [Copied over from issue #4690]
> >
> > (In reply to comment #43)
> >
> > > We need to have a special category when the overall status is Temporarily
> > > closed. That is, if we have a status at the site level or not.
> >
> > I think I missed that in the requirements for issue #4689. It certainly didn't
> > show up in the statement of logic presented in
> > http://verdi.nci.nih.gov/tracker/show_bug.cgi?id=4689#c6 (as I read it). Do we
> > need to re-visit that issue?
>
>
> It may not be necessary to revisit this if Lakshmi agrees that we should not
> include the special category for trials that are temporarily closed and also do
> not have a site status. However, I am not sure if you need to make any changes
> to the import software.

I am adding the email response from Lakshmi to the question about Overallstatus.

**Email from Lakshmi**
I do not believe we publish sites for temporarily closed trials .

Bob, when we import CTGOV trials, do we import sites for closed, completed and temp closed trials?

I don’t see a problem with inserting or not inserting the Special Category for all closed/completed/Temp closed trials If we insert the Special Category for temp closed trials, it will not be as much of an issue since they will be retrieved by the NIH clinical trials search only if someone specifically searches for closed protocols. And in that case, they will still have the protocol chair or research nurse’s contact information. If we don’t insert it for the temp closed trials, that is also Ok because most people use the search feature for NIH trials for active trials only.

Bottom line, either way is OK – the one that requires the least effort or change is probably the way to go.

Lakshmi

From: Osei-Poku, William wOsei-Poku@icfi.com
Sent: Wednesday, November 25, 2009 1:06 PM
To: Grama, Lakshmi (NIH/NCI) [E]
Cc: Kline, Robert (NCI); vrmeyer@comcast.net
Subject: Global change to move InScopeProtocol information into CTGovProtocol docs
Importance: High

Hi Lakshmi,
Could you please weigh in on OCECDR-3014? We want to know if it is OK to implement the Global and the import software modification such that when the protocol is Temporarily Closed (and have no site status from NLM), we do not assign a Special Category?

We need your input because initially, we had talked about assigning a special category when the protocol is temporarily closed. However, last night, Alan discovered that NLM is not sending us site status for some trials including Temporarily closed trials. I checked in the CDR and it looks like for all trials that are closed, temporarily closed and completed, we do not receive site status from NLM (Bob may be able confirm this from the CTGov import table). I also checked clinicaltrials.gov and for the trials that were temporarily closed (suspended), closed (Active, not recruiting) and Completed, NLM was not publishing phone numbers, which is how I believe we have implemented our global and import software.

If we implement it the way we had initially discussed it, it will mean that for temporarily closed protocols, NLM will not be publishing phone numbers for Warren Magnuson sites but we will be publishing the sites because we assigned the special category.

Comments 41-47 in OCECDR-3014 will throw more light on the issue.

Thanks,
William

BZDATETIME::2009-12-04 12:09:53
BZCOMMENTOR::Bob Kline
BZCOMMENT::77

My understanding from yesterday's discussion is that there is nothing more I need to do for this issue.

BZDATETIME::2009-12-04 12:58:24
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::78

(In reply to comment #77)
> My understanding from yesterday's discussion is that there is nothing more I
> need to do for this issue.

That is correct. I am keeping this issue open to review the preservation of the blocks on Bach.

BZDATETIME::2009-12-14 10:39:43
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::79

I lowered priority to P6. All changes have been tested and found to be working correctly. The only one I have not been able to test is the non-NIHCCT special categories.

BZDATETIME::2010-01-06 16:57:19
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::80

(In reply to comment #79)
> I lowered priority to P6. All changes have been tested and found to be working
> correctly. The only one I have not been able to test is the non-NIHCCT special
> categories.

Verified on Bach. Issue Closed. Thanks!