CDR Tickets

BZDATETIME::2012-01-05 13:38:18
BZCOMMENTOR::Alan Meyer
BZCOMMENT::1

I'll plan on implementing the following functionality:

1. Generate a new unique PermaTarg ID when required for a new
PermaTarg attribute.

2. Validate that no PermaTarg ID has been dropped accidentally.

This will probably require that we design a method of
intentionally dropping a PermaTarg ID, and a way to
discover it when a PermaTarg ID has disappeared without
having gone through the intentional process.

3. Validate that no PermaTarg ID is used twice in the database.

Question:

Will it be legal for a user to create his own, unique
PermaTarg ID, e.g.:

<Whatever PermaTarg='Radiation Damage Assessment'>
...
</Whatever>

If so, item 3 above is particularly important.

I'm adding Robin, Bob and Volker to the CC list.

BZDATETIME::2012-01-05 13:45:23
BZCOMMENTOR::Bob Kline
BZCOMMENT::2

(In reply to comment #1)
> Will it be legal for a user to create his own, unique
> PermaTarg ID, e.g.:
>
> <Whatever PermaTarg='Radiation Damage Assessment'>
> ...
> </Whatever>

No.

BZDATETIME::2012-01-05 13:51:50
BZCOMMENTOR::Alan Meyer
BZCOMMENT::3

(In reply to comment #2)
> (In reply to comment #1)
> > Will it be legal for a user to create his own, unique
> > PermaTarg ID, e.g.:
...
> No.

In that case I'll also want to implement:

4. No PermaTarg ID is used in a document that is not already
known to the system and generated through method 1 above.

BZDATETIME::2012-03-12 17:59:01
BZCOMMENTOR::Alan Meyer
BZCOMMENT::4

If I understand the PermaTarg concept correctly, we DO NOT intend
to use or support PermaTarg IDs as the target of links from one
CDR document to another. If CDR document A links to an element
inside CDR document B, it will do so with a CDR ID # Fragment ID
link, as it always has. We already support and validate those
links and already have a way to handle attempts to delete an
element and corresponding fragment ID that other documents link
to.

Therefore, we don't need to track what in the CDR links to a
PermaTarg. There won't be any such links.

Can someone officially confirm that that is correct?

I will assume it is correct unless and until I hear otherwise.

BZDATETIME::2012-03-12 18:58:42
BZCOMMENTOR::Alan Meyer
BZCOMMENT::5

I plan to perform PermaTarg validation as part of link
validation. That is just like the way we perform the cdr:id
validation that produces messages like: "cdr:id '_123' used more
than once."

That means that the validation will occur:

When link validation is specifically requested.

When a document is saved as a publishable version (which
automatically invokes both kinds of validation.)

Otherwise, it won't occur.

BZDATETIME::2012-03-12 20:09:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::6

I propose the following user interface in XMetal for adding and
removing PermaTarg attributes:

Adding an attribute:

Add an attribute with the special value 0, i.e.:

PermaTarg='0'

To get the value of a newly added PermaTarg:

Save the document. The saved document will have '0' replaced
by a unique integer that is the new PermaTarg id value, for
example:

PermaTarg='123'

To delete a PermaTarg:

Modify the specific attribute to be deleted by appending a
the string "-DELETE" to it. For example, to delete
PermaTarg='123', edit the attribute to contain a string as
follows:

PermaTarg='123-DELETE'

This string will be validated during Link validation but will
not actually do anything unless and until the document is
saved. When the document is saved the server will delete the
PermaTarg attribute from the element and will mark it in the
database as deleted and never to be used again.

If a user enters any other string, for example any of the
following:

PermaTarg='123DELETE'
PermaTarg='123Deleted'
PermaTarg='123_DELETED'
PermaTarg='123 DELETED'
PermaTarg='124-DELETED'

The server will return a pair of error messages like the
following:

Reporting that 123 was expected but not found:

"Error: PermaTarg '123' was in this document but is now
missing. To delete PermaTarg 123, create an attribute of
the form PermaTarg='123-DELETED'"

Reporting that '123 DELETED' (or whatever string other than
'123' or '123-DELETED') was found but not expected:

"Error: PermaTarg '123 DELETED' invalid. It was not
generated by the CDR server."

Please let me know if this does not meet requirements or if there
is a better way to do it.

BZDATETIME::2012-03-12 20:17:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::7

Schema validation will validate that a PermaTarg attribute can
only be inserted in an element where it is legal. XMetal will
also validate that the PermaTarg attribute can only be added
where it is legal.

Beyond that, I do NOT plan to validate the positioning of a
PermaTarg attribute. If PermaTarg is a valid attribute of a
SummarySection, and if a user moves a PermaTarg from one
SummarySection to another, or if the user deletes a
SummarySection that contained a PermaTarg but attaches the
PermaTarg to a different section or other element where it is
legal, that will be okay. No validation error will occur.

Please let me know if this poses any problems.

BZDATETIME::2012-03-12 20:27:54
BZCOMMENTOR::Alan Meyer
BZCOMMENT::8

One last comment about PermaTargs for tonight.

In my CDR status meeting notes for July 5, 2011 I wrote the
following:

"We should not have a PermaTarg on a miscellaneous document
or anything that can be included in multiple other
documents."

The reason for this is that the PermaTarg can wind up in more
than one document after publishing assembles the final document
copies.

I'm not planning any special programming for this. We'll be
relying on schema validation to enforce it.

BZDATETIME::2012-03-12 20:49:19
BZCOMMENTOR::Alan Meyer
BZCOMMENT::9

I have outlined a crude, high level design for the PermaTarg
management. I'm guessing that it will take not less than two
days to implement and three seems like a safer bet. For me, only
being here two days a week, that is one or one and a half weeks
work.

I'll suspend work on this for now and return to the CiteMS / EBMS
effort. I'll want to get back to deep linking at least several
weeks before we need it for testing. That will be more than
several weeks before release 6.5 of cancer.gov since Volker,
Blair, and others will probably need this for their testing well
before 6.5 integration testing.

BZDATETIME::2012-03-13 12:04:00
BZCOMMENTOR::Volker Englisch
BZCOMMENT::10

(In reply to comment #6)
> I propose the following user interface in XMetal for adding and
> removing PermaTarg attributes:

I am guessing that PermaTarg attributes will be added and/or deleted using a macro (to be created separately) because users typically don't edit attributes manually, right?
This would also eliminate most of the "misspellings" if a PermaTarg attribute needs to be deleted.

BZDATETIME::2012-03-13 13:24:05
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

(In reply to comment #8)
> I'm not planning any special programming for this. We'll be
> relying on schema validation to enforce it.

I don't think that schema validation will be able to help us to enforce this, at least not in all cases.
A PermaTarg is valid for a SummarySection and a SummarySection will be included within a SummaryModule which can be included in multiple different summaries.

BZDATETIME::2012-03-13 18:24:34
BZCOMMENTOR::Alan Meyer
BZCOMMENT::12

(In reply to comment #11)
> ...
> A PermaTarg is valid for a SummarySection and a SummarySection will be included
> within a SummaryModule which can be included in multiple different summaries.

We might try to write a custom rule for SummaryModule to handle this, or we might just say People have to be careful and know what they're doing when they create a PermaTarg.

Another issue, mentioned by Bob to me, is that blocking a document is potentially dangerous if there is a PermaTarg inside. We have checks that can warn us if other CDR documents link to a document we want to block, but the CDR may or may not, depending on what kinds of checking that cancer.gov provides, be able to tell if there are any external links.

My inclination is to not over protect ourselves with automated checks. PermaTargs should be used carefully and only in highly stable situations. We can add more automated checks if and when we get dozens or hundreds of them and changes start to become common.

BZDATETIME::2012-03-14 10:37:22
BZCOMMENTOR::Volker Englisch
BZCOMMENT::13

(In reply to comment #12)
> My inclination is to not over protect ourselves with automated checks.
> PermaTargs should be used carefully and only in highly stable situations.

I would agree with that.

In the case of PermaTargs within a module we could possibly protect ourselves by adding a filter template that reports an error if it finds one of those critters but it would still have to be fixed manually.

BZDATETIME::2012-04-17 20:39:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::14

I have added a new link_permatarg table to tables.sql and to the CDR database on mahler as a first step to support deep linking.

The table includes columns for:

auto generated PermaTargId attribute value
CDR document ID
active status - 'A' = active, 'D' = deleted don't reuse
datetime activated
datetime deleted

The datetime data isn't needed but it doesn't cost anything to keep and may help us if problems come up later.

I am going to meet with Blair and Lauren on Thursday to find out when they need us to be able to supply sample data.

BZDATETIME::2012-04-27 00:23:18
BZCOMMENTOR::Alan Meyer
BZCOMMENT::15

I'm working on the coding for this issue.

Some of the tasks include:

During validation:

[ ] Validate no PermaTargId used more than once:
[ ] In the same document
[ ] Across documents

[ ] Validate all PermaTargIds that were in the document before
editing began are still there during validation, i.e., no
PermaTargIds have disappeared. PermaTargId may appear with
the suffix "-DELETED", see below.

[ ] Validate that no new PermaTargId appears in the document
unless:
[ ] it has the exact value '0', or
[ ] it has the value 'X-DELETED', where X is the exact value
of a PermaTargId that was in the data before editing
began.

During document save:

[ ] Create a new, unique PermaTargId to replace any value of '0'.
Store it in the document and in the link_permatarg table.

[ ] Delete an existing PermaTargId with the suffix "-DELETED"
from the document. Change the row for it in the
link_permatarg table, changing the dt_deleted column from
null to the current datetime (I'm using that instead of an
active_status column.

Other issues?

I don't know which, if any, of the following changes are
required, or if they are required, who should do them:

[ ] Create XMetal macros to insert '0' or 'X-DELETED' attribute
values?
My inclination would be to only do this if we're producing a
lot of PermaTargIds and making mistakes doing it.

[ ] Create a custom schema rule to prevent adding PermaTargIds
to anything other than top level SummarySections?
Again, my inclination is to only do this if we're producing
a lot of PermaTargIds and making mistakes doing it.

BZDATETIME::2012-04-27 11:17:18
BZCOMMENTOR::Volker Englisch
BZCOMMENT::16

(In reply to comment #15)
> [ ] Validate no PermaTargId used more than once:
> [ ] In the same document
> [ ] Across documents

Why are you validating (a) and (b)? Testing (b) should be sufficient, shouldn't it?

> [ ] Create XMetal macros to insert '0' or 'X-DELETED' attribute
> values?
> My inclination would be to only do this if we're producing a
> lot of PermaTargIds and making mistakes doing it.

Inserting a PermaTargId=0 attribute could be done via the templates.

I agree with you that deleting a PermaTargId is probably something used rarely.
By the way, will the case matter for the deletion of a PermaTargId?
Could I write 'X-DELETED' or 'X-deleted' or 'X-Deleted' or 'X-deLeTeD'?

BZDATETIME::2012-04-27 12:09:17
BZCOMMENTOR::Alan Meyer
BZCOMMENT::17

(In reply to comment #16)
> (In reply to comment #15)
> > [ ] Validate no PermaTargId used more than once:
> > [ ] In the same document
> > [ ] Across documents
>
> Why are you validating (a) and (b)? Testing (b) should be sufficient,
> shouldn't it?

The above is an artifact of the design I'm using. I keep a row in a new
table, link_permatarg, for each PermaTargId + CDR Doc ID combination.
Since that only tells me what doc has what PermaTargId, I still need to
check inside the document to be sure that it doesn't exist in two places.

...
> Inserting a PermaTargId=0 attribute could be done via the templates.

I'm not sure what you mean. I don't think we want to create these
attributes automatically. Let's talk about it on Tuesday.

> I agree with you that deleting a PermaTargId is probably something used rarely.
> By the way, will the case matter for the deletion of a PermaTargId?
> Could I write 'X-DELETED' or 'X-deleted' or 'X-Deleted' or 'X-deLeTeD'?

I was going to require the exact form "-DELETED". I don't want this to be
easy to do. The user has to think carefully before performing this rare
action.

BZDATETIME::2012-04-27 12:16:36
BZCOMMENTOR::Volker Englisch
BZCOMMENT::18

(In reply to comment #17)
> > Inserting a PermaTargId=0 attribute could be done via the templates.
>
> I'm not sure what you mean. I don't think we want to create these
> attributes automatically. Let's talk about it on Tuesday.

Forget this remark! I was thinking in terms of the DTD where the PermaTarg starts out as an element but in the XML summary they are attributes of SummarySection (and other) elements.
So, yes, we definitely don't want to add this via a template.

BZDATETIME::2012-05-01 23:15:37
BZCOMMENTOR::Alan Meyer
BZCOMMENT::19

There are a number of ways to optimize the checking of
PermaTargIds in a document. These include:

1. Only check Summary and DrugInformationSummary document types.

2. Only check SummarySection elements within summaries.

3. Only check top level SummarySection elements.

If I leave out all of these optimizations, the code will not
change in any way if we start adding PermaTargs elsewhere.

I don't think the PermaTarg processing will take a huge amount of
time, but, the way I'm writing it, a big protocol document with
thousands of nodes might take a noticeable amount of time. I
won't know for sure unless I try it.

What I propose to do is to apply the first optimization only. I
will call the PermaTargId processing routine only if we're
dealing with a Summary or DrugInformationSummary, and only if
link validation has been requested. That's automatically true
when storing a publishable version, and optionally true if a user
requests it.

Making this check will just take a couple of lines of code and
will be easy to remove or amend if we need to do so.

If we are processing a summary type document, I'll check every
node, in case someday we want to have tables or paragraphs or
subsections, etc. be PermaTarg linkable from outside.

Anyone please speak up if you think there is a better way to do
it.

BZDATETIME::2012-05-17 09:44:26
BZCOMMENTOR::Alan Meyer
BZCOMMENT::20

This was sent out by email last Tuesday (5-15-2012) night when
the server hosting Bugzilla was shut down due to air conditioning
failure in the server room. Here it is for the Bugzilla record.

----------------------------------------------------------------

I think I've got everything working on Mahler for this task.

I had to make some decisions about how the program should behave
in validation when saving a document and when not saving it, just
validating. I did the following:

If a user enters PermaTargId='123-DELETE' and clicks "Validate":

I report that PermaTargId='123' is in the database but not in
the document. It's invalid. That's because '123' isn't in
the document any more, it's now '123-DELETE'.

If a user enters PermaTargId='123-DELETE' and clicks "Save":

I delete that PermaTargId from the XML and report everything
is okay.

To add or delete a PermaTargId, the user must click Save.
PermaTarg processing and validation will be performed whether the
user wants it or not.

I can probably change that behavior, but I don't know if it's a
good idea to do so.

For testing - on Mahler only, it's not installed on Franck or
Bach:

Pick a Summary.

Add one or more PermaTargId='0'

Do they get translated to new numbers?

Add an arbitrary PermaTargId='X' or '999', or whatever.

Does an appropriate error message appear?

Delete an existing PermaTargId=N (PermaTargId='N-DELETE'),
where N is an integer.

If N exists, is the PermaTargId deleted?

If N doesn't exist, is there an error message?

If N used to exist but was already deleted is there an
error message (it should be the same error message as the
last test)?

When this passes QA I'll need to:

[ ] Put the new CdrLink.cpp in Subversion.
[ ] Put the two new filters in Subversion.
[ ] Install the filters on Bach and Franck.
[ ] Rebuild the CdrServer om Bach and Franck
[ ] Ensure that the updated schemas are on Bach and Franck (and
in Subversion.)

In the meantime, I'll go back to EBMS / CiteMS work.

BZDATETIME::2012-05-17 11:48:31
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::21

(In reply to comment #20)

>
> For testing - on Mahler only, it's not installed on Franck or
> Bach:
>
> Pick a Summary.
>
> Add one or more PermaTargId='0'
>
> Do they get translated to new numbers?

Yes.

>
> Add an arbitrary PermaTargId='X' or '999', or whatever.
>
> Does an appropriate error message appear?
>

Yes.

> Delete an existing PermaTargId=N (PermaTargId='N-DELETE'),
> where N is an integer.
>
> If N exists, is the PermaTargId deleted?
>
Yes.
> If N doesn't exist, is there an error message?
>

Yes.

> If N used to exist but was already deleted is there an
> error message (it should be the same error message as the
> last test)?
>

Yes.
>

BZDATETIME::2012-05-17 11:50:25
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::22

I ran into an error two times (attached) but I wasn't sure what I did wrong. I couldn't reproduce it either. I thought I typed the letter “O” instead of a one but I tried to reproduce but didn't get the same (attached) error.

Also, how would you recommend fixing a user error in the following scenario ?

PermaTargId = 18 is already assigned by the system in the Pathology Section of the summary. I accidentally enter "18-DELETE" under the Overview Section of the summary. 18 gets deleted from the database and 18-DELETE is no longer showing under the Overview section of the summary document. I also get a validation error saying that PermaTargId=18 not in the database for this doc() so I delete 18 from under the Pathology Section to fix the error. How do I re-assign 18 back to the Pathology Section of the summary? When I tried this, I kept getting 18 not in the database error.

This seems like an error that may not easily happen but if it happens users may have to add a new PermaTargId='0' to get a new PermaTargID generated and this may prompt changes on the requester’s end as well.

Attachment Permatarg error.doc has been added with description: PermaTarg error

BZDATETIME::2012-05-17 17:54:58
BZCOMMENTOR::Alan Meyer
BZCOMMENT::23

(In reply to comment #22)
> Created attachment 2252 [details]
> PermaTarg error
>
> I ran into an error two times (attached) but I wasn't sure what
> I did wrong. I couldn't reproduce it either. I thought I typed
> the letter "O" instead of a one but I tried to reproduce but
> didn't get the same (attached) error.

I have tried and tried to reproduce this, but without exact
success. I was able to crash the program, though with a
different message, by creating a pathologically long PermaTargId
value consisting of hundreds of characters. So I fixed that. If
the value is longer than 20 characters I truncate it and then go
on to validate the truncated value which will always still
generate a validation error, but not a crash. However I suspect
that's not what William did to create the crash.

I looked in our various logs - debug log, command log, and audit
trail and found the transactions that crashed. This was easy to
do because the crash reports that William captured had the exact
date and time in the images. However, there wasn't enough
information in the logs to tell me what happened. I did learn that
one crash was on a validate and one was on a save, but the
PermaTargId value wasn't recorded and I don't know what part of
the program was working when it crashed.

So I propose to do the following:

Log the action (validate or save), the CDR Doc ID, and each
PermaTargId, on each trip through the PermaTarg processing
code. I'll write and flush these with the time and date to
our file system debug log.

I may also log some other info.

If William or anyone else who tests this crashes the computer
again then he should:

Capture the screen image, exactly as William did.

Attach it in Bugzilla.

I'll compare the crash report with the log file and see if that
enables us to figure it out.

Unless I hear otherwise, I'll do that before I leave today. This
will be on Mahler only. Hopefully we'll see this again, fix it,
and won't need it on Bach

> Also, how would you recommend fixing a user error in the
> following scenario ?
>
> PermaTargId = 18 is already assigned by the system in the
> Pathology Section of the summary. I accidentally enter
> "18-DELETE" under the Overview Section of the summary. 18 gets
> deleted from the database and 18-DELETE is no longer showing
> under the Overview section of the summary document. I also get
> a validation error saying that PermaTargId=18 not in the
> database for this doc() so I delete 18 from under the Pathology
> Section to fix the error. How do I re-assign 18 back to the
> Pathology Section of the summary? When I tried this, I kept
> getting 18 not in the database error.
>
> This seems like an error that may not easily happen but if it
> happens users may have to add a new PermaTargId='0' to get a
> new PermaTargID generated and this may prompt changes on the
> requester’s end as well.

As we discussed in our meeting, this error should not happen in
real life because:

1. Deletions will be rare.

2. Inadvertent deletions should be even rarer. We require the
exact string "...-DELETE" precisely in order to make this
hard to do by accident,

3. The right way to do a deletion is not to add a new value like
"18-DELETE", but to edit the existing value "18" to make it
"18-DELETE".

So I'm hoping this will never happen.

If it does happen anyway, then we can fix it either by finding
the people who link to the document or, maybe better, asking a
programmer to fix the data by hand. The way to do that for
PermaTargId 18 is:

UPDATE link_permatarg SET dt_delete=NULL WHERE targ_id=18

That will reinstate 18.

BZDATETIME::2012-05-17 18:35:03
BZCOMMENTOR::Bob Kline
BZCOMMENT::24

(In reply to comment #23)

> ... the PermaTargId value wasn't recorded

It's not in the command logs?

BZDATETIME::2012-05-17 19:45:16
BZCOMMENTOR::Alan Meyer
BZCOMMENT::25

(In reply to comment #24)
> (In reply to comment #23)
>
> > ... the PermaTargId value wasn't recorded
>
> It's not in the command logs?

My mistake. SQL Server Management Studio didn't show it to me but I wrote a program to get it out.

I see what William did. I'll test it.

BZDATETIME::2012-05-17 20:33:05
BZCOMMENTOR::Alan Meyer
BZCOMMENT::26

This is one of those real problem bugs.

I reproduced the actions that William took that caused the failure. In one he entered "253-DELETE" and in the other he entered "0". There were no problems.

I then did the same on the record that William was using. Still no problems.

Then I modified the record to have the exact same PermaTargIds as it had when William's transaction failed. Still no problem.

I looked for any PermaTargIds that might have been very long - causing the crash that I saw when I tested with the huge string. There weren't any.

My head is sore from beating it against the wall.

I'm giving up for now. I've saved the transactions that failed for William and I may have more ideas later. Maybe I'll get back to this if and when it happens again.

BZDATETIME::2012-05-22 14:08:32
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::27

I was able to reproduce the bug but I think it is probably not related to Perma Targs per se. Permar Targs just happened to be the scapegoat and It appears to be an bug that is happening on Mahler only. One way to reproduce it is to enter an equal to sign "=" in the Perma TargId attribute field, it generates the same error message I attached. Also, if you attempt to delete a whole section of a summary or summary title by adding the deletion markup, the same kind of error message is generated. There are probably additional ways to reproduce the error but those are the two I recorded. However, the error does not happen each time I attempt to delete a section or a section title or by just adding the equal to sign so you may have to try a couple of times in order to see it. I have attached the error message I saw while working on CDR269044 this afternoon just in case you’re interested in investigating further. It happened at about 1:46 and the error message actually contains the timestamp.

Apart from the above, I tested some more and did not come across any other errors.

Attachment cdr error - 269044.doc has been added with description: PermaTarg error

BZDATETIME::2012-05-25 01:08:47
BZCOMMENTOR::Alan Meyer
BZCOMMENT::28

Here's the status of this issue:

I tried to reproduce the failure but couldn't do it. I modified
a record, I tried various combinations of values including '='
characters in PermaTargIds. I did a total of about 50 validation
or save operations.

I found the exact transaction that William submitted and retried
the same record a total of 20 times, but still no luck.

So I did two things:

1. Put logging statements all over the new code.

Each of the relevant routines now has, at a minimum, logging
when the routine is entered and when it leaves. If the
program is crashing in the new code, I'm hoping we'll at
least find a routine that has an entrance but no exit and
I'll know where to concentrate my efforts and what the
PermaTargId was that it was processing.

2. Read every line of the code.

I read every line of the new code looking for anything that
might be suspicious.

I found two things I was doing that could conceivably pose
risks. I'm almost certain that one of them was safe but I
made it absolutly, clearly, positively safe. The other thing
I did was more questionable. I was getting at an XML node by
two different paths and I can conceive of a possible way that
the two paths could diverge - leading to a memory access
error when I referred to one via one variable and the same
one (that maybe wasn't really the same one) through a
different variable.

I think it was unlikely that this caused it, but this is an
error that occurs only sporadically, so maybe this was
involved. I fixed it.

So, William, if you can stand it, I'd like you to do some more
testing. Do it as follows:

Click "Options" in the Cdr Loader box.

Select Mahler as the Update Server.

Enter CDR Port: 2030

I have a newly built CdrServer with the above fixes and all of
the new logging running on that port.

I'm hoping that one of two things will happen:

You won't be able to generate the error, or

If you do, I'll have logging info that will let me figure it
out.

I think I might prefer the second outcome to the first since, as
I know from my own experience, that this bug isn't easy to
reproduce and if you don't generate the error that might not
necessarily mean that the bug is fixed.

If you do hit the bug, do exactly what you did before. Get a
screen capture of the error message box. I have time stamps in
the log file that will enable me to find the relevant logging
information.

Thanks.

BZDATETIME::2012-05-29 13:29:35
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::29

(In reply to comment #28)
> So, William, if you can stand it, I'd like you to do some more
> testing. Do it as follows:
> Click "Options" in the Cdr Loader box.
> Select Mahler as the Update Server.
> Enter CDR Port: 2030
>

I am unable to log into Mahler with the Port changed to 2030. If I change it back to the original 2019, I am able to log in successfully.

BZDATETIME::2012-05-29 13:50:45
BZCOMMENTOR::Alan Meyer
BZCOMMENT::30

(In reply to comment #29)
> (In reply to comment #28)
> > So, William, if you can stand it, I'd like you to do some more
> > testing. Do it as follows:
> > Click "Options" in the Cdr Loader box.
> > Select Mahler as the Update Server.
> > Enter CDR Port: 2030
> >
>
> I am unable to log into Mahler with the Port changed to 2030. If I change it
> back to the original 2019, I am able to log in successfully.

Well, it works for me 🙂

There may be a be a firewall rule at one or other end of your connection that has an opening for port 2019 but not for 2030.

I've taken down the Cdr service and restarted the version of the server with the debug logging. It's running under my own credentials on port 2019.

Hopefully, anyone can use it.

Please let me know if that works.

BZDATETIME::2012-05-29 16:07:54
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::31

The error came up again while working in CDR0000062869. It came up after entering "=" in the PermaTargId field. I had previously tried the same thing in at least another document but the error was not generated in that other document. Also, after I copied the error window and exited the dialog box, I tried reproducing the error in the same document but the error did not come up this time around. It rather gave me the second window in the attached document.

Attachment deepbug.docx has been added with description: PermaTarg error

BZDATETIME::2012-05-29 16:25:09
BZCOMMENTOR::Alan Meyer
BZCOMMENT::32

(In reply to comment #31)
> Created attachment 2259 [details]
> PermaTarg error
>
> The error came up again while working in CDR0000062869.

The logging stopped at a place I wasn't expecting. If what it appears to be telling me is accurate, it happened on one of two lines of code, before most of the work is done. I'll work on that for a while and report back when I've figured out some more.

Thank you very much for your patience with this.

BZDATETIME::2012-05-30 00:38:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::33

I haven't done much with this today. I've been working mainly on EBMS stuff.

I'll put some more thought into it on Thursday. I may start up the server on Mahler under a debugger and try to make the error happen again. I know about where it's happening. If I can't break it myself, I may ask William to try again, but I'll give it a good shot before I call for help.

BZDATETIME::2012-05-31 14:32:25
BZCOMMENTOR::Alan Meyer
BZCOMMENT::34

I've taken down the debug version of the CdrServer and restarted the regular one. I'm going to attempt some debugging on a separate copy of the server for a while.

BZDATETIME::2012-06-14 16:49:35
BZCOMMENTOR::Alan Meyer
BZCOMMENT::35

In an attempt to reproduce the bug I ran batch validation twice against all of the summaries in the database. There were no problems in 1,824 validations.

It may be that the bug is really rare, or it may be that it only occurs when working with data coming in from outside the database - though it's hard to see why that would be since it's the same XML either way.

I'll keep trying to think of something.

BZDATETIME::2012-07-06 01:02:54
BZCOMMENTOR::Alan Meyer
BZCOMMENT::36

I made another attempt today to find any problems in the code. I re-validated all 1067 Summary and DrugInformationSummary documents. I experimented with XMetal introducing various kinds of erroneous data into a Summary and various combinations of adds, deletes and invalid PermaTargIds. But I couldn't cause any errors.

I then read the code from top to bottom. I actually found a minor bug where I produced an error message incorrectly, but it was one that could never have been hit because the error would only have occurred if the database were corrupt, which it is not. However I fixed that and also added or modified some of the comments in the code to try to make them more explanatory. Then I commented out all of the debugging statements but left them in the code so that I could re-activate them if required.

I recompiled the CdrServer and, again, re-validated all 1067 documents. There were still no failures.

I put the new copy of the server into the \cdr\bin directory on Mahler and made it the running server there. All of the code changes to all source files are now committed to version control.

I'm not convinced there are no bugs. Looking in the log files on Mahler I see some that may have come from this program. But I don't know if it was related to this or not. The exception reports are all concentrated in the week from June 15-22. That could be because users were using Mahler heavily those days, or could be because Windows had a problem during that period. The pattern is not altogether unlike what we saw on Bach on June 15 which was resolved by a reboot.

I'll leave the status of this as "Assigned". I'm not ready to say it's resolved-fixed.

BZDATETIME::2012-07-23 20:15:20
BZCOMMENTOR::Alan Meyer
BZCOMMENT::37

I have promoted the new filters to Bach, rebuilt the CdrServer on Bach, and installed it in production.

As a test, I called up CDR340944, an old blocked copy of the Colon Cancer Treatment Summary used for testing. I added a PermaTargId to it, validated, and checked the document and the link_permatarg table in the database. Everything looked good.

I then modified the new PermTargId to an illegal value and saved again. The document now failed validation.

Then I requested the server to delete the PermaTargId that was added - validated and saved. The PermaTargId was deleted from the document and marked as deleted in the database.

I'm hoping that the Exceptions we saw in the past on Mahler were an artifact of conditions in the server OS when testing was done. If someone runs into the exception again. It is safe to simply retry the operation and continue if everything is okay the second time. Please let me know that the exception occurred and anything you remember about what was you were trying to do. Don't worry about a screen capture. There will be a record of the exception in the database. I'll be able to see the exception and examine the document that was being stored.

BZDATETIME::2012-07-23 21:03:05
BZCOMMENTOR::Alan Meyer
BZCOMMENT::38

I have rebuilt the CdrServer on Franck to match the one on Bach and Mahler.

Franck now has the new software. It does NOT have the new filters. The standard UpdateFilter.py script won't install new filters on Franck without some special pre-processing - which we probably don't want and don't need. When the database is next copied from Bach to Franck, the new filters will go with it.

I did install the new link_permatarg table in the database on Franck. The new software expects it to be there for all link validation, so I couldn't wait for the next Franck database refresh to put a table there. So, everything should work the same on Franck as it has in the past but any attempt to actually add a PermaTargId before the next refresh will fail with an error - which I presume is fine for now.

I did test the CdrServer on Franck and it seems to work okay.

On both Bach and Franck I saved the old CdrServer as:

d:\cdr\bin\CdrServer-20120723.exe

If something has gone wrong with either installation, we can restore the old server until I figure it out, e.g.,:

cd \cdr\bin
net stop cdr
rename CdrServer.exe CdrServerBAD.exe
copy CdrServer-20120723.exe CdrServer.exe
net start cdr

I'm marking the issue as resolved-fixed.

BZDATETIME::2012-07-24 14:42:39
BZCOMMENTOR::Volker Englisch
BZCOMMENT::39

(In reply to comment #38)
> Franck now has the new software. It does NOT have the new filters. The
> standard UpdateFilter.py script won't install new filters on Franck without
> some special pre-processing - which we probably don't want and don't need.
> When the database is next copied from Bach to Franck, the new filters will go
> with it.

I'm in favor of refreshing FRANCK anyway since it has been a while since this was done. However, I'd like for Bob to say first if and when this could be done because he had some test data on FRANCK related to the CTRP protocols, if I remember correctly.

As a side note:
The command to move a new filter to FRANCK is
InstallFilter.py

BZDATETIME::2012-07-26 14:40:49
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::40

(In reply to comment #38)

>
> I'm marking the issue as resolved-fixed.

We decided in today's CDR Meeting to close this issue so I am closing it now.