CDR Tickets

BZDATETIME::2009-10-02 00:12:23
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

The filters need some special handling. Unlike other CDR documents, the DocTitle cannot be derived from the contents of the document. So we'll need a --title command-line option for when we create a new filter (or change the title for an existing filter). Also, we don't put the filter into version control until after the first time it's been installed on Bach, since the file name under which it's stored is based on the CDR ID for the filter in the production repository (which is almost certainly different from the ID under which it was created on Mahler). We might want to re-think the sequence of events for creating new filters, in order to make it possible to put the filter under version control earlier in the process. One possibility:

Step 1. Run CreateNewFilter.py --title "Title of new filter – whatever"
This program (which doesn't exist yet) creates a stub filter
document on Bach, then creates a file in the current working
directory with the XML for the document, using the filename
generated from the CDR ID of the newly created document. Now
you have something you can install in version control. The
CreateNewFilter.py script also needs you to give it command-line
parameters for your CDR ID and password.

Step 2. Edit the file, replacing the stub with the real filter code.

Step 3. Install the file in Subversion.

Step 4. Run InstallFilter.py with command-line parameters indicating
the name of the filter file, your CDR user name and password,
and the server you want it installed on (Mahler, in the usual
case). The script (which also doesn't exist yet) gets the
Bach CDR ID for the filter from the file name, then goes to
Bach using that CDR ID to get the filter title, which it uses
to create the CDR document for the filter on Mahler (or Franck,
if you need to install it on Franck before Franck picks it up
from a refresh based on Bach's database).

Step 5. Edit some more, check into Subversion, etc. as appropriate.

Step 6. Run UpdateFilter.py passing the filename and CDR uid and pwd
on the command line (and possibly the name of the server
on which to install the filter, if it's not the server
on which you're invoking the command). The program (to
be written) again extracts the Bach CDR ID for the filter
from the filename, gets the title from Bach, then uses the
title to find the CDR ID for the filter on the server where
you're updating the filter (a step that isn't needed, of
course, if you're Updating the filter on Bach itself),
checks out the document, saves a new version of the filter
with the new version it finds in the file, then checks
the document back in.

We'd also need a utility which would coordinate a name change for the filter, and that would need more thought and analysis than my brain has active cells for this late at night. In fact this whole comment needs thinking about in the morning, when I'm more wide awake. Just food for thought and a starting point for discussion of possible approaches.

BZDATETIME::2009-10-02 16:20:14
BZCOMMENTOR::Volker Englisch
BZCOMMENT::2

(In reply to comment #0)
> Create a publishable version, non-publishable version, or CWD
> only, whichever is requested.

FYI: For filters we never create a publishable version.

BZDATETIME::2009-10-28 14:11:24
BZCOMMENTOR::Bob Kline
BZCOMMENT::3

Alan:

I needed to do some extensive work on Charie's EVS filter to match changes in the structure of what we get back from the API into the NCI thesaurus, so I went ahead and created a straw man for the tool described in step 6 of comment #1 above. I've checked it in as Bin/UpdateFilter.py. Modify it or replace it as you see fit. Volker hasn't yet come up with any pronouncements for where we should put different kinds of utilities going forward, so I just picked a reasonable place figuring we can move it later.

I followed your suggestion to look at the newer optparse module, and decided to go ahead and use that. Let me know if you have any suggestions for better techniques to handle the options. One result of the use of this module is that after reading the documentation's discussion of required arguments versus optional options, I wrote this tool in keeping with the philosophy pushed there, so things that were "required options" in the sibling "CreateNewFilter.py" tool are now positional arguments on the command line.

BZDATETIME::2009-10-29 23:52:43
BZCOMMENTOR::Alan Meyer
BZCOMMENT::4

Progress report:

I'm about halfway through the implementation of InstallFilter.py.
I'm doing tons of checking to be sure we have what we need, that
what we've got matches what is on Bach, that it doesn't already
exist on the server where we're installing it, etc.

When I get it working I'll go back and review the quick scripts
that Bob wrote and add any checking, or whatever, that seems like
it might be desirable.

BZDATETIME::2009-11-03 20:30:22
BZCOMMENTOR::Alan Meyer
BZCOMMENT::5

I'm thinking of making two small additions to cdr.py:

New function:
getTitle(docId, conn=None, server=localhost):

Gets the title for a document as stored on the requested server.
Pass either a connection or a server name, not both.
Raise an exception if the docId is not found.

Modified function:
makeCdrDoc(xml, docType, docId=None, title=None)
"title" is a new parameter. If present, the program will add a
DocTitle element to the CdrDocCtl element of the CdrDoc wrapper.

Typical usage in the new filter progams would be:

docTitle = cdr.getTitle(docId, cdr.PROD_NAME)
docXml = cdr.makeCdrDoc(xml, 'Filter', docId=docId, title=docTitle)

...
cdr.addDoc(...)
or
cdr.repDoc(...)

I found a getDocTitleFromId() function in CdrLongReports.py that
is almost the same as getTitle(), but I didn't see anything else
and it's something that it seems might be useful in cdr.py.

BZDATETIME::2009-11-03 22:00:20
BZCOMMENTOR::Alan Meyer
BZCOMMENT::6

(In reply to comment #5)
> I'm thinking of making two small additions to cdr.py:

Disregard the above. I think I found everything I need
ready to hand.

BZDATETIME::2009-11-06 00:15:30
BZCOMMENTOR::Alan Meyer
BZCOMMENT::7

I've put an InstallFilter.py script into Subversion and have
copied it to \cdr\bin on Mahler.

I want to review the other programs again before declaring
the whole issue as resolved, but InstallFilter.py is ready
for testing.

I tested by the following procedure:

Find an insignificant filter on Mahler.
Get it with GetCdrDoc.
Mark it deleted with DelCdrDoc.
Install it again with InstallFilter.py.

I tried various options, installing without first deleting,
going to an alternate server, putting in invalid data, etc.
Everything appears to work.

BZDATETIME::2009-11-24 16:14:36
BZCOMMENTOR::Alan Meyer
BZCOMMENT::8

Bob has made some small modifications to the programs he wrote
and I modified the interface to the one I wrote in response to
suggestions from Volker.

I believe everything is now under version control.

I'm marking the task resolved-fixed.

BZDATETIME::2010-03-29 16:09:34
BZCOMMENTOR::Volker Englisch
BZCOMMENT::9

Closing issue.